Anyone deployed this hot fix yet? Given the success rate of previous iterations, sound off with progress on implementation.
Thanks
James
Yes, updated about 10 customer sites so far?
Yes, updated, and it is working for 98% of our users, but something has changed. We're using Azure for authentication, and for a few users, they can't login now. The system is throwing an error about CSRF for them, but its only for the account, and it travels between devices. The user can login under an alternative account on the same device, using the same method, but their own account is not authenticating. I have an open ticket already. With debug logs on, the error is: [https-jsse-nio-443-exec-15] ERROR com.macsdesign.whd.ui.Application - UNHANDLED EXCEPTION: org.springframework.security.web.csrf.MissingCsrfTokenException: Could not verify the provided CSRF token because no token was found to compare.
How are users logging out, do the do this probably or just close the Tab because it tries to auto-login again? Also were you on HF2 before?
Closing tabs generally - they have no need to log out, and never have needed to. Yes we were on HF2 before and had no issues with authentication. HF3 has introduced this problem.
Interesting, those sorts of errors in your log file have been common since 12.8.1 release, do your users get entries within the whd-session.log ? I am more inclined to think this is cookie error rather than CSRF. Do you have a Load Balancer/WAF installed in your infrastructure where a WHD session might hit one Load Balancer then another?
Looks like the ability to delete checklists is broken in this hot fix as well. You can create and modify, but if you try to delete the checklist you get an error after the confirmation dialog:
Validated this is two different environments. Will be opening a case after I go through the diags/screenshots hoops.
UPDATE 1: You can seemingly delete items from the checklist and save the checklist, but when you save the ticket the checklist items return.
UPDATE 2: Case #01781660
Reproduced in my Lab system as well
We are having issues with tickets "missing" in the "My Tickets" tab. The count number in the tab is accurate, but the number of objects displayed in the window does not match the number of open tickets. If you d a search for the assigned tech's tickets all of the teickets do show. My initial thought was it was different statuses but that does not seem to be the case as I can see all of our different types in the tickets that do show. I am working on some screenshots to demonstrate my finding.
I have that reported as well at one site, I am waiting for clarification of Tech Permission Group membership and whether 'Limit to Assigned Tech Groups' is in use.
We are not using limited tech groups in my instance.
If you know the ticket numbers that are missing. Can you look in the DB to see what the Status Type ID is set to for that ticket? If it's NULL then It might be a bug, which has still not been fixed!You can also use the GUI to create a query will all display all tickets that have a NULL status type set..These are my Status Types, so change it to exclude all your own Status Types
It looks like it may actually be that it is not showing any request types that do not have a native assigned tech group. EDIT: Nevermind, looks like this was disproved by example in my environment.
Yes, all seems to be working fine for us.
I just thought the versioning/build nos inconsistencies were/are strange ...
Felt like I was crossing my fingers at times through the process. Feels strange to be on a lower build number with a later release. Might have been better to badge this release as v12.8.4 and issue a new incremental build number??? But I'm not a dev, just an admin.
This is strange, you would think they would always go higher. Should it not the display the version within Setup > General > System Information not show Build #12.8.3.2230-HF3 i.e. show HF3 just like it did for HF2 (12.8.3.2944-HF2).?
Odd to me as well since this "hot fix" was a full installer which also required a DB update. For sure this should have been at least a dot revision as backing out of this would be non-trivial. At least the typical hot fixes are easily reversible.
This also appears to affect the Flagged tickets view as well.
It seems that it has been more than 2 years since a release of WHD didn't have some core component broken. We are between a rock and a hard place with regard to patching for critical CVEs and hours chasing things that should have been caught in development cycles.
Given there was a database update associated with this "patch", I wonder if the app can be safely rolled back to HF2, but then again things are broken in that release that aren't broken in this one so would just be trading one issue for another.
Happy Friday...
I took a snapshot of the app server before I implemented HF 3 and I believe it updated the DB in some fashion. Realizing it broke my authentication to sign into the help desk I reverted the snapshot on the app server and everything is normal and working fine without touching the DB server. Actually did this twice to make sure I didn't miss something on the upgrade.
good to know! Not to that point yet, but looks like there may be some issue again with special character parsing like '<' and '>'.
It certainly "updated" my SQL database. Thankfully I do have a full DB backup and VM snapshot available if we need it.
I will add that the password reset URL issue from HF2 is resolved in this release, for what that is worth.
Hi PeterB,
Noted , we are working on streamlining the versioning Nos..thanks
Hi Rels,
Did you try clearing the cache?
Another bug to report and confirmed in several environments. Using the combo of characters:
<MD
In any ticket text field (Subject, Request Detail, Notes) will cause a session crash. There could be other combos, but this is what I boiled my individual case down to. The original content was an email address wrapped in the '<>', but that is the part that causes the crash. It is case insensitive and has to be structured like it is, a space after the <, (i.e., '< MD') is processed as expected.
Confirmed in 3 environments.
Good times.
UPDATE 1: Case #01785881
Of course.
I also have a similar situation on an Outbound Closure E-Mail being sent to a third party "How well did we do" system which contains a complex URL which used to work :-(
Super frustrating.
@jplurker did you open a case on this?
Received notice from support that this "may be due to a bug in the code". At least it is a response.
I have an open case for a seperate issue that this issue has been reported on. I do not have a seperate case for this issue in particular.
This is the response I recieved this monring.
We appreciate the update and feedback. Yes, your case has been added in our tracker for monitoring as well.We have informed our Engineering team about the thwack page.
I also linked them this forum post as I wasn't seeing any SW employee posts. I assume thats why we see @Avadhut.Deshpande in here now
Add 'cmd' to character combos that cause session crash. Seems like some erroneous regex content filtering at work here.
Interesting on this issue is that whatever content checking is occurring on field input does no occur when the note is added via email response. But, if you try and edit that note and save with that content still in there it will crash the session.
'java.' is another...
We just deployed this today after previous attempts to upgrade from 12.8.1 failed. The update failed initially, and we were experiencing the same issue with the UI not displayed correctly, user sessions not sticking, contact your administrator errors over and over, etc. After we added a setting into the WHD config file that enabled IP binding enforcement, it resolved all of the issues that we were experiencing.
However, we have noticed that when we try and cc or bcc email addresses on tickets and click "Save & E-Mail" that WHD does not send an email to the cc or bcc email addresses. It will send to the client, tech, etc.
Is anyone else experiencing this after upgrading to 12.8.3 HF3?
Here is more info on the IP binding enforcement that we enabled, case numbers 01550538, 01673485: documentation.solarwinds.com/.../whd_12-8-3-hotfix-3_release_notes.htm
Just tried a CC & BCC and works fine for me. Do you have sufficient "Scheduled E-Mail Delivery Threshold" configured within Setup > E-Mail > Options? Also are you copying back a bespoke wrapper_template.conf from {webhelpdesk}/bin/wrapper/conf each upgrade so losing the entry: wrapper.java.additional.19=-Dwhd.12.7.8.not.send.mail.individually=false (assuming this is a Windows Based Installation - Linux will be different)?
Have you discovered/observed the issue where the "My Tickets" tab displays a number, but the actual number of tickets that users see is less?
Not yet ourselves, but the support person we worked with did mention that might be an issue we experience with 12.8.3 HF3.
YES!
Yes, flagged tickets not showing up under the Flagged Tickets tab, but it's not affecting all techs. I've founds it's affecting all NEWLY CREATED techs, and so far, only one existing tech (from before install).
Have you had any luck fixing this issue?
Any update yet on this? About to put in my own ticket regarding the same issue but with Google SAML getting the same CSRF token errors.
Had a new issue today where a tech who is only in 1 tech group can see every ticket ever created in the past that wasn't auto assigned to a tech group at the time in his "My Tickets" tab, even tickets with a closed status. ¯\_(ツ)_/¯
We have seen that! Can't wait for 12.8.4 !!
Same. Thankfully a relog and cache clear fixed the issue for the tech.
Question for solarwinds team:Have you ever tried to test your releases before publishing them?
@Avadhut.Deshpande , do you all have any ETA for the new patch? We would all really like to get this resolved. If not a patch, do you have any temporary fixes you have discovered along the way?
I created Advanced Ticket Searches that select all Tickets which are not Closed or Cancelled along with Request Types which are part of the Tech Group and save as a search for each Tech Group. The Techs can then recall these via the Query drop-downs.
I created a REPORT as a work around that was programmed like this:
You can then send out a link to that report to everyone that they can bookmark in their browser and run again and again.
To be fair, it's not like they got hacked. Or some sort of hard-coded password issue. Oh, wait...now I understand.Critical hardcoded SolarWinds credential now exploited in the wild
:-)
12.8.4 is out, will use this new thread to continue this convo: thwack.solarwinds.com/.../whd-12-8-4-is-out-start-your-testing
Cisco ASR Devices.pollerCisco ASR Devices
SWQL is built on the framework of SQL and as such supports most of the standard clauses as part of a query. A very simple example query is: SELECT Caption, IPAddress, Vendor, ResponseTime FROM Orion.Nodes Dissecting this query is relatively straightforward: show some fields (Caption, IP address, Vendor, and Response Time)…
Active SSL Tunnels-ASA.UnDPThis poller *should* display the number of Active SSL VPN (Anyconnect) Tunnels currently connected to your box. However, I tried it with my 5520 running 8.0.4 and for some reason I get a OID Not Supported. Here's my post in the forum: *EDIT* This is confirmed working with at least Interm release…
HP ProCurve Switches Hardening check.xmlGeneral hardening for HP switches
