cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post
Level 12

Upgrades Gone Wrong

So I'm on day three of what was suposed to be a simple upgrade. I'm taking NPM 12.1 to 12.2 and SAM 6.4 to 6.6 while removing some products where the license has expired or we aren't finding useful at this time. Additionally, I was looking to eval WPM so I added it to the install list. The first problem I ran into was the installer stuck at "Importing Sample Maps". Initially I removed IPAM and tried again with no luck. Then I removed NCM, VNQM, and UDT. This got me past that point but, there were issues with the web site setup. I ended up blowing away the website completely and allowing the Configuration Wizard to rebuild it. That seemed to work. At this point I had a working primary/core. My envinorment has the Primary, 2-2APEs, and one remote APE. I took the provided installer from the Primary and placed them on the APEs. Initally, they complained that my APE licenses, active and due to expire in 3/2019 according to my Customer Portal, where not active. I made multiple failed attempts through the Primary to sync licenses via the portal provided License Manager. Following a link about reseting licenses, I ended up downoading/installing a standalone license manager tool to the APE. This didn't work either. I tried the same on the Primary which also didn't work. Support tells me that this corrupted me so I guess don't do that!. Support walked me through the deactivation of all my product licenses. I'm told this was neccessary because the sync would not work with the EVAL status that WPM was in. For purposes of getting through this I decided to uninstall WPM. Still, none of my licensed product would reactivate. Support got some license expert on thier end who did some license reseting on thier end and using the portal provided License Manager, I was able to reactivate NPM and SAM but, had to manually add the APEs. There was no sign of WPM so I figured we were good. By my way of thinking we have a working Primary with NPM and SAM. I went to each of the APES and placed the provided installer on them. When running the installer it would fail at a point where it was trying to download a file "UpgradeFix.exe" from the main server. Looking at the installer log file, this was a file supposedly at downloads.solarwinds.com. From each of the APEs, I had no issues using IE to go to that wesite and downloading the file. So I have to assume the its actually trying to get the file from the Primary but, that file does not exist anywhere on it, nor could I figure our where I could put the file where the APE upgrade installer would find it. Support tells me that this file is related to WPM and there must still be remanents of WPM on my Primary. So they send me a "Burn" script. Basically this script wipes the server of any and all Solarwinds products completely. So I run this on the working Primary. After a considerable amount of time it completes. You are admentaly warned not to interrupt the process. I reboot after the burn and reinstall selecting only NPM and SAM, fight through some website configuration issues ending up with a working Primary except for the annoying IE security prompt when opening the UI, which I still have but, will resolve later...I hope. So again, back to the APEs, using the SW portal on an APE, I navigate to the Polling Engines setting and download an installer. When I run it, it pases all the checks but, there it is again, WPM. The installer intends to place WPM on the APE even though there is no WPM. I can't get rid of this thing. I have combed through the Primary removing any and all mentions of WPM from the registry. I have no idea where else to look or what else to do. I've listed this as a question in hopes maybe someone else has had a similar issue and was able to resovle it. Support will get back to me around 1pm my time today after reviewing all the diagnotic information I have provided but, I thought I reach out here to see if I could find some more ideas.

2018-05-22 18:36:16,879 [17] DEBUG (null) SolarWinds.Administration.SystemInfoCollector.Detectors.InstalledProductsDetector.InstalledModulesDetector - Modules detected in msi library: ACTIVEDIAGNOSTICS:1.8.0.287, COLLECTOR:2.13.0.292, CORE:2017.3.5320.1819, CORTEX:1.0.0.4329, DPAIM:11.1.0.1191, DPI:2.4.0.182, HA:2017.3.1.851, JOBENGINE:2.11.0.328, MIBS:1.0.43.0, NETPATH:1.1.1.4015, NETWORKATLAS:1.17.0.177, NPM:12.2.5300.1871, ORIONHA:2017.3.1.3303, ORIONIMPROVEMENT:2.8.0.196, RABBITMQ:1.1.40.0, RECOMMENDATIONS:1.4.0.4855, SAM:6.6.0.3735, SQLCE35:3.5.8080.0, SQLNATIVECLIENT2008:10.0.1600.22, SQLSMO2012:11.0.2100.60, SWA:1.3.0.339, SWIS:2017.3.0.710, VIM:8.2.0.7071, WINPCAPSILENT:4.1.3.3024, WPM:2.2.1.0

This is from the AdminstrativeService log on the Primary, which I beleive the APE installer is using to understand what it shold put on the APE.

25 Replies
Level 12

I was having the same trouble. Migrating servers I was left with a mixed 2012 and 2016 environment.

UNEXPECTED WEBSITE ERROR

The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.solarwinds.com/2008/Core:GetNodeWithOptionsResult. The InnerException message was ''EndElement' 'Comparer' from namespace '' is not expected. Expecting element '_compareInfo'.'.  Please see InnerException for more details.

I was also getting this error:

  • pastedImage_7.png

               Website configuration failed:

                •  Access to the path 'C:\Program Files (x86)\SolarWinds\Orion\AgentManagement\Microsoft.AspNet.SignalR.Core.dll' is denied.

I ended up running through these steps.

1. Installed Orion Platform 2018.2 HF6 - includes NPM v12.3 HF1  installer from September 28, 2018.

2. Installed .Net 4.7.2 (KB4054530) on all servers.

3. Cleared all temp files.

4. Ran C:\Program Files (x86)\SolarWinds\Orion\OrionPermissionChecker.exe and fixed some failures.

Everything seems to be running okay now.

Level 9

We are experiencing the same problem. We have two polling servers. We upgraded to the newest version of Solarwinds. During the upgrade, our primary was patched over the weekend, which started to cause problems. I did everything...ensured same version (latest) of .NET is on both servers, went through all the services...etc...I see two Microsoft patches that are installed on primary but not on secondary. The problem is, they are un-installable, per Microsoft. Of course our server team is totally baffled by this that a software would have such requirement. What if we have additional role installed and updates will be pushed to that server?

Here are the patches that are installed on primary but not on the secondary:

KB4049065

KB4089510

I spent so much time on this already. Not to mention, we had a Solarwinds partner vendor do the upgrade for us so we paid pretty penny for this as well. I am spinning in circles here. This is just insane.

0 Kudos

Let me guess, if you try to install the patches on the secondary, its says the patches are not applicable?
These are the patches I have on my core and my secondaries

KB3186568
KB3192137
KB3211320
KB4013418
KB4023834
KB4035631
KB4091664
KB4132216
KB4457127

KB4457131

0 Kudos

Yup! That's exactly it!

We just fixed it. Well...until next patching I guess. This is crazy.

We uninstalled all patches that were installed on both servers from 8/31/2018 (That was our last patching for these boxes) I can now view the details for nodes. Now for a kicker......you ready? Both polling engines are on a different patch levels. lol

0 Kudos

The issue is back.

I am uninstalling patches now.

0 Kudos
Level 9

having the same issue, .net are the same versions across the main poller and the additional. Cant edit nodes. Updates look consistant over the two servers as well. help!

0 Kudos

Some have found that removing this KB4103720 helped.

0 Kudos

I am getting the same error when trying to add or edit a device to NPM. However, in my case, KB4103720 is not present on my servers.

If this is a KB issue, why did the problem surfaced only after the latest upgrade?  This seems like a bug in the SolarWinds code somewhere.

0 Kudos

We had the same issue on SAM with add and edit node, and this was also related to .NET Windows patches.
We have polling engines with both 2012 and 2016. The 2012 servers got their monthly patches last week causing this issue.Removing of the following KB's solved the problem for us: KB4338605, KB4338424 and KB4338419.

If you don't have the same patches, or removing them don't work, you need to investigate if there are other patches related to .NET, and remove one by one until it works again.

Unexpected website error when editing nodes - SolarWinds Worldwide, LLC. Help and Support

In my case, this problem is happening only with our APE's, not with our core/main SolarWinds server.  Still, I checked and confirmed that our APE's and our core SW server do not have any of the KB's that are posted in this page.  Also, I have about a dozen SolarWinds servers which are running MS Windows 2012, Windows 2012 R2 or Windows 2016.   I almost file like we're on a scavenger hunt. 

0 Kudos

Have you tried the last portion of the KB that talks about 2012 and 2016 servers?

Note: If you are using Windows 2012 or Windows 2016 on your Orion servers and the procedure above does not fix the issue, the next step would be the update Net Framework to version 4.7.2

3. Ensure all relevant rollups are installed for your OS and version of .NET:

0 Kudos

We did not upgrade to the latest version and experienced this issue. The timing of both the .net Microsoft Updates and the latest version of the Orion suite software was just coincidental.

0 Kudos
Level 9

Solution 2: Uninstall kb4284880

I have been struggling with this for a week. I have one 2016 APE, 1 2012 R2 primary, 3 2012 R2 APE's, and 1 2012 R2 AWS. KB4103720 was replaced by the June rollup, KB4284880, after uninstalling this one from the problem server (Windows 2016), I was immediately able to edit nodes again. No configuration wizard necessary.

Level 12

It is really odd that in some cases installing KB4103720 solves the problem, while in others uninstalling KB4103720 solves the problem.

Level 8

Hi,

I had the same error when I tried to review the "events" of one node in the Additional Web Server.

After seeing your comments I reviewed my servers (Orion, 2 APE and 2 AWS), I saw that in one AWS the KB4103720 patch was not installed. After installing it, the error was resolved and everything worked correctly .

Thanks a lot for your comments, it was very helpful to see what could be happening.... .

Level 12

To close the loop on this, I ended up restore the Db from backup, restoring the primary from snapshot, uninstalling all SolarWinds software and burning the pollers. I reran the installer choosing only the products I wanted to upgrade and then using the provided installer for the pollers. This got me running again with some residual issues. Could not edit user groups. I fixed this with a Db update where groups where looking for limitation IDs that did not exist.

I'm still having Edit Node issues.

Unexpected Website Error

The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.solarwinds.com/2008/Core:GetNodeWithOptionsResult. The InnerException message was ''EndElement' 'Comparer' from namespace '' is not expected. Expecting element '_compareInfo'.'.  Please see InnerException for more details.

Have you made any more progress on this? I have the exact same issue and support is basically ignoring me and the DBA and I have been beating our head on the wall for a couple days. I'm about 3 days away from just cutting it all over to Zabbix/Rancid and calling it a day.

0 Kudos

Yes, this was finally resolved by insuring all of my pollers had the same MS patch levels and .Net 4.7. I was at .Net 4.6.2 so I updated that first, nut it had no effect. I had one APE missing two patches compared to my other three and the Primary was missing one compared to the other three. I'm running on MS Server 2016 w/MSSQL SQL Server 2016 (SP1)

KB3186568

KB3192137

KB3211320

KB4013418

KB4023834

KB4035631<--May not be applicable

KB4132216

KB4103720

I had the same issue, not able to edit nodes monitored from one of the polling engines.

We have 3 polling engines with Win2016, and 3 with 2012R2.

The polling engine having the problem had two MS patches installed which the other ones didn't have.

By uninstalling KB4103720, everything worked fine again.

Thanks for the tip!!

Thanks for this info. I finally just sat on hold until I got someone. My patch and .Net version levels seem to match across all servers.