cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post
Level 13

Solarwinds is now horribly unstable.

Jump to solution

Last week I did an upgrade to all of the latest versions of solarwinds and now every two days it just stops working to a point where I have to reboot the server.

Alerts still work, but you cant view anything on the main front page.  I have done a config on the web front end several times and it still doesn't work.  I have included an any/any exception so that nothing will be blocked, though it never had this problem prior to the upgrade.

Does anyone else have this issue?  Or something similar?

pastedImage_0.png

Tags (1)
1 Solution
Level 10

Alright, we’ve gotten enough feedback to officially declare the fix for this issue is to change the TransferMode property in the Information Service settings from Streamed to Buffered.  To recap, you can do this in the centralized settings section of your website:

1) Go to http://YOURHOSTNAME/orion/Admin/AdvancedConfiguration/Global.aspx
2) Find "SolarWinds.Orion.InformationServiceClient" and change TransferMode to "Buffered"
3) To apply changes please restart Orion services on all SolarWinds Servers.


You are not losing anything by making this change – Streamed was introduced in 12.3/2018.2 to help manage SWIS Memory consumption.  Moving to buffered is basically how SWIS has worked in every other version. 

 
We will soon be introducing an official fix that will make this change. We’ll also be revisiting Streamed mode on our end to see if we can make it work without causing port exhaustion, but that will be at some point down the line. 

View solution in original post

197 Replies
Level 13

That very well might be the case, but it appears as if there is an issue with user experiencing a loss in services.

Level 9

Concerning indeed, I was pushing for upgrading myself. Hope these issues get worked out.

0 Kudos
Level 9

Same here, was suppose to upgrade aswell, but might delay.  I am going to upgrade SQL 2012 to 2016 before I upgrade all licenses.

Orion installer did recommend SQL 2016, this is the output from SW setup Wizard system check results:

1.   "Supported SQL Servers (Orion Database health)

DISCOVERED: 6/11/2018 2:18:29 PM

DESCRIPTION: Support for SQL Server 2012 is deprecated as of this release.

RESOLUTION: Support for SQL 2012 is deprecated as of this release. Future releases will no longer support SQL 2012.

SolarWinds recommends that you upgrade directly to SQL 2016 at your earliest convenience."

2. "Operating system supported (Local System)

DISCOVERED: 6/11/2018 2:18:28 PM

DESCRIPTION: Support for operating system Microsoft Windows Server 2012 R2 Standard version 6.3.9600 is deprecated as of this release.

RESOLUTION: Installation on Microsoft Windows Server 2012 R2 Standard 6.3.9600 is deprecated as of this release. Future SolarWinds Orion Platform product releases will no longer support installation on Microsoft Windows Server 2012 R2 Standard 6.3.9600. However, monitoring Microsoft Windows Server 2012 R2 Standard 6.3.9600 will remain supported.

SolarWinds recommends that you upgrade to Windows Server 2016 at your earliest convenience.

KNOWLEDGE BASE: http://www.solarwinds.com/documentation/kbLoader.aspx?lang=en-US&kb=MT109004"

Only reason I haven't is because I have another unrelated issue that needs fixing with SW's and the above (first result).

0 Kudos
Product Manager
Product Manager

2012 is still viable for you to use with your NPM 12.3 installation. We do of course recommend 2016, because that will save you time in your future upgrade, as Microsoft has already announced end of support in October. However, this is a courtesy notice to help you plan for the overall state of your environment.

Level 9

Extended Microsoft support of Windows 2012 lasts until 10/10/2023 (https://support.microsoft.com/en-us/lifecycle/search?alpha=windows%20server%202012​).  I would estimate that nearly every single corporate environment will have a high percentage of Windows 2012 servers for quite a while.

While I appreciate that SolarWinds is introducing new updates and improved code, "forcing" customers to upgrade to the latest Microsoft O/S and latest SQL version is not ideal.  I don't have a single SQL 2016 or a single Windows Server 2016 in my environment.  I would also venture to say that there are others like me.  Please reconsider supporting Windows Server 2012.

0 Kudos
Product Manager
Product Manager

thwackerinva  wrote:

Extended Microsoft support of Windows 2012 lasts until 10/10/2023 (https://support.microsoft.com/en-us/lifecycle/search?alpha=windows%20server%202012).  I would estimate that nearly every single corporate environment will have a high percentage of Windows 2012 servers for quite a while.

While I appreciate that SolarWinds is introducing new updates and improved code, "forcing" customers to upgrade to the latest Microsoft O/S and latest SQL version is not ideal.  I don't have a single SQL 2016 or a single Windows Server 2016 in my environment.  I would also venture to say that there are others like me.  Please reconsider supporting Windows Server 2012.

That's absolutely true that extended support exists until 2023, but customers end up having to pay a large amount of additional money to take advantage of that support. The question then becomes, does it become more financially sound to pay out the nose for support or to go to the latest version of a system that will have the latest security fixes and features to ensure that your IT department stays competitive in the modern environment? Each customer has a different answer here. In addition, in no way are we forcing the customer to go to 2016 for Orion. Orion SQL 2014 will still be a standard for the platform until Microsoft moves to deprecate, but in the interest of making your lives easier, we recommend that 2016 be the platform of choice for you. That will save you future migrations and upgrades. NTA 4.4 is the exception because they have a specific requirements for the columnstore technology, available in SQL 2016, to be able to deliver the performance results that the customer base demands. We know that there are other customers like yourself where going to SQL 2016 isn't the right fit at this particular moment. That's exactly why we're provided this preflight check to force the question far in advance for admins who are planning for the future of their environment. We are also not enforcing that customers upgrade directly to 4.4 if they indicate they aren't ready with a 2016 instance. All of these items are built in mind for customers like yourself, and by speaking up we'll continue to evaluate and see how we can smooth out this obstacle.

Level 9

Well, a customer doesn't generally have to pay Microsoft for extended support unless they're running some kind of custom software or if they have special requirements.  Windows 2008 R2 has been in extended support for 3.5 years, and security patches are still released for it to "regular" customers every month.  We still have a number of Windows 2008 servers, and feel very comfortable with them.  We will obviously upgrade them before security patches are no longer released for that platform, but we don't (and have never had to) pay for extended support.

And I don't foresee us paying for Windows 2012 R2 extended support either, if it works the same way as Windows 2008 R2.  So, for our company, it's not necessary about the financials.  It's about familiarity and stability, especially for our critical systems.

I understand Solarwinds not being able to support every single Windows Server platform.  It makes the development, releases, and support that much more difficult.  But to deprecate support for what I believe is still the most prevalent operating system (Windows 2012) at this time doesn't make sense to me.  My two cents.

Thanks,

Chris

0 Kudos
Product Manager
Product Manager

SolarWinds will continue supporting versions of Orion product modules running on Windows Server 2008 and 2012 for quite some time to come. This depreciation notice is notifying you of future versions.

The Failover Engine product was ceased support with less than 1 year notice.

0 Kudos
Product Manager
Product Manager

Failover Engine was an exception as it was licensed by a 3rd party which went bankrupt. This regrettably, was beyond our control.

shaun_9999

I had similar issues on previous upgrade, in regards to the "null value" error message. In my case, it seemed like the custom properties that were flagged as required were not actually enforced until post upgrade. Going through all my custom properties, filling in values for all null fields, seemed to help a bunch. I eventually ended up just disabling the required option, making all custom properties optional. (at least technically, in the settings... as far as my users go, they are still required.)

0 Kudos

wluther​ I just went thru and set all mine to NOT REQUIRED. Here is hoping for a few stable days with no NULL value errors.

0 Kudos

Unfortunately wluther that did not appear to be the issue for me. I got the same error this morning and had to reboot. serena​ I did open a support case, number 00122061, and collected Orion Diagnostics before I did a reboot. Hopefully to capture the system state while it was issuing the error. Once I reboot the system operates normally.

0 Kudos
Level 13

How do you do that?

Level 9

I just checked mine and they have always been set to NOT REQUIRED and i still get the NULL value errors

0 Kudos

I have been having similar issues with the exact same error sine the upgrade. "Value cannot be NULL." In addition to several other odd or quirky things. If I reboot the system it seems to work fine for a while. I do not have a ticket open yet with SW. I simply have not had time.

0 Kudos
Level 17

Had a system that was having this issue, it was due to hung services mostly. Also did experience view issues with the VMAN summary while getting that error.

SW support was able to get a quick resolution.

0 Kudos

Can you elaborate on the symptoms a bit more, and the resolution?

0 Kudos
Level 17

We were getting the same error as above (in the thread post), Value can not be null.

I was looking into this and found the support page for the .net fix that justthwackit​ posted above;

1. stop IIS

2. Stop all Solarwinds Services on that device

3. delete the following folders:

     - "C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\root" delete the folder (in my case named "53bc9024") in there

     - "C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files\root" delete the folder (in my case named "e22c2559") in there

4. Start IIS and Solarwinds services

And in checking with our man derik.pfeffer​ that normally takes care of that specific instance, I was informed that he just got done with Support and they fixed the hung services.

    ​​

Specifically on the views, VMAN summary was having an issue showing as well.

Initial/Main Issue and Fix :

"Admin service dropping along with module service. When we stopped all services, there where about 10 or so processes still running that were business layer

Manually killed processes and then restarted all stopped Orion services"

so I never ran the .net fix... the system has been stable since then.

Level 12

Thanks, what was the quick solution?

0 Kudos
Level 17

I had to ask the engineer that worked directly with SW support on that box; thanks derik.pfeffer​!

"Admin service dropping along with module service. When we stopped all services, there where about 10 or so processes still running that were business layer

Manually killed processes and then restarted all stopped Orion services"