Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Solarwinds Auto Updating the Orion services

We noticed at 3:45AM on 4/10/20, Solarwinds downloaded “SWA -2.4.0.1750 Administration Service installer” and tried to install this service. When the installer tried to start the Services back up the Orion Module Engine did not start and encountered an Error. This caused our System to go down.

Has anyone else encountered Solarwinds Auto Downloading and installing service updates, or is this something new in 2019.4?

Also

Is there any way to prevent Solarwinds from Auto Updating Services like the “Administration Service” so we can prevent any future production System outages.

Find more posts tagged with

Accepted answers

All comments

sum_giais

I too... got foolishly hit by this. 1st support case found that the administration service disappeared on the primary HA engine (causing a failover) -- no RCA. We manually reinstalled and got it back up and running.

2nd support case after 3 months went by, no changes to my environment, saw the exact same thing... Administration service had disappeared on the primary HA engine causing another outage and failing over. Support determined that there was a failure with permissions when it tried updating. I haven't looked at that even to today, it requires 'Logon as a service' permission which it obviously didn't have (at least with whatever account it initiated this autoupdate.) The case revolved more around the permissions issue more than determining an RCA on why this even happened in the first place.

After that.. I was rather displeased. With both a lack of answers and the fact that this appeared to be an out-of-the-box default. Production should NEVER auto-update.

I then went on the hunt for answers, digging through every possible SolarWinds log, settings page, Windows event logs, etc.. You name it lol. After a few hours I found exactly what did it, how, when, why... AND how to disable it.

Check the log 'C:\ProgramData\SolarWinds\Logs\Administration\AdministrationService.log', that's where the proof of SolarWinds doing this auto update is. Basically I believe ActiveDiagnostics checks if it has the latest version and if not, kicks off an update for the administration service.

Go to the advanced configuration page at https://{servername}/Orion/admin/advancedconfiguration/global.aspx. Below are the snippets of the settings you'll need to adjust based off your preference. I disabled only AllowSwaAutoUpdate, but the update is triggered off a daily check (DailyMaintenanceDiagnosticsEnabled) that gets run.

Apologies for the long-winded answer here... I hope this helps.

acurrent

Did you find out which account its using to install this? Event log says 'system' cant find which account its using that would need the log on as a service rights.

sum_giais

Unfortunately not yet. Haven’t had enough resources to spend time on this but definitely will dig in at some point... initiating a check for an update on the my Orion deployment pages will trigger the same update to start also. HA services don’t take this update / check into account and see it as a hard service failure causing unnecessary failovers.

I saw somewhere you can omit services from HA checks. I’ll try and dig that up.

sum_giais

I want to say its the system account doing this update, but I don't have anything specifically telling me that and haven't had time to dig into it. Willing to bet that there's logs of it in the installer folder as I believe these are done via an MSI and it logs it.

Anyway, I did happen to find where the blacklists and whitelists are for services. You could look in "C:\Program Files (x86)\SolarWinds\Orion\HighAvailability\Plugins\Whitelists" and "C:\Program Files (x86)\SolarWinds\Orion\HighAvailability\Plugins\Blacklists" for those. It looks pretty straight forward but my 2 cents would be to reach out to SolarWinds and get an engineering / dev response on how to properly use these. Not sure the impact of messing with this file yet.