Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Post 2020.2.6 upgrade Agent issue

I upgraded our platform to the 2020.2.6 hotfix over the weekend. Reviewing one of our servers, it looks like a false monitoring of the CPU. we got a lot of alerts that it was high CPU, but looking at the server directly (remoted in and looked at the perfmon) it was fine.

For the heck of it, I restarted the agent service for that server.

Then, reviewing the SolarWinds perfmon, I noticed that the CPU levels returned to a reasonable amount after the agent restart (see image)

Should it be a standard procedure to restart all the agents in our environment after an upgrade?

Or did i miss something in the release notes?

Find more posts tagged with

Agent

upgrade

Accepted answers

aLTeReGo

If the alert triggered, which should be easily confirmable by reviewing the Orion event log, then the Agent, and the poller its associated with did their job. The fact that the alert triggered, also means that the alerting service on the main poller completed its job of triggering on the alert. Where the breakdown appears to have occurred is within the alert trigger actions.

To determine why the alert action did not fire successfully, you will likely need to review the appropriate logs. Detailed information on troubleshooting Orion's alerting service can be found at the link below.

https://support.solarwinds.com/SuccessCenter/s/article/Orion-Platform-Troubleshooting-Alerts?language=en_US

All comments

aLTeReGo

No, this would not be considered normal operating procedure to restart Agents after an upgrade. However, Agents and/or their plugins are often upgraded also as part of the normal Orion upgrade process. I cannot imagine what would cause what you observed above to occur, but it definitely appears to be related to the upgrade somehow. A few questions.

Was this an isolated issue or Is this reproducible across many/all of your agents or a specific subset of agents, such as all Linux Agents?
If not did you happen to grab diagnostics from the Agent and the polling engine the Agent is associated with?
If this is reproducible, have you opened a support case so we can assist with determining the cause?
1. For any that are still experiencing this issue, I would start by comparing the Agent and plugin versions of the affected machines with those that are not exhibiting this behavior. This can be done easily from within Settings under Manage Agents

ednelgmot

@aLTeReGo thanks for the info. I need to do more research on it. I'm hoping it's isolated, but I didn't see this in other 2020.2.6 posts so I wasn't sure if I hit on something new.

So far, it's only been that server that was noted to have a weird CPU alert. The only other oddity that was noted was that some reboots for a group of servers didn't get delivered. They look like they triggered in the platform, but no email was received.

If i find this is something bigger than expected, i'll def put in a ticket.

aLTeReGo

The Agent should never initiate a reboot. Rather, it should appear in the Agent Management as awaiting a reboot. The next time a reboot occurs for that machine, the Agent upgrade process should complete.

The Agent should only require a reboot if an Agent binary or associated dependency (usually .NET) is locked or in use by another process. This prevents the upgrade from completing until the machine is rebooted to release the lock on the file/s.

ednelgmot

Right. I could have been more specific in my response... I meant that the alert notification for the regularly scheduled reboots on those servers were not emailed out. I did the update Sunday evening and the servers were already scheduled to reboot on Monday morning. the notification didn't get received even though it showed the alert triggered.

For the server that was showing High CPU, i went into the agent admin screen and selected the one - restart agent service. That's when I noticed that the CPU went down on that server and stayed low.

Digging through the agent diagnostics today to see what I can find.

Thanks!