Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Alert from a script running a "Show" command

Is there a way to run a Job or similar on a schedule that will pull a Show command from a Cisco device and fire off an alert if the value is above a value?

We got a HA pair of Cisco Catalyst 9800-80s wireless controllers that have been beyond a nightmare to deploy. We as well as many other owners of the 9800s have been tripping on Bug after Bug, all the way to finding new ones. SolarWinds been great on setting up some custom proactive alerts through SysLog and physical as well as emotional state of the WLCs.

Long story short, we are faced with a new issue with a run away WNCD process. (I am not the wireless guy, just the DC, Core routing and switching wrangler/engineer asked to setup an alert).

Here is what I am trying to do:

1. On a schedule, run "show processes cpu platform sorted | include wncd"

2. Get this output:

ciscowlc9800#show processes cpu platform sorted | include wncd
27225 26489 11% 8% 8% S 440964 wncd_2
26874 26019 6% 8% 8% S 483104 wncd_1
26402 25441 1% 1% 1% S 365224 wncd_0
28352 27962 0% 0% 0% S 293096 wncd_7
28148 27733 0% 0% 0% S 290808 wncd_6
27920 27504 0% 0% 0% S 292968 wncd_5
27692 27275 0% 0% 0% S 292624 wncd_4
27466 26963 0% 0% 0% S 500068 wncd_3

3. If any of the percentages is say above 40%, fire off an email alert.

If you have an even better idea, please let me know. I know there are may other ways to do this.

The physical CPU does not climb when the WNCD process goes crazy. I searched the OIDs through an SNMP walk and could not find anything that I could use from there, even through Google-Foo (unless you know more that I do).

Find more posts tagged with

Accepted answers

All comments

yaquaholic

Hi rjrothwell,

We had a similar issue, where one of many CPUs would max out, and caused all sorts of issues.
But the average CPU polling did not reflect this. So we added a SNMP poller to poll for the individual CPU utilisation, and alerted when any core alone went above 95%.

KMsigma if he were to set this up as a scheduled NCM job, would the results be available from Cirrus.NCM_NCMJobs?
I have tried to search for them, but my SWQL studio keeps on timing out.
But if you can pull the data from there, i see a solution.

rjrothwell

Hi Yaquaholic!

Thank for the idea of just polling each CPU. I did some Google Foo and Cisco seems only to give total CPU usage rather than individual CPUs.

Source:

https://support.solarwinds.com/SuccessCenter/s/article/How-Orion-counts-multiple-CPUs-from-a-node-and-how-it-collects-CPU-load-data?language=en_US

I did an SNMP Walk and even attempted to use the Universal Poller per the instructions above. All I can get is the Total CPU usage followed by the Total 1 min and 5 min.

Even Cisco's fancy DNA Center and Prime Infrastructure only provides Total CPU with no individual break down. Tell you the truth about DNA and Prime is all it does is using SNMP to monitor as well.

As for the Cirrus.NCM_NCMJobs, are you referring to creating a job to run "show processes cpu platform sorted | include wncd" and creating a SQL script (probably wrong term here) to process the output?

yaquaholic

Yes, that what i was thinking, but I've been having issues pulling data from that table using SWQL to extract the job result, as mine keep timing out. Hence why I asked @KMSigma but I am also struggling to get the @mention working correctly.
SWQL is very similar to SQL, but if the data is in the SWQL API it will also be in the Orion DB.

Failing that, a you could create PowerShell/Python script to SSH onto the device, run the commend and return a value. If you have SAM, this could be done with a monitor (though it may not be easy), If you don't then a script could post result back to the Orion API.
Looking at the manual (https://www.cisco.com/c/en/us/td/docs/wireless/controller/technotes/8-8/b_c9800_programmability_telemetry_dg.html#id_89656) it should be possible (if Ansible can do it, it can be scripted), though I fully understand you'd rather keep it Orion native.

rjrothwell

Yaquaholic,

I never messed with Ansible but I been hearing more and more about it on the network automation part. It may be time to start looking into that direction. I will start looking in that direction. Currently I just have a job that runs and sends the results out every hour so we just glace at the email.

BTW, the Cat 9800 Programmability Telemetry is going to spark some ideas on the Ansible usage.

I have been doing some additional pondering, thinking, brain storming, and such. Under the jobs in SolarWinds, there is a check box about Send email if failure. Could something like this work?

show processes cpu platform sorted | include wncd ${FailureRegEx:4*%}

What I got highlighted and bold are just values that need to be corrected. This is if value is in the 40% range. Originally it was ${SuccessRegEx:Verified} but I could not find one that checks for failure. I am using the OS upgrade scripts to work off of.

sja

Take a look on that power shell ssh example same problem different hardware “fishing cli output”

https://thwack.solarwinds.com/product-forums/network-performance-monitor-npm/f/forum/94373/health-status-in-fortigate-firewall