How do you keep SolarWinds updated and relevant?

Hi,

IT environments change every day (cloud, hybrid, new rules). How do you keep SolarWinds tuned to stay relevant? Do you review alerts, use custom scripts, or try new features? Share your quick tips and experiences so everyone can pick up new ideas.

Find more posts tagged with

orion

solarwinds

tips

best practice

monitoring

Comments

yellowtj

We have a lot of scheduled discoveries going on and I sometimes audit sections of our installation. Some of those audits include scripts and/or SWQL queries I have written. I also have a couple of dashboards I created. It really depends on what I am looking for.

mesverrum

In my last solarwinds environment I was on a team of people who owned all the contracts and internal policy around monitoring tools, but when it comes to actually using the tools we were mainly intended to be internal consultants who provided a framework and then the various users would self service what they needed. Lots and lots of automation and custom built integrations.

I ran reports out of SNOW to understand ticket volume, trending patterns, resolution times, and false positives to spot places where users might have set themselves up for a suboptimal situation. We scheduled a rotating series of syncs with app owners across the company (>300 teams so there was always someone who we had not reached yet) to make sure they were up to date on the capabilities we had available and to get feedback on how things were going from their perspective.

When we weren't doing that sort of thing we spun up lab environments where we deployed and tested new releases and features. I had it so I could stand up a fresh SW and have it mirror prod in terms of dashboards and alerts and custom properties, and fill it with a subset of node types. Getting that automation in place (mostly a mountain of powershell) was kind of an ongoing effort over a couple years because it was a backburner priority behind the more user-centric work.

By the time I left that gig I was pretty happy with how our whole operation was humming along.

thwackahsanpk

That sounds great, I’d love to learn from the dashboards..

thwackahsanpk

That’s really impressive especially the automation and lab setup, it sounds like you built a very solid and scalable framework...

shehzadcns

As a network engineer, I fine-tune alerts, leverage custom scripts, and test new features in lab setups to keep SolarWinds optimized for changing environments.

bobmarley

One big issue we faced is that we have dozens of engineers adding different node types every day. Some were good at it, some were not, and some forgot altogether. The same happened during node retirement, all three flavors again. Our node inventory changes daily, what can we do to keep it relevant to what is actually out in the field. We tried a lot of manual synchronization at first of some of our data across data sets but that is not something anyone wants to do every day. We tried the network discoveries, but they were missing nodes also. We decided to tap into other data sources which is a combination of security tools. So, in regard to node inventory, we have interfaced to several secondary sources of truth (DB's) that provide very accurate network inventories, our ServiceNow CMDB, as well as the SolarWinds DB. We have scripts in place that scans through all of the data sources and updates a dashboard with any anomalies that need attention.

We review the dashboard daily and correct any issues each morning. Also, just a policy thing for us but all new nodes get added to the MPE initially. Our team reviews each node to ensure monitoring was set up correctly, they we move the node to the respective polling engine it belongs to. Just a couple house cleaning measures we use to keep our node inventory up to date.

On the back side, when nodes get retired the Rundeck job that does the retirement then unmanages the node right before it gets powered down. We have a rule that deletes nodes that have been down or have been unmanaged\muted without a valid end date after 14 days. So node cleanup is automated as well. If someone accidentally removes a switch from SolarWinds for example, one of the many discoveries will find it and place it back into the missing nodes part of the dashboard, so will get re-added again once we review it. In the case of some of our timeclocks, it is 100% hands off automated and is tied to the timeclock database. When nodes get added or removed from the Time DB our automation add/deletes/renames/assigns custom props to them in SolarWinds.

It took a while to put this all in place the first time. Now with AI we could probably build it in a weekend

Daran

Hi @bobmarley I like the sound of some of this - what is the "rundeck job" and also could you share the code for the rule that deletes unmanaged/muted nodes?

We are using SW as the source of truth for feeding into ServiceNow CMDB for network inventory so I'd be interested to hear what discovery jobs you have running?