nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Cleaning up alerts

gbutler

The Solarwinds implementation I'm working in has been arround for many years. I know that most recipients of alerts toss the email into the trash. Not a good scene. I want to fix this. The obstacle is years and years of alerts set up and trying to deteremine what is valild not valid.

If anyone has insite or ideas on this kind of issue... I'm all ears. Here is my plan...

1. Develop a report that will count the number of alerts in a 24 hour period ( The idea is this way I will know how bad it is and be able to "see" any improvemnt as I progress). Now I don't see a canned report for this... I'm working on the following sql:

Select LogDateTime AS LogDateTime,ObjectID AS ObjectID, ObjectName AS ObjectName, Message AS Message

From AlertLog

Where

(Message Like '%Alert Triggered%')

AND

(LogDateTime BETWEEN 40267.3333333333 AND 40268.375 )

2. Focusing on WINDOWS boxes, create a running list, for each node identify all alerts that are involved with it. Now if there was a way of querying the DB to acheive this, I would be very happy.

3. Remove duplicates, update node contact info.

Performance on my Orion server is not great. I'm hoping this will help a little. I also figure our mailing server would not mind the less load.

Your comments/suggestions would be most appreciated!

Glenn.

Find more posts tagged with

Accepted answers

All comments

gbutler

Changed the sql to run as:

Select count(*)AS NumTriggered
From AlertLog
Where
(Message Like '%Alert Triggered%')
AND
(LogDateTime BETWEEN (GETDATE()-1) AND GETDATE())

I'm running just about 1200 alerts in 24 hour period. I don't know if this is high number or not. Seems high. Anyone have an idea what kind of rate they are running? (number nodes = 1575)

mrxinu

200 alerts in 24 hour period. I don't know if this is high number or not. Seems high. Anyone have an idea what kind of rate they are running? (number nodes = 1575)

I guess it would depend on the number/type of alerts. If everything's quiet, I wouldn't expect any alerts to be going off. If you've got some turbulence in your environment and you have alerts watching latency, packet loss, and bandwidth it might be a little noisier.

gbutler

I have 248 alerts (about 50 of these are new alerts that I'm creating in an effort to get suppression set up... as I validate a new alert I added the targeted node to a "not all" list in the non-supressed alert).

I've removed the duplicates... basic alerts that were triggering along with advanced ones for the same problem.

I realize what you are saying... for example, I have a security alert that is suppose to trigger because the actions are being saved off. I get over 100 of these a night. They are OK. The security guy who gets them stores them away and is happy.

For me, if monitoring is to be a value the alerts need to be important (i.e. they need to mean something). I know that probably sounds elementary... but over the years I've seen very few examples where this is the case.

Now that I'm the one who runs the monitoring system... I want to make this place does it right.

My daily report is now consolidated by email "to" address. This has been quite helpful in that I get a quick summary of who is getting what alert and I can query them.

Figuring there were better ideas out there... my initial post was to see what others were doing to ensure the alerts are important. Then again... maybe I'm the only one.

mrxinu

Actionable alerts are where it's at, for sure. Though 248 alerts - I can't even think of more than 20 things you should need to be alerting on. Are you creating multiple alerts because you need to notify different folks?