Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Beta Alert Manager - help

I'm trying to create an alert based on a multiple nodes going down within a certain time frame.
Basically - alert me when 12 or more nodes go down within a 5 minute time period.

It looks like this should be possible using the new Alert Manager, but I can't seem to find the correct syntax to use to get this to work.

Any ideas?

-=Cheers=-
NG

Find more posts tagged with

Accepted answers

All comments

Network_Guru

Bump!

....anyone?

Don331

NG,

Any 12+ nodes going down, or a specific subset?

I'm thinking of a stored procedure to count the number of active alerts older than 300 seconds - and then set a flag in a custom property to trigger the Alert Manager

Don

Select * from Users where Clue >0
..0 Rows Returned

Network_Guru

Hey Don,

I'm just looking for any 12+ nodes down with the same custom property.

I have 200 remote sites which are being managed by another group (they get alerts for each site). I only want to be alerted when more than 12 - 15 nodes are down at the same time.
This usually indicates an ISP or VPN concentrator problem, for which my group must be notified.

Your idea sounds good, but I'm not sure how you would run the stored procedure every 5 minutes to check for the number of down nodes?
Any examples you can provide would be much appreciated.

-=Cheers=-
NG

Don331

My DBA is working on the code - this would be a stored procedure in SQL Server. Is the custom property for you in the Nodes table?

Don

Select * from Users where Clue >0
..0 Rows Returned

Network_Guru

Thanks Don,

The custom property is called "building" & is in the Nodes table.

Basically:

If Nodes.building = xyz
AND status = down
AND count = 12
Then send alert (or change another custom property used to send the alert)

-=Cheers=-
NG

alfordc

Has anyone been able to find a solution for this?

I have a similar problem with a site that is connected via a wireless link, often the link goes down due to interference and Orion sends out an alert for every device that is down. To get Orion to trigger an alert when it notices say 3 or more nodes (within a minute) are down on that site while suspending the individual node down alerts would be great.

Thanks in advance,
Chris

iunderwo

Looking at this and thinking the same thing, would it be feasible to send a syslog when a node goes down in a specific format, and then let the syslog manager send an alert from there based on that format?

The Syslog manager does have threshold alerts, which is nice.

// Ian Underwood - Network Engineering
// Boston Stock Exchange

Network_Guru

Thanks for the idea Ian,

I've actually done something similar already.
I'm filtering the syslog messages from my VPN concentrator for "client disconnected".
Suppress alert unless there are more than 10 disconnects in 90 seconds.
The problem with this is if/when the VPN concentrator locks up or hangs, I may not get the syslogs.

You have hit the nail on the head - send syslogs from Orion when any remote node goes down, to 127.0.0.1. Then use the Syslog alerter to suppress/manage the alerts.

-=Cheers=-
NG