Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Is it possible to alert off the number of "down" components in SAM?

lorenrichardson

In my template, I have 6 components (TCP Port Monitors) and need to alert based on how many are down:

1) 1 or 2 are down (send low priority incident)

2) more than 2 are down (send high priority incident)

Would this have to be accomplished using 2 separate alert definitions?

Find more posts tagged with

Accepted answers

lorenrichardson

Thanks for the suggestion. I only have a single instance of my application monitor so I'm not sure how I can utilize it as part of a group. I'm leaning towards a custom SWQL query as my alert condition. That seems to give me the ability to alert on the number of down components in the application monitor.

I also need to have two action paths: 1) low priority ticket on 1 or 2 components down OR 2) high priority ticket on 3 or more components down. The only way I know how to do this is by building two separate alerts.
Here's what I have so far:

Alert 1 looks for 1 or 2 down components with this query:

SELECT Application.Uri, Application.DisplayName

FROM Orion.APM.Application AS Application

WHERE Application.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'

AND

-- The number of down components is greater than zero.

(
SELECT COUNT(Components.ComponentID) AS ComponentCount
FROM Orion.APM.Application AS App
JOIN Orion.APM.Component AS Components ON App.ID = Components.ApplicationID
WHERE App.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'
AND Components.ComponentType LIKE '%TCP Port Monitor%'
AND Components.Status = 2
) > 0

AND

-- The number of down components is less than 3
(
SELECT COUNT(Components.ComponentID) AS ComponentCount
FROM Orion.APM.Application AS App
JOIN Orion.APM.Component AS Components ON App.ID = Components.ApplicationID
WHERE App.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'
AND Components.ComponentType LIKE '%TCP Port Monitor%'
AND Components.Status = 2
) < 3

The trigger action tied to this alert sends the low priority ticket to ServiceNow using the built in integration module.

Alert 2 looks for more than 2 down components with this query:

SELECT Application.Uri, Application.DisplayName

FROM Orion.APM.Application AS Application

WHERE Application.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'

AND
-- The number of down components is more than 2
(
SELECT COUNT(Components.ComponentID) AS ComponentCount
FROM Orion.APM.Application AS App
JOIN Orion.APM.Component AS Components ON App.ID = Components.ApplicationID
WHERE App.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'
AND Components.ComponentType LIKE '%TCP Port Monitor%'
AND Components.Status = 2
) > 2

The trigger action tied to this alert sends the high priority ticket to ServiceNow using the built in integration module.

So far, this appears to be working for me.

All comments

dodo123

You could try putting the applications in a group and alert of the status whilst adjusting the way the group status changes. https://documentation.solarwinds.com/en/success_center/orionplatform/content/core-managing-the-display-of-group-status-sw1691.htm

lorenrichardson

SELECT Application.Uri, Application.DisplayName

FROM Orion.APM.Application AS Application

WHERE Application.Uri = 'swis://orion.mysite.com/orion/orion.nodes/nodeid=9876/applications/applicationid=1234'

AND

-- The number of down components is greater than zero.

AND

The trigger action tied to this alert sends the low priority ticket to ServiceNow using the built in integration module.

Alert 2 looks for more than 2 down components with this query:

SELECT Application.Uri, Application.DisplayName

FROM Orion.APM.Application AS Application

The trigger action tied to this alert sends the high priority ticket to ServiceNow using the built in integration module.

So far, this appears to be working for me.

SteveK

I can't validate the query, but the 2 seperate alerts would be the way to go. It looks like you are going about it the right way though. I would have done something similar, where if count is greater than X trigger one alert. Then a second alert but configured to trigger if the count is greater than y.

I would recommend though having the 2 stack. So it looks like on the first one you are saying >2 but , I'm not a big fan of that approach. Say you have a component flopping and 2 are already bad, based on the time threshold of the alert it would never trigger. If you have it set to an immediate alert, then you may end up spamming whomever the target audience is. I've personally have never had a problem with a nodes/sam generating a warning and critical at the same time. Yes when things go bad on the dashboard there maybe a little extra noise, but better to have that than repeated spam or missing the event.

lorenrichardson

Hopefully I've setup my conditions to avoid spamming the user. My reset condition for both alerts is another custom SWQL query. It looks for all components to be back UP. I figured it might be a bit misleading to clear the lower level ticket whenever it's crossed from 2 down to 3 or more down. The default condition was to reset when the trigger condition was no longer true.

My first alert will trigger if more than 0 but less than 3 (1 or 2 are down). User receives a "Warning" ticket and email letting them know things might be getting dicey and maybe it'd be best to do something...

My second alert will trigger if more than 2 are down (3+ are down). User receives a "Critical" ticket and email letting them know their application is down and immediate action is required.

I'm going to let the alerts I've come up with bake in for a bit and see how it works out. Thanks everyone for the help and suggestions!