Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

One application, three component statuses. How to create an alert trigger that compares them?

I have an application monitor that has three PowerShell component monitors. Call them ComponentA, ComponentB and ComponentC.

When Component B OR ComponentC have status Critical I want to raise an alert ONLY if the status of ComponentA is UP.

I quess BNF for the trigger condition would be

On an individual node running MyApplicationNameHere:

IF ( ( ( ComponentB-status -eq Critical ) OR (ComponentC-status -eq Critical)) AND ( ComponentA-status -eq UP) ) THEN raise alert

I've tried to implement this with the GUI alert definition tool in a number of ways without success. Here is a cut-n-paste from the Edit Alert->Summary screen

of the alert definitions.

----------------------------

Trigger Condition try #1:

At least one child condition must be satisfied (OR)

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentB

Component - Status - is equal to - Critical

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentC

Component - Status - is equal to - Critical

And

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentA

Component - Status - is equal to - Up

vvvvvvvvvvvvvvvvvvvvvvvvvvvvv

But this never works because ComponentA never appears to get considered, if I remove the last test (the last AND clause for ComponentA)

then the alert gets generated whenever ComponentB or ComponentC have Critical Status

--------------------------

Trigger Condition try #2 (using a secondary Section in the alert def):

All child conditions must be satisfied (AND)

Application Alerting Properties - Application Name - is equal to - MyApplicationNameHere

Application - Status - is not equal to - Up

All child conditions must be satisfied (AND)

At least one child condition must be satisfied (OR)

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentB

Component - Status - is equal to - Critical

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentC

Component - Status - is equal to - Critical

And

All child conditions must be satisfied (AND)

Application Alerting Properties - Application Name - is equal to - MyApplicationNameHere

All child conditions must be satisfied (AND)

Component Alerting Properties - Component Name - is equal to - ComponentA

Component - Status - is equal to - Up

vvvvvvvvvvvvvvvvvvvvvvvv

The problem with this is that if the application is installed on 2 nodes the test does not match ComponentA to Components B&C. For example, if

the template is deployed on node1 and node2 AND node1 has ComponentC Status -eq UP while node2 has its ComponentC status (not) -eq UP, then a

critical status of ComponentB or ComponentC on node2 raises an alert because ComponentA status on node1 is UP. If both node1 and node2 have ComponentA status (not) -eq UP, then no alert is generated. I conclude that test is not matching all three components to the same application running on the same node.

-----------------------

So maybe the answer is a SWQL-based alert? Since I cannot find examples for how to accurately construct an alert using SWQL, I'm asking for some help here.

Would it be something like "Select the nodes running MyApplicationNameHere where (ComponentB OR ComponentC status is Critical) and (ComponentA status is OK)" and then for each node found, generate a separate alert? The selection statement strikes me as just a matter of me crawling up the SWQL learning curve but generating a separate alert for each instance found that meets the conditions doesn't line up with my conception of how Solarwinds works, I hope I am wrong about that.

I'm also hoping that I can get some guidance from someone, somewhere on how I might try to either resolve this problem OR to declare it unresolvable. Thanks

In Advance!

Find more posts tagged with

advanced alerting

Accepted answers

All comments

christopher.t.jones123

Something that might get you pointed in the right direction for the SWQL method might be that when you go to one that you've attempted to create and click the down arrow all the way on the right it will show you the SWQL query it is using for the trigger condition

before you do that maybe you can attempt this method if you haven't already taken a look at it is using condition, this might be the easiest option for you.

using the complex conditions you could set the scope as component B&C and then say if it is critical, that should satisfy the whether either of those is critical then in the secondary section you could set the scope for your component a and say it is up. something like below

bobpassl

Thanks for the hint about using the SWQL generated by the failing GUI-defined alert as a jumping-off point.

ComponentA is the semaphore and I only want to alert on ComponentB OR ComponentC IFF ComponentA status

is Warning (it should never go Critical, if it is Unknown no alert, if it is OK no alert).

I was able to get the following SWQL command to do exactly what I wanted using SWQL studio

and running through a test suite of alert conditions with my application monitor!

VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

SELECT E0.[Uri], E0.[DisplayName], E0.[ComponentID], E0.[ApplicationID],T0.[ApplicationID],T0.[DisplayName],T0.[Status]

FROM Orion.APM.Component AS E0

LEFT JOIN Orion.APM.Component AS T0 ON E0.[ApplicationID]=T0.[ApplicationID]

WHERE

(

( E0.[Application].[Name] = 'MyApplicationNameHere' )

AND

( E0.[Application].[Status] = '14' )

)

AND

(

( ( E0.[DisplayName] = 'ComponentB' ) AND ( E0.[Status] = '14' ) )

OR ( ( E0.[DisplayName] = 'ComponentC' ) AND ( E0.[Status] = '14' ) )

)

AND

( ( T0.[DisplayName] = 'ComponentA' ) AND ( T0.[Status] = '1' ) )

)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I was ready to declare victory BUT when I tried to add my successful SWQL query into a custom SWQL alert condition,

I discovered that I cannot modify the SELECT statement. If I alert on component, I am forced to use this unalterable

SELECT statement:

SELECT Component.Uri, Component.DisplayName FROM Orion.APM.Component AS Component

Which makes my LEFT JOIN to create T0 impossible! Nuts! Solarwinds is pretty powerful & I appreciate that BUT

it is SO frustrating when it comes to being extensible at times like this. So I think I will have to start over with

alerting on Application where the Application is named MyApplicationNameHere and the Application status is 14 (Critical)

then check the status of ComponentC to determine if an alert should be generated. I'll report back if/when I can

implement that successfully. I considered myself "OK" at SQL a decade ago so I have a lot of rust to shake off. I can

easily see that one advantage of using SWQL is the built-in table linkages that SWQL studio makes available.

bobpassl

Oops. Change "check the status of ComponentC to determine if an alert should be generated" to "check the status of ComponentA to determine if an alert should be generated" in the last paragraph above.

mesverrum

Since this whole thing is a set of script monitors I feel like the easiest way is just to combine them into a single script with 3 outputs and build in the logic to just exit with a 3/critical status. Way simpler to control the script than having to build some fancy one-off alert logic.

bobpassl

If I alert on Application, I can successfully retrieve just the items I want with this SWQL query in the alert (italics used to identify the prescribed SWQL portion of the query when I select "Custom Swql Query" from "I want to alert on") :

SELECT Application.[Uri], Application.[DisplayName]

FROM Orion.APM.Application AS Application

Left join Orion.APM.Component AS T0 On Application.ApplicationID = T0.ApplicationID

WHERE ( ( ( Application.[Name] = 'MyApplicationNameHere' ) ) AND ( ( Application.[Status] = '14' ) ) )

AND ( ( T0.[DisplayName] = 'ComponentA' ) AND ( T0.[Status] = '1' ) )

So far this reliably generates an alert for each test case scenario. I was poised to declare victory again...BUT...the text in the alert Trigger Action email no longer presents usable data. Its tried-and-true text that we use for all our other alerts, as follows:

VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

An issue on an object you are monitoring occurred

in ${N=SwisEntity;M=ApplicationAlert.ApplicationName}

on ${N=SwisEntity;M=Node.AssetInventory.ServerInformation.HostName}

at ${N=Alerting;M=AlertTriggerTime;F=DateTime}.

Check the System reporting the alert, review the Application matching "${N=SwisEntity;M=Application.ApplicationAlert.ApplicationName}"

The Application Monitor provided these details for the alert:

${N=SwisEntity;M=Application.ApplicationAlert.ApplicationName} "

Alerting Component Name(s) = " ${N=SwisEntity;M=Application.ApplicationAlert.ComponentsWithProblemsFormatted} "

Component Details were:

${N=SwisEntity;M=ComponentAlert.MultiValueMessages}

View full object details here: ${N=SwisEntity;M=DetailsUrl}.

View full alert details here: ${N=Alerting;M=AlertDetailsUrl}

Click here to acknowledge the alert: ${N=Alerting;M=AcknowledgeUrl}

This message was brought to you by the alert named: "${N=Alerting;M=AlertName}"

At the time the alert was generated, the status of "${N=SwisEntity;M=Application.Node.DisplayName}" was "${N=SwisEntity;M=Application.Node.StatusDescription

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And the formatted email reads:

VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV

An issue on an object you are monitoring occurred on "${N=SwisEntity;M=Application.Node.DisplayName}" at Thursday, April 11, 2019 12:59 PM.

Check the System reporting the alert, review the Application matching "${N=SwisEntity;M=Application.ApplicationAlert.ApplicationName}"

The Application Monitor provided these details for the alert:

${N=SwisEntity;M=Application.ApplicationAlert.ApplicationName} "

Alerting Component Name(s) = " ${N=SwisEntity;M=Application.ApplicationAlert.ComponentsWithProblemsFormatted} "

Component Details were:

${N=SwisEntity;M=ComponentAlert.MultiValueMessages}

View full object details here: https://["ourOrionServer"]:9797/Orion/View.aspx?NetObject=AA:34187.

View full alert details here: https://["ourOrionServer"]:9797/Orion/View.aspx?NetObject=AAT:86481

Click here to acknowledge the alert: https://["ourOrionServer"]:9797/Orion/Netperfmon/AckAlert.aspx?AlertDefID=86481

This message was brought to you by the alert named: "MyApplicationHereAlertName"

At the time the alert was generated, the status of "${N=SwisEntity;M=Application.Node.DisplayName}" was "${N=SwisEntity;M=Application.Node.StatusDescription

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So it gets the Time of day right, he alert name right and the URLs right (they work). I ["disguised"] our Orion Server's true name. If anyone knows how to adjust the definitions for the inclusions to get the data from the custom SWQL output into the alert email message, I'd appreciate a tip on what direction to go in. At this point, I'm almost confident that Solarwinds can be extended in such a way as to make this work!