Alert when an Element is Muted

I am trying to create an alert to trigger if any object is set to Mute. I have the report that looks good but now I want an alert. Any ideas would be helpful.

  • This is a non-trivial request because the only place that the mute 'action' is stored is in the Auditing tables as far as I know.  The Auditing tables aren't directly accessible by the alerting engine - because they aren't 'things' that you are monitoring.  You can look in the AlertSuppression tables which have some of the info you might want, and that's where I went with this build out.

    I took a stab at seeing if I could link the things and you can, by using a Custom SWQL query.  I have not tested this, but it should work.  The alert message (email you send) would need significant work to get it to something that's useful to the recipient.

    1. Create a new Alert.
    2. Name it whatever you like, but use a meaningful name and description.  I'd also change the Evaluation frequency to something other than 1 minute.  I can't imagine that you've so many that you need to worry about.
    3. Under Trigger Conditions, select "Custom SWQL Alert (Advanced)" for the "I want to alert on."
    4. Leave the Condition type as Node (You'd have to repeat this process for Groups, Applications, and Interfaces) and in the lower section of the query put:
      -- Copied From 'upper' condition
      -- SELECT Nodes.Uri, Nodes.DisplayName FROM Orion.Nodes AS Nodes
      --
      -- Lower Portion
      INNER JOIN Orion.AlertSuppression AS Muted
      ON Nodes.Uri = Muted.EntityUri
    5. This is doing a VERY simple check.  If a Node is listed in the AlertSuppression table, then trigger.  It does not care about time, expiration, or anything else.

    Like I said, it should work as a trigger condition.  The rest (Reset condition, Trigger/Reset actions, etc.) would vary based on your needs and it gets very complex to try an go through the Audit tables via the alerting engine.

    I'm also attaching an export of this alert that I built.  Again note that this is NOT tested.

    Alert+me+on+a+muted+thing.xml
    <AlertDefinition xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/SolarWinds.Orion.Core.Alerting.Models">
    	<AlertID>254</AlertID>
    	<AlertMessage>${N=Alerting;M=AlertName} was triggered for ${N=SwisEntity;M=Caption}</AlertMessage>
    	<AlertRefID>ec3b9abf-19b5-49ce-b657-69d2948aa743</AlertRefID>
    	<Canned>false</Canned>
    	<Category/>
    	<CreatedBy>admin</CreatedBy>
    	<CustomProperties xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
    		<d2p1:KeyValueOfstringanyType>
    			<d2p1:Key>ResponsibleTeam</d2p1:Key>
    			<d2p1:Value xmlns:d4p1="http://schemas.datacontract.org/2004/07/System" i:type="d4p1:DBNull"/>
    		</d2p1:KeyValueOfstringanyType>
    	</CustomProperties>
    	<Description/>
    	<Enabled>true</Enabled>
    	<ExecutionTimePeriods xmlns:d2p1="http://schemas.solarwinds.com/2008/Core"/>
    	<Frequency>PT1M</Frequency>
    	<LastEdit>2021-05-25T20:10:24.7Z</LastEdit>
    	<Name>Alert me on a muted thing</Name>
    	<NotificationEnabled>true</NotificationEnabled>
    	<NotificationSettings xmlns:d2p1="http://schemas.solarwinds.com/2008/Core">
    		<d2p1:Enabled>true</d2p1:Enabled>
    		<d2p1:NetObjectType>Node</d2p1:NetObjectType>
    		<d2p1:Severity>Critical</d2p1:Severity>
    		<d2p1:Subject>Alert me on a muted thing</d2p1:Subject>
    		<d2p1:_properties xmlns:d3p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
    			<d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    				<d3p1:Key>IP Address</d3p1:Key>
    				<d3p1:Value>
    					<d2p1:Name>IP Address</d2p1:Name>
    					<d2p1:Value>${IP_Address}</d2p1:Value>
    				</d3p1:Value>
    			</d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    			<d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    				<d3p1:Key>Object Sub Type</d3p1:Key>
    				<d3p1:Value>
    					<d2p1:Name>Object Sub Type</d2p1:Name>
    					<d2p1:Value>${ObjectSubType}</d2p1:Value>
    				</d3p1:Value>
    			</d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    			<d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    				<d3p1:Key>Status Description</d3p1:Key>
    				<d3p1:Value>
    					<d2p1:Name>Status Description</d2p1:Name>
    					<d2p1:Value>${StatusDescription}</d2p1:Value>
    				</d3p1:Value>
    			</d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    			<d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    				<d3p1:Key>Node Name</d3p1:Key>
    				<d3p1:Value>
    					<d2p1:Name>Node Name</d2p1:Name>
    					<d2p1:Value>${SysName}</d2p1:Value>
    				</d3p1:Value>
    			</d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    			<d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    				<d3p1:Key>Vendor</d3p1:Key>
    				<d3p1:Value>
    					<d2p1:Name>Vendor</d2p1:Name>
    					<d2p1:Value>${Vendor}</d2p1:Value>
    				</d3p1:Value>
    			</d3p1:KeyValueOfstringAlertNotificationProperty9sQWCBBt>
    		</d2p1:_properties>
    	</NotificationSettings>
    	<ObjectType>Node</ObjectType>
    	<Reset>
    		<Conditions>
    			<ConditionChainItem>
    				<AndThenTimeInterval i:nil="true"/>
    				<ChainType>ResetInverseToTrigger</ChainType>
    				<Condition xmlns:d5p1="http://schemas.datacontract.org/2004/07/SolarWinds.Orion.Core.Models.Alerting" i:nil="true"/>
    				<ConjunctionOperator>None</ConjunctionOperator>
    				<IsInvertedMinCountThreshold>false</IsInvertedMinCountThreshold>
    				<NetObjectsMinCountThreshold i:nil="true"/>
    				<ObjectType i:nil="true"/>
    				<SustainTime i:nil="true"/>
    				<Type i:nil="true"/>
    			</ConditionChainItem>
    		</Conditions>
    	</Reset>
    	<ResetActions xmlns:d2p1="http://schemas.solarwinds.com/2008/Orion"/>
    	<Severity>Critical</Severity>
    	<Trigger>
    		<Conditions>
    			<ConditionChainItem>
    				<AndThenTimeInterval i:nil="true"/>
    				<ChainType>Trigger</ChainType>
    				<Condition xmlns:d5p1="http://schemas.datacontract.org/2004/07/SolarWinds.Orion.Core.Models.Alerting" xmlns:d5p2="http://schemas.datacontract.org/2004/07/SolarWinds.Orion.Core.Alerting.Plugins.Conditions.Swql" i:type="d5p2:AlertConditionCustomSwql">
    					<d5p2:Command>INNER JOIN Orion.AlertSuppression AS Muted
    						ON Nodes.Uri = Muted.EntityUri</d5p2:Command>
    				</Condition>
    				<ConjunctionOperator>None</ConjunctionOperator>
    				<IsInvertedMinCountThreshold>false</IsInvertedMinCountThreshold>
    				<NetObjectsMinCountThreshold i:nil="true"/>
    				<ObjectType>Node</ObjectType>
    				<SustainTime i:nil="true"/>
    				<Type xmlns:d5p1="http://schemas.datacontract.org/2004/07/SolarWinds.Orion.Core.Alerting.Plugins.Conditions.Swql" i:type="d5p1:ConditionTypeCustomSwql"/>
    			</ConditionChainItem>
    		</Conditions>
    	</Trigger>
    	<TriggerActions xmlns:d2p1="http://schemas.solarwinds.com/2008/Orion">
    		<d2p1:ActionDefinition xmlns:d3p1="http://schemas.solarwinds.com/2008/Core" i:type="d3p1:ActionDefinitionEx">
    			<d2p1:ActionProperties>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EmailBCC</d2p1:PropertyName>
    					<d2p1:PropertyValue/>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EmailCC</d2p1:PropertyName>
    					<d2p1:PropertyValue/>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EmailFrom</d2p1:PropertyName>
    					<d2p1:PropertyValue>noreply@solarwinds.com</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EmailMessage</d2p1:PropertyName>
    					<d2p1:PropertyValue>An issue has been detected at ${N=Alerting;M=AlertTriggerTime;F=DateTime} on ${N=SwisEntity;M=MachineType} device named ${N=SwisEntity;M=Caption} (IP: ${N=SwisEntity;M=IP_Address}, DNS: ${N=SwisEntity;M=DNS})
    						View full device details here: ${N=SwisEntity;M=DetailsUrl}.
    						View full alert details here: ${N=Alerting;M=AlertDetailsUrl}
    						Click here to acknowledge the alert: ${N=Alerting;M=AcknowledgeUrl}
    						This message was brought to you by the alert named: ${N=Alerting;M=AlertName}
    						The node is monitored by the polling engine ${N=SwisEntity;M=Engine.ServerName}
    						Audit Message:
    						${N=SWQL;M=SELECT AuditEventMessage
    						FROM Orion.AuditingEvents
    						WHERE ActionType=56 -- user muted alerts
    						AND NetObjectID = ${N=SwisEntity;M=NodeID}
    						AND CONCAT(REPLACE(NetObjectType, ' ', ''), ':') = ${N=SwisEntity;M=OrionIdPrefix} }</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EmailTo</d2p1:PropertyName>
    					<d2p1:PropertyValue>kevin.sparenberg@solarwinds.com</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>EscalationLevel</d2p1:PropertyName>
    					<d2p1:PropertyValue>0</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>executionIfAknowledge</d2p1:PropertyName>
    					<d2p1:PropertyValue>True</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>executionRepeatTimeSpan</d2p1:PropertyName>
    					<d2p1:PropertyValue>0</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>MessageContentType</d2p1:PropertyName>
    					<d2p1:PropertyValue>0</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>Priority</d2p1:PropertyName>
    					<d2p1:PropertyValue>0</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>Sender</d2p1:PropertyName>
    					<d2p1:PropertyValue>Network Performance Monitor</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>SmtpServerID</d2p1:PropertyName>
    					<d2p1:PropertyValue>1</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    				<d2p1:ActionProperty>
    					<d2p1:IsShared>false</d2p1:IsShared>
    					<d2p1:PropertyName>Subject</d2p1:PropertyName>
    					<d2p1:PropertyValue>Device Muted: ${N=SwisEntity;M=Caption} at ${N=Alerting;M=AlertTriggerTime;F=DateTime}</d2p1:PropertyValue>
    				</d2p1:ActionProperty>
    			</d2p1:ActionProperties>
    			<d2p1:ActionTypeID>Email</d2p1:ActionTypeID>
    			<d2p1:Description>To: kevin.sparenberg@solarwinds.com &lt;br/&gt;From: noreply@solarwinds.com&lt;br/&gt;Subject: Device Muted: ${N=SwisEntity;M=Caption} at ${N=Alerting;M=AlertTriggerTime;F=DateTime}</d2p1:Description>
    			<d2p1:Enabled>true</d2p1:Enabled>
    			<d2p1:ID i:nil="true"/>
    			<d2p1:IconPath i:nil="true"/>
    			<d2p1:IsShared>false</d2p1:IsShared>
    			<d2p1:Order>1</d2p1:Order>
    			<d2p1:TimePeriods/>
    			<d2p1:Title>Email on Muted Node</d2p1:Title>
    			<d2p1:TransitiveID i:nil="true"/>
    			<d3p1:BackUpSmtpServer i:nil="true"/>
    			<d3p1:SmtpServer>
    				<d3p1:Address>eastexmbx01v.demo.lab</d3p1:Address>
    				<d3p1:BackupServerID>0</d3p1:BackupServerID>
    				<d3p1:Credentials>
    					<d2p1:Description i:nil="true"/>
    					<d2p1:ID i:nil="true"/>
    					<d2p1:IsBroken>false</d2p1:IsBroken>
    					<d2p1:Name>eastexmbx01v.demo.lab</d2p1:Name>
    					<d2p1:Owner i:nil="true"/>
    					<d2p1:Password/>
    					<d2p1:Username>orion@demo.lab</d2p1:Username>
    				</d3p1:Credentials>
    				<d3p1:EnableSSL>false</d3p1:EnableSSL>
    				<d3p1:IsDefault>false</d3p1:IsDefault>
    				<d3p1:Port>25</d3p1:Port>
    				<d3p1:ProtectionHash/>
    				<d3p1:ProtectionIV/>
    				<d3p1:ServerID>1</d3p1:ServerID>
    			</d3p1:SmtpServer>
    		</d2p1:ActionDefinition>
    	</TriggerActions>
    	<Uri>swis://NOCKMSMPE01V.demo.lab/Orion/Orion.AlertConfigurations/AlertID=254</Uri>
    </AlertDefinition>

  • You can do this via the normal web alerts too. See image below, you may want to delete the unmanaged conditions. This works pretty well for me. The trigger action sends an email with the audit message. THe message includes who and what. Here is the variable string that I use: 

    ${N=SwisEntity;M=AuditEventMessage} on ${N=Generic;M=DateTime;F=DateTime} 

    Trigger condition:

  • This worked perfectly! I tried the first suggestion but was having trouble with it firing the Alert consistently, this works everytime! thank you

  • Definitely go with 's response.  I wasn't aware that the Audit logs were part of the default options for web based reporting.  That was an oversight on my part.

  • Hi,

    I did exactly what you mentioned.

    Having trouble firing the alert.

    Orion Platform HF3, IPAM, SCM HF2, NCM HF1, NPM HF3, PM HF1, NTA HF2, VMAN HF3, SAM HF3: 2020.2.6 

  • - any chance you still have this alert somewhere on a system and can share it to the Alerts Content Exchange?

  • I believe this was intended to be .

    can you provide more details about what is not working. Is it not triggering or not performing your configured actions?

  • You are correct sir - sorry for calling you out there.  That'll teach me to reply via mobile while juggling other threads.  My bad.

  • I created a new alert, but there are still artifacts in the xml output. Custom properties and whatnot. trying to clean them up without breaking the xml. Is there a simple method to do this? 

    Since the question was how to do this, here is a short walk through in lieu of the alert upload.

    Bottom line, create an alert, in the I want to Alert on: Auditing Event

    The scope can be left with default.

    Choosing the trigger condition.. Again Auditing Events, and remember to change the condition to an OR from the default AND.

    Leave the Auditing Event in the trigger condition, for the Field, you will most likely need to choose browse all  fields and select Action Type. Then select Equal To and keep reading for the events. You will need to add five conditions in total.  

    I wish there was a way to sort the events better, they are organized in groups, but not really alphabetically. You can refer to the screenshot in this thread as it is still valid. They will load in the drop down list...

    Look for

    • Node Managed
    • Node Unmanaged
    • Alerts Muted
    • Alerts Unmuted
    • Schedule for muting alerts changed

    The top two node conditions will be next to each other, and the three alert muting conditions will be next to each other also. 

    Frequency for the alert is set to 10 minutes. This still catches things pretty quickly. You can modify if you like. 

    Next, RESET Condition.

    I am currently using the "Reset this alert automatically after ' 1' Minutes. 

    The automatic reset option has been hit or miss in my experience. THis will still catch the events though.

    Trigger actions : setup an email. The swql variable above should work. I will paste below what I tend use with audit events. THe audit event information includes a who what statement, so it is actually sufficient. 

    ${N=SwisEntity;M=AuditEventMessage} on ${N=Generic;M=DateTime;F=DateTime}
    This message was brought to you by the alert named: ${N=Alerting;M=AlertName}

    THere is no reset action and your are basically done. Submit the alert.