Is there a way to trigger a single alert from a polled result OR an SNMP trap?

I am trying to set up alarming for a number of devices being monitored in the field. These devices have individual alarm points, each of which is a unique alarm, and devices can have up to a couple hundred configured alarm points. SolarWinds is able to monitor these devices, but I am struggling with the alarming. I can use SNMP traps, which seems to be the best way, and use log fired events to trigger alerts. I also have a custom UnDP that can poll the current state of all the alarm points, and I have figured out how to get alerts to trigger from that as well. My problem is that I don't know how (Or if it is possible) to have the same alert trigger from either a trap coming in, or the polled state showing in alarm.

I believe I could modify the SWQL query I'm using to trigger the polled state alert to also check to see if there is a trap log that is more recent the latest polled state, but with literally thousands of alarm points that need to be monitored, that's a lot of database calls happening constantly since I'd need a complex SWQL query to happen for each alarm point on each device at least every minute if not more often (And that's why the trap would be vastly superior). However, some devices are on unstable connections due to just age and infrastructure and I need a way to check the alarm state without being solely dependent on the traps getting through (Which makes the polling necessary). But I also don't want to have to make two alerts for every alarm. Is there a way to get a single alert to trigger from a trap OR from a UnDP polled state? Barring that, is there a much better solution that I'm just not seeing?

Thanks.

Find more posts tagged with

Accepted answers

All comments

stuartd

I think, the alert we use to poll Fortigate HA state could possibly work. Essentially what we are doing is asking the node "has your state changed from the last time we asked" - you'd need to edit it to suit your custom poller but it looks like this:

The editable SWQL is this:

INNER JOIN
(
SELECT
CustomPollerAssignmentID
, Count(DISTINCT RawStatus) AS Statuses
FROM Orion.NPM.CustomPollerStatistics
where
Datetime>Addminute(-10,GETUTCDATE())
GROUP BY CustomPollerAssignmentID
) AS History
ON CustomPollerAssignmentOnNode.CustomPollerAssignmentID=History.CustomPollerAssignmentID

WHERE
CustomPollerAssignmentOnNode.CustomPollername='Fortigate_HA_State_Change'
AND History.Statuses>1

JonIro

I appreciate the response! I'm not certain about the solution, though - I think I might not have provided enough context.

An example: We have door sensors at all our sites. These door sensors are wired to a remote terminal. The terminal is set up to show an Open Door alarm if the door is opened. That terminal can send an SNMP trap to SolarWinds and be polled using a UnDP in SolarWinds to see the current status of this alarm. The trap seems to be the most obvious way to handle that, and use a Log Fired Event to trigger the Open Door alert for the site. However, if the remote terminal went offline for some reason, the door was opened, and the trap was not sent due to being offline, once the remote terminal was back online, SolarWinds would not have the Open Door alert active since the trap was never received. So that's where the UnDP would come in to check the status - but I can't find a way to trigger the same alert in SolarWinds. I'd have to configure one alert for the Open Door derived from the SNMP trap, and a separate alert for the Open Door derived from Polling the remote terminal directly. Even though it's the same originating alarm.

stuartd

I think I get you ... the sample code is there though to do with as you see fit.

We know it works for polling "statuses" but specifically for polling a status and identifying that it isn't the same as the last poll. So it may work with your direct polling approach. We don't use traps* so I can't help there.

* For a number of reasons but prime amongst them is that Solarwinds took what was a very usable and working standalone EXE file, integrated that into the GUI and reduced its capabilities. Then to add insult to injury acknowledged that it was inferior and then said you have pay for the upgraded OLA, then you will get 80% of functionality back. Ermm, no thanks.