SNMP Trap to send email alert, but a secondary trap can cancel

I am trying to set up an SNMP trap alerting rule for my power distribution unit (PDU). The PDU has power and environmental monitoring capabilities and network functionality that can send alerts (such as SNMP traps) if certain thresholds are breached (e.g. if temperature goes too high, send an alert). I want to be able to cut down on unnecessary noise though, and sometimes a power threshold will be breached very briefly, and then go back into safe territory (system reboots, power threshold goes too low, but then return to normal levels very soon after).

I am not sure if Solarwinds Trap Viewer has this capability that I am after, but maybe there is a way to use several Alert Actions to make this:

If a PDU sends a "power asserted below lower threshold" to the solarwinds server via SNMP trap:

- wait 2 minutes to see if a "power deasserted from lower threshold"

- if a following "power deasserted" SNMP trap comes in within 2 minutes, cancel any alert actions

- if 2 minutes pass without "power deasserted" SNMP trap comes in, send an email alert to IT-On-Call@company.com

Can anyone think of a way to set this up using Solarwinds Trap Viewer? Thank you.

Find more posts tagged with

snmp_traps

trap_viewer_alerts

trap

Accepted answers

All comments

KMSigma.SWI

I'm trying to think through the problem and I've got a couple of questions:

Who is the vendor of your PDU's? What models are they? Do they support SNMP natively?
Are you using Network Performance Monitor (NPM)? If so, what version?
Should this always be 2 minutes or does that need some flexibility?

jeremy789

Thanks for your reply.

The vendor of the PDU is Panduit.

They model #s are P38D25M and P08D11M

They do support SNMP and I can poll them directly with SNMPv3 and they can also send traps with SNMPv3.

I am using Network Performance Monitor. I believe this is the version: Orion Platform, IPAM, VNQM, NCM, NPM, NTA, SAM: 2020.2.4

2 minutes is an example amount of time. I just need to delay a small amount of time so that it allows for the PDU's monitoring to take a quick dip into an "alert' threshold without immediately triggering an alert to the on call team.

Please let me know if I can provide any more details.

KMSigma.SWI

OK - here's how I would do it (and I did it before with APC's PDUs)

Build a new custom poller to pull the information you want from the PDUs. The Panduit MIBs are pretty well documented (if I'm remembering right), so I think you'll have a good place to start.
Monitor that device with that custom poller and set the polling interval 5 minutes.
Create an alert that triggers when the power disasserted (sorry, I'm unfamiliar with the term) 'thing' happens. Have the trigger condition be that the 'thing' needs to be that way for at least 2 polling cycles - or have it occur X of Y polling cycles (like 3 of the last 5 cycles must report 'thing.')
Then you can send emails, post to a web service, or whatever else you want from there.

The only 'problem' with this approach is that it's possible (though unlikely) that if the metric you are watching is rapidly flapping you might get false positives. For my previous life, that was ok because that meant something was happening with the power and I needed to investigate.

jeremy789

This sounds like a good idea! I'll give it a shot. Thank you.