cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

SNMP Traps and Syslog Traps can raise an advanced alert in NPM

SNMP Traps and Syslog Traps can raise an advanced alert in NPM

At the moment received SNMP- and Syslog Traps are only parsed and written directly into NPM database

The alerting interface for Traps has nothing to do with the NPM advanced alerting engine - nothing else happens in NPM with that valuable collected information.

Additionally a received critical Trap has no correlation to the nodes status.

So until today Solarwinds NPM is a totally "polling-centric" solution.

Most other competitive monitoring products on market support more comprehensive alerting and node status views based on polling AND on traps.

What we want:

1. Any received Trap - Syslog or SNMP, based on customizable pattern/content criterias should be able to raise an alert in NPM advanced alert manager.

2. The trigger criteria can be regular expression to be able to grep any text content/field out of a Trap.

3. The reset criteria should do the same, to be able to clear raised alerts also by same or different Traps based on customizable pattern.

4. Admin should be able to choose for alert creation:

     a. Repeatedly recieved Traps with same pattern/content should be ignored by alerting engine after alert is triggered once - alert should still be active of course

     b. Repeatedly received Traps with same pattern/content should raise a counter in database to escalate or trigger alerts based on that counter value (e.g. > 10 identical messages should trigger alert)

5. These Trap Alerts (Syslog/SNMP) should change the node status LED in GUI

Why this makes sense:

Most vendors support more alerting features/messages via SNMP Traps than via SNMP polling (also Cisco!)

Alerting based on Traps is much faster than via polling

Alerting based on Traps is more efficient on the network as messages are only created on failure condition compared to polling

Lots of customers use competitive products supporting propper SNMP Trap handling and will not change to Solarwinds as long as this is not supported there.

There are already a lot of requests for Traps (SNMP, Syslog) raising an alert as found in thwack community:

http://thwack.solarwinds.com/message/195721

http://thwack.solarwinds.com/message/122581#122581

http://thwack.solarwinds.com/message/174510#174510

http://thwack.solarwinds.com/message/228483#228483

http://thwack.solarwinds.com/message/192825#192825

http://thwack.solarwinds.com/message/217626#217626

http://thwack.solarwinds.com/message/212761#212761

http://thwack.solarwinds.com/message/36239#36239

113 Comments
Level 11

Looks like we're stuck with more cheesy workarounds (homegrown database hacks or other grassroots efforts) until the voters are finally heard and something is done.... I'm perplexed because this just doesn't sound that difficult to achieve.

Level 8

I confess I'm extremely disappointed that such an obvious resource is not available yet...

Level 10

thank you,

on the end i solved this by polling, and skipped idea to to triger/intercept snmp trap

Level 9

This is causing me a huge problem with one of my customers at the moment. They have UPSs that send traps to warn that they are over a temperature threshold, and while it's easy enough to set up an email alert based on the text in a trap, it won't then show in the Alerts view which is what they focus on all the time.

Level 16

check out APC_Battery_overvoltage.AlertDefinition

this uses APC traps for battery over-voltage conditions; duplicate for the other PAC trap types you have in your environment.

Level 10

Thanks,

Is this SQL alerting method officially supported by Solarwinds NPM ? 

We can use it then (as a workaround until the traphandling and syslog handling in NPM is completely state of the art .. )

Level 10

This can be a workaround indeed, when you have to generate an alert as result of an snmp-trap, and you want to alert on Traptype only. I got it working with an interval time of 15 sec.  Sometimes it is nescessary to alert on a combination of a traptype with a trapdetail. Therefore you have to join the TrapVarbinds table as well. To achive this, I have tested this query, but it didn't get it working until now:

 

WHERE Nodes.NodeID IN (SELECT Traps.NodeID

          FROM Traps

          WHERE Traps.TrapID IN (SELECT TrapVarbinds.TrapID

                    FROM TrapVarbinds

                    WHERE (

                              Traps.TrapType = 'CISCO-UNIFIED-COMPUTING-MIB:cucsFaultActiveNotif'

                              AND TrapVarbinds.OIDName = 'cucsFaultDescription.9558108'

                              AND TrapVarbinds.TrapIndex = '3'

                              AND DateTime > DateAdd(day, -1, Sysdatetime())

                              AND Acknowledged = '0'

                                   )

                                        )

                                             )

In the SQL server Management Studio, this is working fine, and fast, but in practice, it doesn't generate any alert...

Has anybody an idea why this isn't working, or what is missing here ?          

Level 16

You will have to push up the log level to DEBUG on the alert manager to figure out why.

what is your reset condition?

if you did not specify one then the generated one is probably wrong.

also, you can way simplify your SQL to something like:

WHERE Nodes.NodeID IN (

SELECT traps.nodeid

FROM   traps with (nolock)

       INNER JOIN trapvarbinds with (nolock)

               ON traps.trapid = trapvarbinds.trapid

                  AND traps.traptype = 'CISCO-UNIFIED-COMPUTING-MIB:cucsFaultActiveNotif'

                  AND trapvarbinds.oidname = 'cucsFaultDescription.9558108'

                  AND traps.datetime > Dateadd(day, -1, Sysdatetime())

                  AND acknowledged = '0'

)


and the reset condition is simply


WHERE Nodes.NodeID NOT IN (

SELECT traps.nodeid

FROM   traps with (nolock)

       INNER JOIN trapvarbinds with (nolock)

               ON traps.trapid = trapvarbinds.trapid

                  AND traps.traptype = 'CISCO-UNIFIED-COMPUTING-MIB:cucsFaultActiveNotif'

                  AND trapvarbinds.oidname = 'cucsFaultDescription.9558108'

                  AND traps.datetime > Dateadd(day, -1, Sysdatetime())

                  AND acknowledged = '0'

)


Level 10

Richard, thanks for your tips, in the SQL studio  this "streamlined" query is working well again.  I am now testing it in the adv alert manager.

How can we "push up the log level to DEBUG on the alert manager"  ?   (I know where the logfile resides..)

Level 16

the logadjuster -- installed on the server allows you to change the logging level.