cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

FEATURE REQUEST - Not all traps are alerting - Case - # 00332128

FEATURE REQUEST - Not all traps are alerting - Case - # 00332128

Our Aruba Airwave server polls for down Wireless AP's and send traps into Log Manager for each down AP that it finds after every scheduled poll. This means that it sometimes  sends multiple traps at one time (one trap for each down AP). The traps all have the same OID, Type, etc,  but the entry detail is unique in each trap as it contains the name of the AP which is down. All of these traps reliably arrive in Log Manager as separate entries as they should and each is tagged by a rule which also has alert integration with Orion alerting.

The problem that we are having is with Orion alerting when more than one AP down trap arrives at the same time from the Airwave Server. The issue is that although we can see each trap in Log Manager and the rule has tagged them all, only one of the traps makes it to Orion Alerting. That is a big problem for us as we don't get alerted on all of our down AP's. As an example we had a building lose power and 100+ AP's went down.  Log Analyzer correctly showed 100+ tagged entries but Orion Alerting only displayed an alert and notified for one of the AP's and we therefore missed a critical issue as we thought that only one AP was down and therefore we assumed it was not serious.

We opened a case with SolarWinds and they said that the product is working as designed but they acknowledge the issue and the engineer opened a feature request to the development team. To add weight it was suggested I open a feature request here.

This feature request is to therefore please add functionality so that Orion Alerting alerts on all traps when multiple traps are received at the same time from a server/device. This includes where the trap OID, type are the same but the entry detail is different as is the case here.

NPM 12.4

LM 2.0

Sincerely,

Tony D

7 Comments
Level 16

Already requested this feature and had a support case.

My support case #00309262 - unresolved.

Feature Request:

MAKE LOG ANALYZER SO IT IS NOT SINGLE THREADED

https://thwack.solarwinds.com/ideas/11010

Thwack post:

MULTIPLE ALERTS IN A SINGLE ALERT CYCLE POSSIBLE IN LOG ANALYZER

https://thwack.solarwinds.com/message/417708#417708

Did a user poll as well

HOW SHOULD YOUR SYSLOG RECEIVER PROCESS EVENTS

Here is how the poll went:

Only 8 people chose the actual behavior of the tool.

As you can see I chose to instantly make an alert for every event, that is the way the 'old' engine worked.

pastedImage_0.png

This is the word from Support.

Thanks for your patience in this matter. I have received word back from the Dev team for Log Analyzer for this. Here is their reply: " LA triggers an alert per rule every minute and it's because we don't want to overload customers with many alerts in case of problems. "

Level 10

I think the issue happens with syslogs as well, so the solution should apply to both traps and syslogs, or any logs processed by Log Analyzer.

Level 16

I tested Syslog and Log Events and the behavior was the same.

Level 16

i think since its pertaining to same trap so when u try to alert it will always pick the first one... do u have any ticketing tool in place, you can instead send ur traps directly to ticketing tool which will definitely pick every AP down case...

Level 7

Actually, it's the 4th option how it works:

- set the reset condition to "No reset condition – Trigger this alert each time the rule fires"

- insert ${N=OLM.AlertingMacros;M=OLMAlertMessage.HitCount} macro to your alert message

Btw it is possible to trigger alert with every incoming syslog/trap/windows event, but it is extremely dangerous, so the default value is once per minute. LA can process messages faster than Orion Alerting can send alerts, and if alerting gets overloaded (for example when something goes wrong, many alerts are tiggered and many syslogs and traps are received at the same time), it might stop sending all Orion alerts, which is definitely something you don't want.

Legacy syslogs/traps used their own alerting, but it was not possible to integrate the alerts with Orion Alerting, e.g. you couldn't use custom property as a trigger condition.

Level 16

In my testing I wasn't able to trigger an alert for every single event. I was using the example of receiving a message each time a person scanned a badge reader to enter a door. Only the first person that scanned each minute was recorded.

Person 2, 3, 4, etc were missed.

Community Manager
Community Manager
Status changed to: Open for Voting