SYSLOG / Trap overload prevention

tonyreel

Hi Guys,

I have started to use the Syslog and Trap parsers quite heavily and have setup lots of rules for generating emails to our teams on specific conditions etc etc

However when things break or go wrong (as they do) we tend to get 1000's of messages - syslogs especially - which can cause a lot of emails to be sent in quick succession.  So much so that they can back up on our mail server and came close to crashing it last week.

I have configured a rule to notify me when the syslog server receives more than 100 messages in 60 seconds which gives me a heads up that something is spamming the parser.   Ideally tho I would like a rule that could disable all the other rules for a set period of time. 

Anybody got any ideas on how to achieve this.  The best I could come up with was a script to disable the Syslog service however I would still like to collect the syslogs just not send any mails.

Thanks

T

tonyreel

mmm, I wonder would this work:

Rule No 1. Match against any syslog message.  Trigger threshold is set to 100 in 1 min and "suspend further alert actions" is set to 5 minutes.  Alert Action is to send an email saying Syslog parser is overloaded. 

Rule No 2. Match against any syslog message.  Trigger threshold is set to 100 in 1 min and "suspend further alert actions" is not selected.  Alert Action is to "Stop Processing Syslog Rules".

No.1 would mean if i was getting 200 messages a minute I would get one email every 6 minutes (5 minute suspended and 1 minute to count again)

Not sure what way rule 2 would work out.  Does orion check how many messages have been received in the last 60 seconds every time it gets a new syslog, if it does then this would work ?? Or would it count the first 100 messgaes and then stop the 100th from being processed any further and then start counting again.