11 Replies Latest reply on May 14, 2012 2:49 PM by Bryan.Brummond@dtn.com

    Creating an Alert/Event From Syslog Server

    Khue

      I am currently testing the Log Forwarder application on a few servers I have. It works great so far. Easily enough, it forwards alerts from the event logs of individual servers to the NPM syslog server. I would now like to generate alerts and events from those outputs but I am having issues figuring out what the best way to do so would be. Would I use the advanced alert manager to set this functionality up or is there an easier way to do this? The concept is if I forward a log from log forwarder, I want it to be viewable inside of Orion and depending on what I deem necessary send an email on that issue. 

        • Re: Creating an Alert/Event From Syslog Server
          atoomey

          You can find this in the Syslog Viewer itself. Start > All Programs > Solarwinds Orion > Syslog and SNMP Traps.

          Create a Filter for the particular Syslog messages you want to alert on and assign alert actions to that filter.

          Steps on doing this are here

          When creating the filter, you can wild cards and you will want to narrow the filter down to the specific traps and potentially throttle the email when configuring this so you dont get spammed.

          From this tool, Goto File -> Syslog Server settings -> Alert/Filter Rules Tab 

          In here you can filter using various methods, By IP address, by Message Type Patterns, Syslog Message Patterns, Severity, etc… 

          Have a look and let me know if you have any questions on any of this.

          Regards,
          Alan Toomey 

          • Re: Creating an Alert/Event From Syslog Server
            dclick

            Forwarding logs to Orion will give you the ability to create advanced alerts - the easiest way I have found (for servers at least) is to create the alert on all nodes, but to set the cireteria on the text of the event ( 3rd tab in the Alert creation panel).

            In general, you will want to know what type of event you want to alert on - so, for instance, if your looking for  any Error or Critical event (assuming this is a Windows event log you are forwarding) you would want to use *Error* in the "Message Type Pattern* of the alert.   It took me a few tries to get it right. Once you have this part figured out, you would modify the alert like any other alert for orion - either log it, email it, page someone, etc.

            As for vewing the alert in Orion - you can use either the Message Center or Syslog view from the main Orion server web interface. We have customized our menus, and moved these around, so I dont remember the exact location.

            HTH.

            Don

            • Re: Creating an Alert/Event From Syslog Server
              itnetworkteam@ghg.co.uk

              Hi guys, hoping you can help. I have a similar question. We have been using NPM syslog server quite well for a long time. We have alerts set-up which pattern match on incoming SYSLOGs and fire off emails as this thread describes... so no problems there.

              *HOWEVER* for a particular type of syslog (specifically an Aruba access point going down/up) we are getting SLAMMED with emails on account of us having a few malfunctioning access points on our WAN... so i was hoping for a better way to do things. What im hoping we can do is, rather than event driven emails (so we get one each time it goes down and up and down and up etc) change it to a view on the NPM homepage. Call it "Down APs" ie - When a DOWN syslog is received, an NPM event is listed. When the corresponding UP syslog is received, that event is cleared. That way we get a real time status view of our down APs. 

              I know a lot of you are probably thinking, just add them as an NPM node and you will get this info via the node status, but we have over 1000 APs (chew through more licenses than we have left), they are all (as per aruba recommendation) dynamic IP hosts. The syslog -> email alert system has worked well, cept for when an AP goes crazy and reboots every 2 minutes.... as we currently have a couple doing which has forced us to turn off that syslog action... which means we are blind to APs going down.

              any help, or creative thoughts would be apprecieated!!

              cheers

              Keiran.

                • Re: Creating an Alert/Event From Syslog Server
                  dclick

                  The first thing I would suggest is to modify the email alert alarm. Under the Trigger Threashold, you can set it to suppress alert actions for ## of messages in a XX time (hours, minutes, etc).  This would aleast slow down the influx of emails until you can get the AP issue fixed.

                  Secondly, since you want to have these show up on your home page, your going to have to add them to NPM.  I understand they are using DHCP to get an IP, however, depending on you have the dhcp scope setup, it possible to still get accurate status. For us, our DHCP scopes are set on average to 8 day lease times. This gives me a 4 day half life. If your ap's are bouncing as much as you indicate, the DHCP server is going to re-assign the same IP for at least 3 days (again, with a 8 day lease).  Another option - and this gets into network design, is configure your management interfaces on the AP's for a dedicated AP IP Subnet/Vlan - that way, all the ap's are on the same vlan, and independant of the dhcp lease, if one goes down, you would know that only AP's are on that subnet/vlan and have an issue.   I am not familiar with Arubas - we use Cisco AP's and have Cisco Wireless Contrllers that manage the AP's for us - I only monitor the controllers. (The controllers are configured to send email alerts to us, as well as traps to Orion).

                  You can also create a syslog event "window" on the home page that would show the last XX events, but without having the AP's in Orion, it might be difficult to filter out everything else.

                  HTH-

                  Don

                    • Re: Creating an Alert/Event From Syslog Server
                      itnetworkteam@ghg.co.uk

                      Thanks Don.

                      Guess i was hoping for a way to create a new NPM event from a received syslog. From an event, we could alarm etc. But i guess all of this is focused around and assumes you have a NPM node.... which for these APs, we do not.

                      FYI, Aruba also uses a centralised controller model, and the syslogs are actually coming from the controller when it looses/re-establishes comms with an AP. Unfortunately, Aruba also recommend their AP's wired interface to be in the same vlan/network as the rest of your data (for things like rogue AP detection), so i cant follow up on your other suggestions. 

                      i will look into the suppression for now. I guess that is my best option :(

                      thanks, and if you think of anything else creative, let me know!

                      thanks guys

                      Keiran.

                        • Re: Creating an Alert/Event From Syslog Server
                          dclick

                          As far as being able to create a NPM log event from Syslog - Yes - that would be a great feature additon.  There are other issues with the syslog/snmp trap log that need to be looked at to (like being able to highlight a existing message, and generate a rule based on it with one click, rather than having to maually generate the rule, and/or the ability to manage the log server, including alerts, from the web interface, etc..)

                           

                          Good luck!

                            • Re: Creating an Alert/Event From Syslog Server
                              Dogeron

                              I would support both Keiran and Don in requesting that both the syslog and trap viewer are enhanced to allow them to create NPM events.

                              It just makes sense to put all events, however they are generated, in one place (against the NPM node).

                              Dog.

                                • Re: Creating an Alert/Event From Syslog Server
                                  smartd
                                  Absolutely.  Never understood why this wasn't done.  Seems so easy.
                                    • Re: Creating an Alert/Event From Syslog Server
                                      jmay

                                      I agree totally. Need to be able to record  to NPM log from syslog trap viewer . This way all located in one location and reporting is much easier.

                                        • Re: Creating an Alert/Event From Syslog Server
                                          Bryan.Brummond@dtn.com

                                          Agreed that the way Syslog Alerts /Filter Rules needs to be reviewed. The Syslog Filter Rules are not reliable due to the Syslog message coming in is filtered by each rule going down the list and if Syslog message finds one that matches before reaching the one you setup the Syslog message is no longer processed. We have multiple groups that manage the filters, and if they add one before ours and it matches we will not be notified of an Alert. And then trying to find the rule that's above yours that has knocked yours out is big waste of time. Also in my testing the filtering is case sensitive making it difficult when testing. And the filtering of hostname's is not friendly.

                                          That's why it would be very nice to have some kind of interface to manage Syslog messages like the Alert Manager tool. This is a SQL query for exactly what your looking for and sends out results when found. You don't have to worry if someone else making a change and your Filter Rule might be obsolete. And being able to use Custom Properties with Alert Manager makes the tool very robust in focusing on just the nodes or applicaitons your monitoring.

                                          Syslog messages are very important. Any help would be appreciated.

                                          Thanks,

                                          Bryan