how to setup grouped node alerts

Question

I am new to solarwind and need to create email and sms alerts for monitored servers and applications (oracle database, MS sql database,exchange and Java).

I want nodes which belong to a particular group to have thier alerts assigned to the users in the group only.

humejo · Answer

No, I haven't used the custom properties for alerts yet.  The alerting system I use doesn't really require them.  I like to set the alerts up to where they don't really need to be edited at all once they are created, since I control who gets them and whether or not they are alerted on at all by using the element's custom property (I use another custom property called "Alert_SuppressAll" that is a True/False property.  The following condition is in the Trigger Condition Scope section of every single alert I create, whether it is a Node, Interface, Volume, Hardware Sensor, or Application alert:  "Node Alert_SuppressAll is equal to False".  Sometimes, if the client needs this level of control, I will also create similar suppression type custom properties for other element types, like Interface, Volume, Application, etc...  Something like "Alert_SuppressInterface" or some variation of that depending on what type of element it is.  This allows monitoring an element without alerting on it.  It is a whole lot easier and faster than trying to control what element gets alerted on in the alert since you start to get all kinds of unnecessarily complicated trigger conditions to keep certain nodes from being alerted on that way.  This way all you have to do is go to that elements properties and set its suppress custom property to True and now ALL alerts will ignore it.  Once again, more time consuming to originally setup but far less time consuming in the long run.).

I used to work in a data center for a decently sized Healthcare organization (3 large hospitals and around 100 office locations) and I was the "Orion guy" there.  We had hundreds of alerts for all kinds of things.  Every time an Application Analyst or Sysadmin wanted an alert or a set of alerts we would go create them one/some.  Sooner or later that person would leave the company or get new responsibilities and sometimes we would remember to go update all of the alerts they had and sometimes we wouldn't.  Then the new person would come in and request their own set and sometimes we would just edit their predecessors old alert or sometimes we would forget all about those (leaving those in place of course...) and create all new ones.  It was just a mess.  Granted, this was before the web-based alerts so it would definitely be easier to manage that kind of system nowadays, but still, its a lot of work regardless and a lot to keep track of.  I mean, we had anywhere from 5 to 20 alerts of each kind of thing (Node Down, Application Critical, Interface Utilization, etc...).  All with their own set of conditions to filter down which nodes were alerted on for that particular alert and all of which had different To: and CC: fields and all of which had their own little alert messages.

So, I developed this system.  Its nothing original of course (I believe someone on Thwack has a really detailed and thorough alert system they documented that uses very much the same philosophy as what I describe mine to be and I'm pretty sure it was before I had designed my system).  The way I designed mine was to allow for minimal effort after the initial setup.  The end-result is that when a new Node is added to Orion or a personnel or responsibility change occurs within the company, as long as the custom properties are properly filled out then the alerts will trigger properly and automatically go to the correct person, no further action required.  I also setup all of my Groups (Dynamic Groups of course), Views (custom Limitations that reference either Dynamic Groups or Custom Property values), and Reports this way.  Now all the administrators of the Orion system have to do is properly label custom properties and everything else just works; the elements appear on the proper view, show up in the proper groups (and thus show up on the proper Network Atlas maps), are reported on in the relevant Reports, and are triggered on in the proper alerts upon which they are sent to the proper person(s).  The alternative would be to go and edit all of those different things each and every time.  It would be an insane waste of time.

I'm sure that was much more than you were looking for but hopefully this general philosophy will help you and/or others since it leads to an Orion system that doesn't just run well and is easy to use soon after configuring it, but also one that runs well and is easy to use 6 months down the road.

bluefunelemental · Answer

Are you adding any alert custom properties?

While I don't do it for groups I have done it for types of alerts including object types, storage vs network vs servers. It could be used to sort the active alerts page by group.

humejo · Answer

Yes, as bluefunelemental states, you don't assign alerts to nodes/groups or any kind of elements.  Alerts are rule based in Orion.  Much more flexible, although it it takes wrapping your head around it at first.  What I do is create two Node Custom Properties, "Alert_ContactNodePrimary" and "Alert_ContactNodeSecondary".  I use these to assign an email address to each Node.  Then, when I create my node alerts (Node Down, Node Reboot, Node CPU Load, Node Memory) I put a variable that points to those custom property values in the To: and CC: field and nothing else.  So in the To field of all of Node alerts I would have ${Node.Alert_ContactNodePrimary} and in the CC: field I would have ${Node.Alert_ContactNodeSecondary}.  Thats it, nothing else.  We don't control who gets the alert in the alert actions window but instead control who gets an alert at the node level.  It is very flexible because if the person or persons responsible for a Node changes, I simply go edit that Node's properties and put the new email address(es) in either/both of the two properties and boom, all node alerts are immediately updated without me having to go edit each of them.

You can use this same concept for Group alerts.  Create two custom properties, "Alert_ContactGroupPrimary" and "Alert_ContactGroupSecondary".  In all of your Group alerts simply put the variables that point to those, ${Alert_ContactGroupPrimary} and ${Alert_ContactGroupSecondary}.  Then, go into the Settings > Manage Custom Properties screen and select those two properties, then click View/Edit Values.  Now sort/filter/search/group your groups in whatever way you need and assign the proper email address to the proper groups.  Now all of your group alerts will go to whomever is in the custom property fields for them.  You want to change who gets an alert for a particular group or group of groups in the future then you won't need to touch the alerts at all, just go edit those groups custom property values for those properties and put in the new address.  Easy-peasy.

This method takes a little bit longer to setup initially, but saves you TONS of time over the long-run and also insures your alerts are always going to the correct person.  This method also reduces the number of alerts you have and reduces the complexity of your trigger conditions since you don't need to have a different version of every alert for each email address.  You only need one of each type of alert (one Group Down, one Group Warning, etc...).  Companies who don't use this method almost always have a whole bunch of alerts to manage and almost always have alerts that are sending emails to people who no longer work for the company or are sending emails to people who are no longer responsible for the items they are getting alerted about.  Without this method your alerts can easily turn into a big mess after 6 months to a year.  With this method it is super easy to apply changes with very little effort or time.