Hi,
I have a question relating to alerting, but specifically around how you manage alerts out of hours. We alert at a Group Level, within each group are the nodes/applications/DB's etc that make up the Business Service (Group) When 1 of these components are "Down" an alert is triggered. We have been collating these out of hours to analyse any false alerting and see what the hit would be to our on call team if these where to raise a genuine out of hours call. The issue we have here is that most of these services also have a differing "Service Hour" so some services are 24 Hour Mon-Fri, others are 24x7 etc (there are 9 different ones in total)
During analysis I have found that the majority of the calls raised outside of service hours are raised as an element of the business service is undergoing some form of maintenance - scheduled tasks, batch, backup and in some cases patching (although Win Updates arent causing alerts)
We could change our alert model from 24x7 for all systems to a Service Hour model (all 9) however, this will not allow us to alert for systems being down out of hours so that we can return it to service before the business need it.
Create maintenance windows using the tool SolarWinds provide is not an option, the creation of multiple scheduled tasks on the environment is far too cumbersome.
So, answers on a post card if you can please?
Any help would be greatly appreciated.