Hello all,
Yesterday we upset the email team that handles all outbound emails. The firewall that sits between us and them failed Saturday morning, and thus Solarwinds was collecting alerts (by the thousand) but it couldn't send them out due to the firewall outage. When the issue was resolved on Monday, all of those collected alerts were able to make their way out.
I'm a complete noob when it comes to Solarwinds alerts. Probably due to how advanced they can be. I do want to explore more into getting more detailed alerts. In this case, if possible I would like to make the following changes:
-
Only send alerts if a group of servers are available (the ones that would process the request). If these servers are down, do not send any alerts, even after they come back up.
-
Limit the number of alerts sent. I know we can add a schedule, but perhaps something only like "stop alerting after 3 were sent".
-
Prevent alerting for each individual server. If one server is down, sure, an email is fine. If two servers are down, send one email informing of both servers.
Are all of these items possible? Any tips or tricks related to this would be greatly appreciated as well. We don't have real 'on call' so there's really no point in sending thousands of alerts until someone acknowledges them. Plus in this specific case they wouldn't have worked until connection was restored anyways.
