Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

How to create scaling storage alerts without sending repeat emails?

Hi All,

I have been trying to set up some pretty simple alerts based on % storage utilized. We have some of the default alerts set up to notify us at when captivity reaches 20%, 10%, and 5% remaining. These work great, but tend to create a lot of alerting noise since once the 5% alert triggers, it will also trigger the alert for the 20% and 10% again. So we receive 3 emails for a single event. Is there anyway to change the alerts to stop the cascade of emails? I feel like its probably something simple that I am overlooking.

Thanks for any help or guidance!

Find more posts tagged with

Accepted answers

All comments

mesverrum

I've always been pretty negative on having alerts ramp up like that, it just feels like its destined to get spammy. If you aren't going to actually get up an resolve a problem at 20 and 10 then what's the point? Set your alert threshold at the point where you will actually fix the problem and when you get an alert you know it's something you need to fix instead of this pattern of alert, ignore, alert, ignore, alert, oh no! Everything else can be handled via reviewing dashboards and reports at your leisure.

What people sometimes do is create upper and lower thresholds in their alerts. So " > 20 and < 10" for the first condition, and on for the second and and third ranges.

SteveK

I don't think there would really be a way to avoid the cascade of emails with the 3 tier approach like that. As mesverrum mentioned, you could add additional trigger conditions so the value is between a range or greater than/less than branches. In that case the original alert would clear but trigger the new alert. To go along with what he said, this would definitly get to be a bit spammy. Alert fatigue is a real thing and the recipients would start to ignore the messages.

What i would recommend doing is utilizing the volume thresholds (select the volume, edit volume, and its at the bottom of the page). The defaults are at 90% and 95%.

You are bound to run into systems that run pretty lean on available space, and 20%/10%/5% drastically varies depending on the base storage size of a system. If I see 10% free on a 1TB system I wouldn't be as concerned as 10% free on a 100GB system (of course pending the purpose of the drive).

In the alert trigger conditions, you would then utilize the various "Percent Disk Space Used Threshold" options. For our critical threshold, we use the following:

Then for warnings:

We used the Or statement in it since I've seen some inconsistent issues on the Warning Value Reached/Critical Value Reached in the past, where that flag wasn't tripping. Keep in mind we have been using this alert for several versions/years, so it is most likely not an issue anymore. We do still receive 2 alerts, but we have branching logic (utilizing the alert scope and custom properties) in our alerts where the warning my just show up in the webconsole itself, and the critical sends an email. You could even branch out further to send a page or open a ticket automatically, depending on how your infrastructure is setup.

The main benefit of this approach is you can now change the threshold on a volume by volume basis without having to update or create new alerts.