Is there a way to build that out within the original 95% capacity alert I have built?
You cannot trigger actions upon breaching different thresholds. However, you can have an 'Escalation' for your alert action that can repeat the action (such as email) every X minutes. The email can include the latest utilisation % in the subject and/or body. You can use SQL macros to insert some different conditional keywords based on the % value.
Am I doing this under "reset condition" tab within the alert?
Trigger Condition
alert when - Volume - Volume Percent Available - is greater than - 95%
Hope this helps!
No, reset condition is when the alert returns to normal. To do escalation go in to Trigger Actions and select your email trigger alert. In there you'll see the Alert Escalation tab. Fill it out as you see fit.
But if you really want alerts on 95% , 96% etc you would need an alert for each threshold. But remember if you do that, you will get multiple reset alerts once the usage goes below 95%.
You can create multiple identical Alerts with different thresholds and only configure RESET on 95% - this will be more initial work to setup, but it will work exactly how you have envisioned. If you go this route - I would suggest to set 96%-100% alerts with much larger reset Time, something like 1 hour, whereas 95% would reset after 15 minutes - this is to avoid alerting on spikes and alerting on fluctuations... Brrr... gets complicating...
I would recommend creating just 2 instead - at CPU warning level and at CPU critical level. You can configure both CPU global warning and critical levels under SETTINGS, and you can also override these values per node individually. This will be more scalable and flexible solution I beleive. We have warning at 80% and critical at 95% here - both alerts will wait for condition to be present 30 minutes before firing off - to avoid alerting on spikes and short-term CPU activity
