Polling and Alerting Timing

Question

If I have an application monitor that is polling every 2.5 mins (150 seconds) and an alert that is evaluating every 1 minutes, and a requirement that the condition must exist for 2.5 mins (150 Seconds), what is the longest an application would be down before the alert triggers?

How is the "condition must exist for" counter handled? Is it based on the application polling, or the condition evaluation?

For example, if an application polls as down at 11:59:30 but the condition doesn't get evaluated until 12:00:00, will the "condition exists" counter start at 30 seconds (since polling down), or 0 seconds (since condition matched)?

mlandman · Answer

There is a property of "Last time UP" on components. I believe that this is used in a Component Alert to calculate the conditions longevity. So for you, max timings:

*  -00:00:00.0001 Component polls for condition and all is well
* 00:00:00 - "Bad Thing" happens on the node/application
* 149.99999999 seconds later, alert polls. Nothing to see here, sleep for 60 seconds
* 150 seconds after item 2, Component polls for condition and finds "Bad Thing", sets Warning condition and Last Time UP
* 60 seconds later alert polls and sees "Bad Thing" warning
* Alert looks and sees that "Bad Thing" happened 60 seconds ago, it is therefore not alertable
* 60 seconds later alert polls and sees "Bad Thing" warning
* Alert looks and sees that "Bad Thing" happened 120 seconds ago, it is therefore not alertable
* 60 seconds later alert polls and sees "Bad Thing" warning
* Alert looks and sees that "Bad Thing" happened 180 seconds ago, it is therefore alertable
* Fires an alert 
* Wait for the alert recipient's Minimum Time To Notice!!!

so...

150 + 60 + 60 + 60 = 5 1/2 minutes from "Bad Thing" to alert firing maximum

It could be as little  ~150+ seconds if all the start align in polling cycle timings

designerfx · Answer

I'd leave alert eval as low as possible, but is this right for what you're trying to do? You want the alert to trigger if something happens more than twice, right?  You can set this as a threshold on the application monitor itself (edit the template), and then have an alert based on the threshold status such as critical/warning/etc. Such as "greater than X/Y/Z for (specified number)" of polls?