Alert configuration for transaction down

There are three timers you need to consider here:

1. The playback interval - 5 mins- this is how often the system will run through the transaction (go through the playback steps) and record a status
2. The evaluation frequency of the alert - 5 mins - this is how often you want SolarWinds to check this alert condition, you probably want to drop this back down to 1 minute. 
3. The alert trigger wait time - 3 mins - this is how long the alert condition has to be true before the alert will trigger

Going with the timers in your original post:

• T0 - WPM polls the transaction; status = Up
• T1 - Alert checks the condition (this time could be anywhere between T0-T5 depending on when the alert first polled). Does not match down condition.
• T5 - WPM polls the transaction; status = Down
• T6 - Alert checks condition. Matches trigger condition, starts 3 minute timer.
• T10 - WPM polls transaction; status = Down
• T11 - Alert checks condition. 3 minute timer condition met.

You got the alert at T11 when you were expecting it at T9.

You can use a 5 min timer for the evaluation frequency but you may run into weird issues with timing. (i.e. if you are polling at T0,T5,T10 but the alert is checking at T4,T9,T14 - the transaction could be down for 4 minutes before the alert even checks it.)

(Edit: Thinking about it further I'm not 100% sure about the timer condition being met at T9 or T11 as I have never had someone use a condition shorter than the alert evaluation frequency...  happy for someone to correct me here)

------

If you used the default 1 minute evaluation:

• T0 - WPM polls the transaction; status = Up
• T0.5 - Alert checks the condition (this time could be anywhere between T0-T1 depending on when the alert first polled). Does not match down condition.
• T1.5 - Alert checks the condition. Does not match down condition.
• T2.5 - Alert checks the condition. Does not match down condition.
• T3.5 - Alert checks the condition. Does not match down condition.
• T4.5 - Alert checks the condition. Does not match down condition.
• T5 - WPM polls the transaction; status = Down
• T5.5 - Alert checks the condition. Matches trigger condition. Starts 3 minute timer.
• T6.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
• T7.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
• T8.5 - Alert checks the condition. Matches trigger condition. 3 minute timer condition met. Triggers alert  (all you've done here is delay the alert by 3 minutes)
• T9.5 - Alert checks the condition. Matches trigger condition. Alert already exists, does nothing.
• T10 - WPM polls transaction; status = Down
• T10.5 - Alert checks the condition. Matches trigger condition. Alert already exists.

From the above, with a timer condition shorter than the polling interval the system doesn't have time to collect a second metric of whatever you are monitoring. Therefore you should use a timer condition longer than the polling interval.

Recommendation 1 - if you want to put in a "condition must exists for" timer, make it longer than the polling interval. Otherwise you just delay the alert.

Recommendation 2 - Use a shorter evaluation frequency. 

There are three timers you need to consider here:

1. The playback interval - 5 mins- this is how often the system will run through the transaction (go through the playback steps) and record a status
2. The evaluation frequency of the alert - 5 mins - this is how often you want SolarWinds to check this alert condition, you probably want to drop this back down to 1 minute. 
3. The alert trigger wait time - 3 mins - this is how long the alert condition has to be true before the alert will trigger

Going with the timers in your original post:

• T0 - WPM polls the transaction; status = Up
• T1 - Alert checks the condition (this time could be anywhere between T0-T5 depending on when the alert first polled). Does not match down condition.
• T5 - WPM polls the transaction; status = Down
• T6 - Alert checks condition. Matches trigger condition, starts 3 minute timer.
• T10 - WPM polls transaction; status = Down
• T11 - Alert checks condition. 3 minute timer condition met.

You got the alert at T11 when you were expecting it at T9.

You can use a 5 min timer for the evaluation frequency but you may run into weird issues with timing. (i.e. if you are polling at T0,T5,T10 but the alert is checking at T4,T9,T14 - the transaction could be down for 4 minutes before the alert even checks it.)

(Edit: Thinking about it further I'm not 100% sure about the timer condition being met at T9 or T11 as I have never had someone use a condition shorter than the alert evaluation frequency...  happy for someone to correct me here)

------

If you used the default 1 minute evaluation:

• T0 - WPM polls the transaction; status = Up
• T0.5 - Alert checks the condition (this time could be anywhere between T0-T1 depending on when the alert first polled). Does not match down condition.
• T1.5 - Alert checks the condition. Does not match down condition.
• T2.5 - Alert checks the condition. Does not match down condition.
• T3.5 - Alert checks the condition. Does not match down condition.
• T4.5 - Alert checks the condition. Does not match down condition.
• T5 - WPM polls the transaction; status = Down
• T5.5 - Alert checks the condition. Matches trigger condition. Starts 3 minute timer.
• T6.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
• T7.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
• T8.5 - Alert checks the condition. Matches trigger condition. 3 minute timer condition met. Triggers alert  (all you've done here is delay the alert by 3 minutes)
• T9.5 - Alert checks the condition. Matches trigger condition. Alert already exists, does nothing.
• T10 - WPM polls transaction; status = Down
• T10.5 - Alert checks the condition. Matches trigger condition. Alert already exists.

From the above, with a timer condition shorter than the polling interval the system doesn't have time to collect a second metric of whatever you are monitoring. Therefore you should use a timer condition longer than the polling interval.

Recommendation 1 - if you want to put in a "condition must exists for" timer, make it longer than the polling interval. Otherwise you just delay the alert.

Recommendation 2 - Use a shorter evaluation frequency.