Alert configuration for transaction down

Hi,

I have doubt regarding alert configuration for transaction down. I have set the playack interval for transaction to 5 minutes, evaluation frequency for alert to 5 minutes and alert wait time(trigger condition exists for) to 3 minutes. Is this configuration correct?

If not, can anybody suggest the correct configuration since i am not getting alerts at right time.

  • There are three timers you need to consider here:

    1. The playback interval - 5 mins- this is how often the system will run through the transaction (go through the playback steps) and record a status
    2. The evaluation frequency of the alert - 5 mins - this is how often you want SolarWinds to check this alert condition, you probably want to drop this back down to 1 minute. 
    3. The alert trigger wait time - 3 mins - this is how long the alert condition has to be true before the alert will trigger

    Going with the timers in your original post:

    • T0 - WPM polls the transaction; status = Up
    • T1 - Alert checks the condition (this time could be anywhere between T0-T5 depending on when the alert first polled). Does not match down condition.
    • T5 - WPM polls the transaction; status = Down
    • T6 - Alert checks condition. Matches trigger condition, starts 3 minute timer.
    • T10 - WPM polls transaction; status = Down
    • T11 - Alert checks condition. 3 minute timer condition met.

    You got the alert at T11 when you were expecting it at T9.

    You can use a 5 min timer for the evaluation frequency but you may run into weird issues with timing. (i.e. if you are polling at T0,T5,T10 but the alert is checking at T4,T9,T14 - the transaction could be down for 4 minutes before the alert even checks it.)

    (Edit: Thinking about it further I'm not 100% sure about the timer condition being met at T9 or T11 as I have never had someone use a condition shorter than the alert evaluation frequency...  happy for someone to correct me here)

    ------

    If you used the default 1 minute evaluation:

    • T0 - WPM polls the transaction; status = Up
    • T0.5 - Alert checks the condition (this time could be anywhere between T0-T1 depending on when the alert first polled). Does not match down condition.
    • T1.5 - Alert checks the condition. Does not match down condition.
    • T2.5 - Alert checks the condition. Does not match down condition.
    • T3.5 - Alert checks the condition. Does not match down condition.
    • T4.5 - Alert checks the condition. Does not match down condition.
    • T5 - WPM polls the transaction; status = Down
    • T5.5 - Alert checks the condition. Matches trigger condition. Starts 3 minute timer.
    • T6.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
    • T7.5 - Alert checks the condition. Matches trigger condition. 3 minute timer continues.
    • T8.5 - Alert checks the condition. Matches trigger condition. 3 minute timer condition met. Triggers alert  (all you've done here is delay the alert by 3 minutes)
    • T9.5 - Alert checks the condition. Matches trigger condition. Alert already exists, does nothing.
    • T10 - WPM polls transaction; status = Down
    • T10.5 - Alert checks the condition. Matches trigger condition. Alert already exists.

    From the above, with a timer condition shorter than the polling interval the system doesn't have time to collect a second metric of whatever you are monitoring. Therefore you should use a timer condition longer than the polling interval.

    Recommendation 1 - if you want to put in a "condition must exists for" timer, make it longer than the polling interval. Otherwise you just delay the alert.

    Recommendation 2 - Use a shorter evaluation frequency.