Alert Escalation Questions

Question

Hello,

I'm trying to set up some trigger actions for some alerts to use a "follow the sun" kind of notification with escalations.

e.g.

Trigger 1 - 6AM -> 5PM - Group 1 receive notification

Trigger 2 - 5PM -> 7PM - Group 2 receive notification

Trigger 3 - 5PM -> 11:59PM - Group 3 receive notification

Trigger 4 - 12AM -> 6AM - Group 4 receive notification

Trigger 5 - 6AM -> 5PM - (30mins after alert) Escalation to Manager 1

Trigger 6 - 5PM -> 6AM - (30mins after alert) Escalation to On Call Phone

I have a couple of questions based on this requirement.

I have set up the 6 triggers within the same Alert Rule, and I have set the "execute this action only between specific hours" setting to limit the times that they will be used.

So, I'm wondering, if I have a node go down at 4pm (trigger 1 will happen) and then 30 mins later, it hasn't been acknowledged (trigger 5 will happen) and at 5:01pm it still hasn't been acknowledged, what will happen?

I can see a couple of scenarios...

a) Nothing will happen (this is what I want) - Unlikely

b) Trigger 2, 3 and 5 will happen at the same time - Maybe

c) Trigger 2 and 3 will happen, and 30 mins later trigger 5 will happen - Maybe

Another question...

Does the "execute this action only between specific hours" box understand the idea of "the next day" ?

e.g.

Say I want an action to be used between 5pm Monday night and 10am Tuesday morning...  Do I need to set a trigger rule from 5:00pm Monday -> 11:59pm Monday and from 12:00am Tuesday to 10:00am Tuesday? or can I set the times from 5pm->10am and tick the Monday box?

I hope I have explained this well enough.

Thanks,

Alex

mongie · Answer

From my testing it looks like its not actually happening like this.

If I create a test alert, it runs the actions as they are "enabled". I think this relates to the "this alert should be checked every..." setting.

mrxinu · Answer

When an alert is fired, it runs the actions that are "applicable" at that time, and then stops.

So, it would send any e-mails that were enabled at that time, and start a timer for anything that is enabled at that time, to be run after a delay.

Does that sound right?

Right, I think you've got it now.

mrxinu · Answer

Hmm...

I'm not sure if I undestand what you're saying...

I tried to test my first question and I think I came up with the answer i most expected.

If an alert is triggered, and remains "un-acknowledged" and "active", the triggers will go off once they meet their rules (e.g. once it hits the "start" time for an action, if the alert is already active, it will execute that action)

That's not what I would have expected to happen.  Let's say you have 3 alert actions: one that has no delay but only happens during the PM hours, another with no delay but only happens during the AM hours, and another with a 30-minute delay that only happens during the PM hours I would expect the following:

* The alert trigger condition is satisfied.
* All alert triggers that should fire now, happen now.
* 30 minutes later, the alert actions that should fire, fire then.

There's no reason that actions without any delay should be evaluated *again* once the initial trigger condition has fired.

As for "the alerts dont understand the 24hr barrier, does that mean that setting an alert to run from 6PM to 5:59AM will run from 6pm (say) Tuesday night to 6AM Wednesday morning?

There are the days of the week and the time of day.  If you set the times to 6pm to 5:59am it's going to be in effect from 6pm to 11:59am.  It won't understand that you meant 5:59am the next morning.