5 Replies Latest reply on Apr 9, 2019 2:37 PM by brscott@arcbtech.com

    Alert Silencing

    guse

      Hi all,

       

      Specific question here. We are using alerts to notify our on-call team when a production device goes offline outside of work hours. I have other issues with the alert (namely not being able to schedule who the alert goes out to and when for our rotating on-call schedule), but thats neither here nor there. The main problem i am having is actually getting the alert to shut off and not continually alerting until the device is brought back online. I will attach pictures of the current alert set up below.

       

      What I would like, is to be able to configure the alert to send 4 notifications. One for when the device(s) first goes offline, and 3 subsequent messages at 5 minute intervals repeating the first message. Then I would like the alert to STOP sending out alerts. Do I need to change to reset condition to exactly the time it would take to run the entire alert (30 mins. 15 mins for alert to be kicked off, plus 15 mins for each of the three 5 minute waiting periods) to something other than what it is? Is this just how it is? I am still relatively new to Solar Winds (6 months of use) so if any Veterans have some tips, I'd love to hear them! If you have any questions for me I'll be happy to answer as best I can.

       

      Thank you!

       

      Gus E.

        • Re: Alert Silencing
          brscott@arcbtech.com

          You cannot just set a custom reset to a 30 minute timer.  That will reset the alert but since the condition still exists, it will just start alerting again after the 15 minute wait time on the alert.

           

          You would need to do this in the trigger actions as an escalation to "acknowledge" the alert after 15 minutes, but I do not see an easy way to do that.

           

          On a side note, you could change the way you view this.  Send a notification and wait a reasonable amount of time say 1 hour for the re-notification.  Then they will continue to get notified until it's fixed.  My view is that if it's worth getting notified at night, it's worth the effort to fix it.  That's what "on call" means to me.

            • Re: Alert Silencing
              brscott@arcbtech.com

              Wait... I might have an idea, though I have not tried it. 

               

              Using escalations.

              You set the trigger action to happen only once, Do not repeat the action in the execution settings.

              The you set 2 more escalations 5 minutes apart, that do the same notification but also do not repeat.

                • Re: Alert Silencing
                  guse

                  Let me look into this. I'll try and remember to let you know how it works out once I've configured everything. Thanks for the idea!

                  • Re: Alert Silencing
                    guse

                    It doesn't look like I can configure how many times the action will run. Under "execution settings" I only have the option to "Do not execute this action if the alert has been acknowledged already" and "Repeat this action every X minutes until the alert is acknowledged".

                     

                    This got me thinking though... What if for the last escalation level I select the "Repeat this action every X minutes" option and set X minutes to something that would work out to be around 24hrs. That way another alert wouldnt be sent (in theory) and it is very likely that the issue with the specific device(s) will be taken care of within 24hrs. Once the device comes back online the alert should be reset automatically I believe.

                     

                    Let me know if you think that this could be a solution to the issue.