6 Replies Latest reply on Dec 1, 2017 9:22 AM by KMSigma

    Timeframe on muting alerts?

    mharvey

      I've had a few of our techs tell me they've muted alerts in SolarWinds using the new feature only to have those same devices trigger a node down alert within a few days of this being set.  They are not using the scheduler, they are doing this ad-hoc.  Is there a default amount of time where SolarWinds will unmute a device that has been muted?   Is there somewhere in the database I can look and correlate when a device is muted?

        • Re: Timeframe on muting alerts?
          aLTeReGo

          mharvey  wrote:

            Is there a default amount of time where SolarWinds will unmute a device that has been muted?  

          devices are not unmuted automatically or as part of any scheduled routine.

           

          Is there a default amount of time where SolarWinds will unmute a device that has been muted?   Is there somewhere in the database I can look and correlate when a device is muted?

           

          If you're using the new Entity Selector in Core 2017.3, it will show you which nodes have been muted.

           

          1 of 1 people found this helpful
          • Re: Timeframe on muting alerts?
            mesverrum

            Try this in a custom query resource, shows all muted or unmanaged nodes and when they will be coming back online.

             

            SELECT

            'Unmanaged' as [Status]

            ,n.Caption AS [Node]

            ,tostring(tolocal(n.UnManageFrom)) AS [From]

            ,case when n.UnManageUntil is null or n.UnManageUntil = '9999-01-01 00:00:00' then 'Not set'

            else tostring(tolocal(n.UnManageUntil)) end AS [Until]

            ,case when n.UnManageUntil is null or n.UnManageUntil = '9999-01-01 00:00:00' then '-'

            else tostring(daydiff(getutcdate(), n.unmanageuntil)) end as [Days Left]

            ,n.DetailsURL AS [_LinkFor_Node]

            ,'/Orion/images/StatusIcons/Small-' + n.StatusLED AS [_IconFor_Node]

            ,CASE

            WHEN ae.accountID IS NULL THEN 'Audit Log Not Found'

            ELSE ae.AccountID

            END AS [Account]

             

            FROM

            Orion.Nodes n

            JOIN (

                SELECT rec.NetObjectID, max(rec.timeloggedutc) as recent

                FROM Orion.AuditingEvents rec

                WHERE rec.auditingactiontype.actiontype = 'Orion.NodeUnmanaged'

                group BY rec.NetObjectID) mostrecent ON mostrecent.NetObjectID = n.NodeID

            JOIN (

                SELECT ae.NetObjectID, ae.AccountID, ae.timeloggedutc

                FROM Orion.AuditingEvents ae

                WHERE ae.auditingactiontype.actiontype = 'Orion.NodeUnmanaged') ae ON ae.NetObjectID = n.NodeID and ae.timeloggedutc=mostrecent.recent

             

            WHERE n.Status = 9

             

            union all

             

            (SELECT

            'Muted' as [Status]

            ,n.caption

            ,tostring(tolocal(SuppressFrom)) as [From]

            ,case when SuppressUntil is null or SuppressUntil = '9999-01-01 00:00:00' then 'Not set'

            else tostring(tolocal(SuppressUntil )) end AS [Until]

            ,case when SuppressUntil is null or SuppressUntil = '9999-01-01 00:00:00' then '-'

            else tostring(daydiff(getutcdate(), asup.SuppressUntil)) end as [Days Left]

            ,n.DetailsURL AS [_LinkFor_Node]

            ,'/Orion/images/StatusIcons/Small-' + n.StatusLED AS [_IconFor_Node]

            , ae.AccountID AS [Account]

             

             

            FROM Orion.AlertSuppression asup

            join orion.nodes n on asup.entityuri=n.uri

            join (

                SELECT ae.NetObjectID, max(ae.timeloggedutc) as recent

                FROM Orion.AuditingEvents ae

                WHERE ae.auditingactiontype.actiontype in ('Orion.AlertSuppressionChanged','Orion.AlertSuppressionAdded')

                group BY ae.netobjectid) mostrecent ON mostrecent.NetObjectID = n.NodeID

            join (

                SELECT ae.NetObjectID, ae.AccountID, ae.timeloggedutc

                FROM Orion.AuditingEvents ae

                WHERE ae.auditingactiontype.actiontype in ('Orion.AlertSuppressionChanged','Orion.AlertSuppressionAdded')

                Order BY ae.TimeLoggedUtc desc) ae ON ae.NetObjectID = n.NodeID and ae.timeloggedutc=mostrecent.recent

            )

             

            ORDER BY [node] asc, [status] desc

            2 of 2 people found this helpful
            • Re: Timeframe on muting alerts?
              mharvey

              Thanks for the information.  I'll still have to dig then to find why a device that was muted started triggering one of our Node Down alerts.  I'm able to find audit events showing the node alerting was muted, but I'm unable to find a correlating event showing the alerting for the node was re-enabled prior to the alert triggering and alerting our on-call team after hours.

                • Re: Timeframe on muting alerts?
                  mesverrum

                  I heard a few reports from people that when muting was first introduced it wasn't working for them 100% of the time.  I never had that problem so I can't confirm if there was a bug or something else at play.  Are you running NPM 12.2 yet?

                    • Re: Timeframe on muting alerts?
                      mharvey

                      yeah, 12.2 with HF2 to fix another issue we ran into with Mount Point monitoring.

                        • Re: Timeframe on muting alerts?
                          KMSigma

                          You can also use this report that I wrote (or parse it down into a resource).

                           

                          Current Muted Alerts (v5) [SAM & NPM]

                           

                          Resource Content: (for SAM & NPM)

                          SELECT DISTINCT
                                    CASE
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%' AND [EntityUri] NOT LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/%'
                                      THEN [N].[Caption]
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/Interfaces/InterfaceID=%'
                                      THEN [I].[FullName]
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/Applications/ApplicationID=%'
                                      THEN [AA].[FullyQualifiedName]
                                    ELSE 'SomethingElse'
                                END AS [Element],
                                [AE].AccountID AS [By],
                                ToLocal([SuppressFrom]) AS [Start],     
                                ToLocal([SuppressUntil]) AS [End]
                          FROM Orion.AlertSuppression AS [AlertSup]
                          LEFT OUTER JOIN Orion.Nodes AS [N]
                            ON [AlertSup].[EntityUri] = [N].[Uri]
                          LEFT OUTER JOIN Orion.NPM.Interfaces AS [I]
                            ON [AlertSup].[EntityUri] = [I].[Uri]
                          LEFT OUTER JOIN Orion.APM.Application AS [AA]
                            ON [AlertSup].[EntityUri] = [AA].[Uri]
                          LEFT OUTER JOIN Orion.AuditingEvents AS [AE]
                            ON [AE].AuditEventMessage LIKE CONCAT('%', CASE
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%' AND [EntityUri] NOT LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/%'
                                      THEN [N].[NodeName]
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/Interfaces/InterfaceID=%'
                                      THEN [I].[InterfaceCaption]
                                    WHEN [EntityUri] LIKE 'swis://%/Orion/Orion.Nodes/NodeID=%/Applications/ApplicationID=%'
                                      THEN [AA].[Name]
                                    ELSE 'Wrong'
                                END, '%') AND [EntityUri] LIKE CONCAT('%=', [AE].NetObjectID)
                          INNER JOIN Orion.AuditingActionTypes AS [AT]
                            ON [AE].ActionTypeID = [AT].ActionTypeID
                          WHERE [AT].ActionType LIKE 'Orion.AlertSuppression%'
                          ORDER BY ToLocal([SuppressFrom])

                           

                          It looks like:

                          1 of 1 people found this helpful