Create an Alert Trigger if Two interfaces go down

[jmason2000]

Hi,

I want to create an alert that if two interfaces go down on certain devices, I would be able to send an email alert.  But it should only happen IF 2 interfaces go down, not one. 

So the Scope of the alert is:

Any node with the name containing ABC

And the trigger is

Interface name = wan 1

And

Interface name = wan2

and Interface is down

Would this work? Or would this end up triggering if it was on WAN1 on Node1 was down and Wan2 on Node 2 is down?

Any help would be greatly appreciated.

Scott

[d09h]

Another way to approach this would be with groups.  Groups can show best status/ worst status/ etc.  Membership in a group can be static/ dynamic/ a mix.  Your call whether it's more convoluted or less convoluted.  It would make your alerts cleaner and may have benefit outside of the alerting module (views you create/ reports you make).  I always found groups and custom properties to be more self-documenting and easier to maintain than hard-coded references in alerts/ reports/ views.  More extensible and saves time as your network grows more complex.  Since group functionality came so much later than custom property functionality, some of the older guys probably use them less than we should (guilty).

Key to this type of solution is to define your group to show 'best case' status rather than 'worst case' status so that both have to be down for the group to be down.

Further reading:  http://www.solarwinds.com/documentation/orion/docs/groupsanddependencies.pdf

[jameslindsay]

I have this same issue and I think even the accepted solution has issues. As was said, when using an advanced condition how can you be sure the alert is not tripping when Device 1 int A and Device 2 int B are down as opposed to the intended Device 1 int A and Device 1 int B? Groups are useful when dealing with a small number of devices. I have 1500 devices where is anyone of them has both int A and B down, I need an alert. I've used the Advanced condition with mixed results. Meaning, it seems to trip sometimes on a device that has ether one or the other int up. And the reset condition also seems a bit flaky.

From a SWIS standpoint, this works but again, the reset condition is questionable. The alert should reset when either int A or B on the same device is up.

SELECT E0.[Uri], E0.[DisplayName]
FROM Orion.NPM.Interfaces AS E0
WHERE ( ( ( E0.[Node].[Vendor] = 'XXXXXX' ) AND ( E0.[Node].[Status] = '1' ) ) AND ( ( E0.[IfName] = 'ipsec-primary' ) AND ( E0.[Status] = '2' ) ) )
AND E0.[Node].[NodeID] IN
(SELECT E0.[Node].[NodeID]
FROM Orion.NPM.Interfaces AS E0
WHERE ( ( ( E0.[Node].[Vendor] = XXXX' ) AND ( E0.[Node].[Status] = '1' ) ) AND ( ( E0.[IfName] = 'ipsec-secondary' ) AND ( E0.[Status] = '2' ) ) ))

This methos has some issues in the sense that you are now limited to doing active data lookups with the node attributes you may want to pass to an ITSM process as well as email alerts.

[blsanner]

That will NOT work as you have it set.  Basically, what you are saying is that a single interface has a name of both wan 1 and wan2, which obviously cannot be true at any point in time.  In order to get what you are looking for, you would have to use the Advanced Option of "Enable complex conditions...".  Then, you would set up two separate conditions that would look at each interface with an AND operator between the two.  It would be something like this:

---------------------------------------------------

Primary Section

Interface Name = wan 1

Interface Status = Down

---------------------------------------------------

AND

---------------------------------------------------

Secondary Section

Interface Name = wan2

Interface Status = Down

---------------------------------------------------

[john.ta]

blsanner brings up good point, I apologize, since the detail was vague and no screenshots I had assumed this was already a complex condition.  And we all know what happens when you assume...

[jmason2000]

Thanks guys!  that is what I figured.  I was hoping there was going to be a way to apply it to a group of devices that all had the same interface names.

But I didn't  think there would be.

Scott

[sgtp]

you could always put those interfaces into a group and alert based on that.

[jmason2000]

Yeah... I know that.  But that would still not work for applying the alert to Multiple devices.  But thanks.

[d09h]

Why not?  Alert on the group (of interfaces) vs. the node or the interface.  Make as many groups as you need.  Would this not scale for you?

[jmason2000]

Okay based on that, how do you ensure that it is looking at the SAME Device.  Otherwise it could be two different Cisco Devices that went down but each only has one of the Interfaces down.  How do I make sure that it is the SAME device for both Complex alerts?

--------------------------------------------------

Scope of Alert:Vendor = Cisco

Primary Section

Interface Name = wan 1

Interface Status = Down

---------------------------------------------------

AND

---------------------------------------------------

Scope of Alert: Vendor = Cisco

Secondary Section

Interface Name = wan2

Interface Status = Down

---------------------------------------------------

[blsanner · Accepted solution]

Under the scope of the alert section, set it in both sections to just look at a particular node something like the following:

However, if you have multiple devices that you want to use this (or a similar) alert for, I see two options:

1) Create a separate alert for each device.  This would be pretty straightforward: create one and then copy the alert and edit the copy to reflect a different node name.

2) Create the alert as a custom SQL or custom SWQL alert which would allow you to compare the node names for the interfaces to make sure they are the same.

[john.ta]

Because you used AND operator instead of OR operator, the alert should only trigger if both are down.

[jmason2000]

Okay.  So should I be alerting on the NODE with two interfaces named WAN1 and WAN2 down?  Or should I be alerting on the Interface where the interface name = Wan1 and Interface is down AND Interface Name = WAN 2 and Interface is down?