cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

Alert still firing when exclusion conditions are met

Jump to solution

Sorry if this issue has already been addressed in other posts, but I couldn't find anything on it.

In my network we have a couple of hundred ether-channels configured. 90+% of them are 4 Gbps, but a few are running at 1, 2, or 3 Gbps.

My goal is to create an alert that tells me when an ether-channel's speed drops below 4 Gbps. However, I obviously want to exclude alerts on the ones that are designed to be slower.

With that goal in mind, I renamed all of the ether-channels in this format:

1. They contain "** Port-Channel ** " in the interface description on the switch
2. If they are designed to be slower than 4 Gbps then I included the speed in the interface description (I.e., ** Port-Channel to Core 1 - 2 Gbps **)

Then I created an alert that looks like this:

Property to monitor: Interface

Trigger alert when ALL of the following apply
   Interface speed is less than 4000000000
   Full name contains Port-Channel
   Interface status is equal to Up
   Interface status is not equal to Unmanaged
      Trigger Alert when NONE of the following apply
         Full name contains 3 Gbps 
         Full name contains 2 Gbps
         Full name contains 1 Gbps 

The first part of the alert works fine. If the speed is less than 4 Gbps I get an email. The problem is that I am getting alerts for unmanaged ports and ports that have 3, 2, or 1 Gbps in the description.

So I have two questions:

1. Why am I getting alerts on unmanaged interfaces?
2. How do I stop getting alerts for slower ether-channels? Is my exclusion setup improperly, or is this something I need to contact Solarwinds support about?

Thanks!
- Josh 

0 Kudos
1 Solution

I tried changing the "none" to "not all" and that seems to have fixed the problem =). If I notice it being quirky then I'll try your second suggestion.

Thanks for your assistance...I just hope that the issue with the none/not all thing is sorted out soon. I'm one of those people who likes to try to figure out something on my own before I turn to forums for help (I figure you learn better that way) and I wasted at least 3 hours on something that should have been working but wasn't. Anyway, thanks again!

- Josh

View solution in original post

0 Kudos
12 Replies
Level 19

It looks like the last condition (Trigger Alert when NONE of the following apply
) is embedded one level too low. Should be at the same level as the ones above it.

Trigger alert when ALL of the following apply
   Interface speed is less than 4000000000
   Full name contains Port-Channel
   Interface status is equal to Up
   Interface status is not equal to Unmanaged
   Trigger Alert when NONE of the following apply
         Full name contains 3 Gbps 
         Full name contains 2 Gbps
         Full name contains 1 Gbps 

0 Kudos

Thanks for the response, mcbridea. Can you explain what you mean about it being at the same level? The "Trigger Alert when NONE of the following apply" is a complex condition. The SW alert creator nested it the way that it did. Are you saying it needs to be ordered differently?

Questionario, you're right about the unmanaged portion of the alert. It shouldn't be needed. However, I have a similar alert for 1Gbps interfaces and it alerted me on unmanaged interfaces until I added that condition. (It actually will see a speed change during polling, not just during rediscovery).

This is the 1 Gbps, which *is* working:

This is the port-channel one, which is not working:

Both of these alerts are live, I am not just testing against them.

Thanks again for the help everyone...

0 Kudos

There are a couple of different ways to approach this. The None and Not all booleans are actually a bit screwy so you could try replacing the None witn Not All. I know that does not makes sense logically but because the way those 2 options are implemented it may work.

The second option is to assign a Custom Property to the 1-3 GB interfaces and add that property to the alert. ~ "Custom Property does not equal not 4" where you have created a not 4 property and assigned it to the 1-3 GB interfaces.

Example is here - http://www.solarwinds.com/documentation/Orion/docs/UnderstandingOrionAdvancedAlerts.pdf

I tried changing the "none" to "not all" and that seems to have fixed the problem =). If I notice it being quirky then I'll try your second suggestion.

Thanks for your assistance...I just hope that the issue with the none/not all thing is sorted out soon. I'm one of those people who likes to try to figure out something on my own before I turn to forums for help (I figure you learn better that way) and I wasted at least 3 hours on something that should have been working but wasn't. Anyway, thanks again!

- Josh

View solution in original post

0 Kudos

I am not sure if it is a "bug" or just an issue of converting SQL logic to english yet try to stay concise.

In the future, you could try changing your alert to a Custom SQL and look at how the SQL is being generated and try to figure out the logic from there.

Maybe there should be a tab that contains the un-editable SQL and a run query now button.  That could even replace the the "test" piece that doesn't work quite right.

0 Kudos


I am not sure if it is a "bug" or just an issue of converting SQL logic to english yet try to stay concise.

In the future, you could try changing your alert to a Custom SQL and look at how the SQL is being generated and try to figure out the logic from there.

Maybe there should be a tab that contains the un-editable SQL and a run query now button.  That could even replace the the "test" piece that doesn't work quite right.



That's a good idea. I'm guessing bug because the same type of conditions, only with different wording, works fine for a physical interface. It could be the fact that it's a virtual interface that's the issue, maybe it didn't like some of the characters, etc. I had a similar issue in Kiwi syslog where I was telling it to look for messages with a certain string and then stop processing them. That string happened to have an "=" sign in it, and it caused it to stop processing ALL messages, even though I had formatted it correctly (in quotes, etc). No rhyme or reason to that I could think of for that problem. Removing the "=" fixed the problem. So it could be something like that going on here.

Anyway, I've spent too much time on it already so I'm not going to pursue it further, but if it pops up again then running a query is a great idea.

0 Kudos

the thing with the none/not all came to mind as well but I was thinking that it would not give false alerts but instead not alert at all which is why I didnt mention it.

I already had a discussion about this here on the forums and according to the outcome of the discussion which was verified by mcbridea, the "none" should have been the correct choice for your alert.

The discussion only led me to more confusion as noone even at solarwinds was ever really sure on how the log works so I decided not to use none or not all even if it means some alerts can't be done (unless non or not all only contain one condition in which case their meaning would not matter).

 

Here is the link to the discussion:

0 Kudos

Yes - it looks like it's a case by case thing. I look at the alert logic in the database to see which one will work.

0 Kudos

actually I took a look at discussion I posted and the verified answer from ebradford still confuses me but the post right after from Andy (mcbridea) which shows the exact SQL makes more sense to me... according to the Andy's post (showing the SQL output), the NOT ALL condition would be correct for you...

that explains everything, also why you received those false alerts on other speeds (although I still don't get why you would get alerted on unmanaged interfaces)

0 Kudos

In a meeting right now so haven't been able to read the link that Questionario posted, but I'm gathering the issue is with how the none / not all statement translates into SQL? If that's the case, why not create a simple condition of "does not contain"? Then you could do something like this:

Trigger when all of the following apply
   Full name contains **
   Full name contains Port-Channel
   Full name does not contain 100 Mbps

It seems to me that this would make creating triggers much more intuitive and may also solve some of the problems surrounding the none / not all complex condition.

0 Kudos

yes and the "does not contain" has been a requested feature for a long time now but it has never found its way into a release... 😞

0 Kudos
Level 14

first of all, once you get this to work it will not alert you immediately as interface speed is only polled during a rediscovery as far as I know... and that can take quite a while (for us its more than half an hour for sure), depending on how you set the rediscovery... so dont expect to get alarmed right when it happens!

 

You would not need the "Interface status is not equal to Unmanaged" statement though as the previous one already requires it to have a status of "Up".

Therefore I dont know why you would get alerted on unmanaged interfaces or even on lower speed ones if they contain the string you defined in the full name...

I would open a support case with solarwinds if you really get alerted from this alert for interfaces you excluded here without using the "Test Alert"-feature... (you didnt leave out any of the conditions here, did you?)

0 Kudos