Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Alert conditions help!

I'm trying to write an alert that will notify me any time any nodes go down, except a certain group of servers, all of which are named with a similar prefix based on their region.

My alert works fine if I only include one prefix in the condition, like such:

Trigger Alert when all of the following apply:

    Node Status is not equal to Up

    Trigger Alert when none of the following apply:

        Node Name contains PREFIX_EAST

If I run that, it works fine. I'm emailed about nodes that are down that are not named PREFIX_EAST_1 and so on. However, if I add another condition like so:

Trigger Alert when all of the following apply:

    Node Status is not equal to Up

    Trigger Alert when none of the following apply:

        Node Name contains PREFIX_EAST

        Node Name contains PREFIX_WEST

The alert starts to notify me of everything, including things like PREFIX_EAST_1 which it didn't before. Anyone run into anything similar?

Find more posts tagged with

Accepted answers

All comments

skirven

I ran into a similar issue whereby I wanted to NOT alert on a subset of interfaces. What I finally did was create an "Exclude" Custom Property in the Nodes table, and then marked them with text and did a "Exclude not empty" condition for the ones I did want to alert on. I really think the Alert Supression is either 1) not working like I think it should, or 2) just plain broken.

glyngholm

Thanks for the reply. I have a feeling you're right...I forgot to mention in my OP that on my first try I put the "Node Name contains PREFIX" stuff into a "Suppress alert when any of the following apply" group. When I did that it just supressed the alert completely.

patrick.hubbard

Thanks for the details. I'm wondering what the AND/OR/IN condition looks like. Is there any chance you could post the generated SQL for that alert from the DB?

glyngholm

The nonworking query:

SELECT Nodes.NodeID AS NetObjectID, Nodes.Caption AS Name FROM Nodes WHERE ( (Nodes.Status <> '1') AND ( NOT (Nodes.Caption LIKE '%PREFIX_EAST%') OR NOT (Nodes.Caption LIKE '%PREFIX_WEST%')) )

There's something that doesn't look right with the OR NOTs and the parentheses at the end, but I'm not very good at SQL unfortunately. If I run the query in query analyzer it does return everything including those I'm trying to exclude, so that's definitely the problem.

I tweaked the query in query analyzer:

SELECT Nodes.NodeID AS NetObjectID, Nodes.Caption AS Name
FROM Nodes
WHERE ((Nodes.Status <> '1') AND ( NOT ((Nodes.Caption LIKE '%PREFIX_EAST%') OR (Nodes.Caption LIKE '%PREFIX_WEST%'))))

and that seems to work...can I just put that into the SQL table or will the alert engine mess it up next time I open the alert anyway?

patrick.hubbard

Quick work glyngholm, and thanks for taking a pass at it in query analyzer. My guess, (hopefully somewhat educated), is that the SQL is being parsed and dynamically generated in the alert condition editor, and executed in the Service. This would mean that it should work fine if you pasted the query back into the DB query column, as long as you don't then open it- or at least re-save it- in the alert builder.

This doesn't explain why it's not correct when generated, and we'll look into that. Thanks for the question and pulling the query details.

patrick.hubbard

Looking at the nested NOT logic, there is one possibility that you might want to check, and could do by opening both versions in the alert builder. In the topmost element of the alert condition list you can specify how the conditions are aggregated, all, any, none, etc. and I'm wondering what you selected there. If it's set to "all", and then the individual conditions are <> not equal (!=), then it may be wrapping it all up in that outer NOT and OR-ing the element list.

glyngholm

Ok, I'm pretty sure Solarwinds doesn't create the SQL correctly if you use multiple statements in a "none" group...for example, if I create an alert that looks like such:

Trigger Alert when NONE of the following apply:
Node Status is equal to Up

It succesfully returns all nodes in a down or unknown state. The SQL looks like this:

SELECT Nodes.NodeID AS NetObjectID, Nodes.Caption AS Name
FROM Nodes
WHERE (NOT(Nodes.Status = '1'))

Pretty straightforward. However, if I just add another condition to the alert, like so:

Trigger Alert when NONE of the following apply:
Node Status is equal to Up
Node Name contains DOWN_SERVER_1.

Ideally, this should return all my down servers except the one I specified (DOWN_SERVER_1). However, what actually happens is that the alert returns every single node, regardless of status. The SQL generated is

SELECT Nodes.NodeID AS NetObjectID, Nodes.Caption AS Name
FROM Nodes
WHERE (NOT(Nodes.Status = '1') OR NOT (Nodes.Caption LIKE '%DOWN_SERVER_1%'))

Running that in query analyzer also returns every single node. Looking at the query it kind of makes sense - it's selecting anything that's either not up or not named what I want. What it should be doing is selecting everything that's BOTH not up and not named what I want. Thus, if I change that last OR to an AND:

SELECT Nodes.NodeID AS NetObjectID, Nodes.Caption AS Name
FROM Nodes
WHERE (NOT(Nodes.Status = '1') AND NOT (Nodes.Caption LIKE '%DOWN_SERVER_1%'))

The query returns the right set of data - everything that is not up except DOWN_SERVER_1. It's possible my logic on what Solarwinds is trying to do is screwy, but I'm pretty sure this behavior isn't correct. It'd be nice if we could just plug SQL queries directly into the alerts from the alerting engine.