How to alert when an object no longer exists?

Question

If an array disappears, an event is created, stating that the object "...no longer exists. Data collection suspended."

That's fine, as far as it goes.  But how can we alert on this event?  The closest I've come is to alert if the size of the volume changes to zero bytes, but it seems like if there's an event specific to an object disappearing, there should be a way to also alert on that.

jaybone · Answer

@sja wrote:Hi @jaybone

Sorry you are right some type event you can alert on...

I think you could move the thread to "NPM Alert lab" there some SQL gurus that could make

custom SQL alert ..

SELECT TOP 100 EventID, EventTime, NetworkNode, NetObjectID, NetObjectValue, EngineID, EventType, Message, Acknowledged, NetObjectType, TimeStamp
FROM Orion.Events
Where Message like '%Volume% longer exists%'

Any chance to get right from the Volume "status" that has change from up to unknow?

As far as changing the criteria to status unknown, that doesn't really address the problem, unfortunately.  The issue is that Orion is no longer monitoring the volume.  Since it's no longer monitoring, everything about the node looks fine, at a glance - all green on the dashboard.  It's only when digging into the node that anything looks wrong.  And for some inexplicable reason, a situation where, "this is broke and we're not going to monitor it anymore" apparently isn't even an alertable event.  The only place that shows up is in the event log; nobody is staring at that event log all day, or... ever really, unless there's a problem, so it'll likely never be noticed.

It needs to be communicated to people in a format that people actually use (email, text message, etc.) that something just automagically isn't being monitored anymore, and seems nuts that doing so isn't a canned alert.

I guess I'll look into the custom SQL alert, but again, it kinda boggles the mind that such a thing isn't built in.

Thanks.

sja · Answer

Hi @jaybone

Sorry you are right some type event you can alert on...

I think you could move the thread to "NPM Alert lab" there some SQL gurus that could make

custom SQL alert ..

SELECT TOP 100 EventID, EventTime, NetworkNode, NetObjectID, NetObjectValue, EngineID, EventType, Message, Acknowledged, NetObjectType, TimeStamp
FROM Orion.Events
Where Message like '%Volume% longer exists%'

Any chance to get right from the Volume "status" that has change from up to unknow?

jaybone · Answer

Hi, thanks.  I'm not following you.  There is no such option.  "Alert on" has 135 possible things to choose from, but Events aren't one of them.  The only thing I'm seeing in there with the word "event" in it is "Auditing Events."

I'd think that changing the trigger condition from "Field" to "Event" would let me alert on this kind of thing, and maybe that's what you mean, but non of the events I've tried have actually worked.  I'm a loss as to what the trigger criteria should be.

E.g. setting things up like this:

doesn't trigger when the array disappears, taking the volume out with it, despite the canned "Volume no longer exists.  Data collection suspended" event.

"Stat collection" and "data collection" are different things, I guess, though I'm not sure how you can have stats without data.  None of the other options seem relevant, or I'm just missing something.

edit: And I should also state that I've tried using the "status has changed" option, which was a mistake, lol.  Every minute change to any volume's status (like, say it's 1% more full than it was last poll) triggers that, resulting in a flood of alerts, though none of the ones I observed were actually relegated to the volume disappearing.