Instant notification when a node stops responding?

Hello. I have a question that I can't seem to find an answer to. I am using Orion NPM 9.5 SP2. I monitor around 70 servers, the majority with SNMP but some only with ICMP. I have set up an alert that sends me an e-mail if a node "goes down". However, in my timing tests, I don't receive the notification until anywhere from 2 - 4 minutes after the node is down. I have modified the default values under the Polling Settings screen to 10 Seconds for the "Default Node Poll Interval", 15 Seconds for the "Default Interface Poll Interval" and 3 Seconds for the "Node Warning Level (Mark devices as down that have not responded for configured time.)". So, from the sounds of that configuration, I should receive the alert within 45 seconds of the node going down (which, in my opinion, is too long).

My question: Is there any way to set-up a constant ICMP/ping to all of my monitored server and base a "down" threshold on that. I would like it to send ICMP packets every second of the day (just like I could do a ping -t from my Windows PC) to all servers I monitor and if, say, 5 ICMP packets time-out, I consider the node as down. Once it starts responding again, I consider the node up. Each time it should send me an e-mail telling me of the status change. Is something like this possible using Orion NPM??? Thanks in advance,

Frank

Find more posts tagged with

Alert

polling_settings

node_down

email_alert

Accepted answers

All comments

MagnAxiom

I can see scenerios where I might want to have "real time" monitoring of high profile network devices. I do think that expecting Orion on a single box to poll every device you monitor every second would kill the box.

I think there might be more optimization that could be done with the polling engine to reduce your time from 45 seconds, but I don't know of any "continual ping" like system in orion. Orion basicly round robin's the whole list of devices it is monitoring. SNMP takes more time to poll and process than an ICMP ping response, so I could possibly see 2 deployments of Orion, one dedicated exclusivly to ICMP polling, and another for SNMP interface/data gathering. My understanding is you could have the ICMP polling system report all its data to the other server for one place to look up data.

This is of course the guesses of a relativly new SolarWinds Orion operator.

mcbridea

Hi Firewall,

ICMP is not a reliable protocol and is given low preference on most networks. As your network gets busy ICMP packets can get dropped. When ICMP is used for near real-time monitoring the chances of loosing packets goes up and so you will get false positives for a server down. For these reasons I don't recommend using it do determine realtime status. I have found that SNMP trap and syslog are better tools for determining an issue near real-time. Some servers have the ability to send a last dying breath trap for a full system failure but of course this won't help if the failure is equivalent to an unplugged device. I have found that it is normally a component that fails and not the box as a whole. Traps and syslog work well for this. Also checkout APM 3.0 if you want to monitor application components.

Hope this helps.