Polling testing, for environments with HA and APE

yaquaholic

Create a poller test function, testing each node, from each poller and reporting back those that fail any communication.

Effectively something like a port scanner run on each poller, that can extract a list of nodes from the Orion database and populate back a table of its findings

From this we could then see where the network security/configuration is stopping things from working.

Some intelligence, such as if the device responds to either (ICMP + SNMP) OR (ICMP + WMI), from all known HA IP addresses and all known APEs, then the device will be marked as a pass, but then poll known ports used by Orion, agent ports, SSH, common database and other ports used in Orion to establish if all monitoring will be successful.

This should be part of the node adding process, so that Orion checks that the newly added node can be polled from primary/secondary (under HA) and from each APE.

To have this as retrospective tool (to review those already added to the DB), would also be very handy.

The reason for this, is that we operate one Orion platform for over 40 customers, so reviewing the ACLs is not a task the network guys undertake without good reason and we frequently are unaware which network rules are

not correctly in place, until something goes wrong and we start shifting the polling load around.

Credit nust go to mr.e​​ and his post https://thwack.solarwinds.com/ideas/9508#comment-290898

dsimpkins

We developed a workaround for this using a SWIS, SNMPGet and a powershell script to pull it all together.

The script works like this:

Using SWIS retrieve a list of polling engines and the nodes being monitored

For each node:   

     Compile the SNMPget command string

     For each polling engine:

          Run the SNMPGet command string and return the results

All of this takes a while to run and comes back with a CSV file that can be filter and sorted to show which poller responded for each node.

We are using this data to balance load across our polling engines automatically without the risk of putting a node on a polling engine that can't monitor it.