I have a strange issue with SolarWinds that came to a head when a bunch of our backup circuits went down at once. At each of our remote branches we have two routers, each with it's own MPLS circuit. We do not load balance because our backup circuit is a 4G wireless connection, so we just use a VRRP address for failover. When the backup router goes down about half the switches in the branch show as down. I can ping these switches from anywhere else in the company, except SolarWinds. I can ping from VPN, my desk, at the branch itself, and even from a VM on the same server as Orion. When this happens, I cant even ping from the command prompt on the server. I ran a PCAP from the core switch at these locations, and the pings from SolarWinds are arriving, and getting a response from the switch.
When I log into these devices, they all have the backup router's IP in their ARP table even though they should only have the VRRP address. The switches that don't have the issue, do not have the IP in their ARP table. If I clear the ARP SolarWinds can immediately start pinging.
Anyone have any insight on what might be going on here? Is it possible SolarWinds tries to poll across both circuits somehow?