I have a strange issue with SolarWinds that came to a head when a bunch of our backup circuits went down at once. At each of our remote branches we have two routers, each with it's own MPLS circuit. We do not load balance because our backup circuit is a 4G wireless connection, so we just use a VRRP address for failover. When the backup router goes down about half the switches in the branch show as down. I can ping these switches from anywhere else in the company, except SolarWinds. I can ping from VPN, my desk, at the branch itself, and even from a VM on the same server as Orion. When this happens, I cant even ping from the command prompt on the server. I ran a PCAP from the core switch at these locations, and the pings from SolarWinds are arriving, and getting a response from the switch.
When I log into these devices, they all have the backup router's IP in their ARP table even though they should only have the VRRP address. The switches that don't have the issue, do not have the IP in their ARP table. If I clear the ARP SolarWinds can immediately start pinging.
Anyone have any insight on what might be going on here? Is it possible SolarWinds tries to poll across both circuits somehow?
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process. Learn more today by joining now.