I have two individual Cisco ISR4221 routers that each have been installed over a year. The problem I am about to describe has only developed in the last month or so.
We have hundreds (literally) of identical routers installed that are all connected in precisely the same way: monitored through Internet IPSEC tunnels. NONE of the other routers configured the same way with the same code with the same type of connectivity behave this way. Of course they are at diverse physical locations.
Ten to fifteen times a day, these two routers (which actually have nothing to do with each other and they are in different physical locations on the Internet) will log a bunch of hardware sensor events, saying that the hardware sensors are UP. They never say they are DOWN, they just continually log as UP, when they didn't used to do this repetitive behavior at all.
They don't log any OTHER messages and the routers are reliable and there does not appear to be any connectivity slowness dropped packets with these routers.
I am trying to determine what is causing this repetitve behavior in the logs for just these two routers because we have roughly 1500-2000 events daily from these two routers alone, logging only the hardware sensor messages.
Below is an example of the repeating entries in our event logs for these two routers:
4/4/2022 8:05 AM Hardware sensor Slot 6 - Temp: Internal of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - Temp: Outlet of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - Temp: CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Power Supply Module 0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - I: 12v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - P: pwr of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: VCC CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 0.75v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.0v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 3.3v STBY of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: DDR of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.8v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.0v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.1v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: VNN CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.2v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.8v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 1.35v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 2.5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 3.0v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 3.3v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 6 - V: 12v of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor subslot 0/1 power Sensor 0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 11 - P: pwr of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 11 - RPM: fan1 of hardware health monitoring on [hostname removed] is Up
4/4/2022 8:05 AM Hardware sensor Slot 11 - RPM: fan0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 7:35 AM Hardware sensor Power Supply Module 0 of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - I: 12v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - P: pwr of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - Temp: Internal of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - Temp: Outlet of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - Temp: CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 3.3v STBY of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.5v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.0v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: VCC CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 0.75v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.2v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: VNN CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.1v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.0v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.8v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: DDR of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 3.3v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 3.0v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 2.5v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.35v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 1.8v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 11 - RPM: fan0 of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 11 - RPM: fan1 of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 11 - P: pwr of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 12v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 7:35 AM Hardware sensor Slot 6 - V: 5v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - Temp: CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Power Supply Module 0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 0.75v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - I: 12v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - P: pwr of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - Temp: Internal of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - Temp: Outlet of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.0v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.8v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: DDR of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 3.3v STBY of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.0v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: VCC CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 2.5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.35v CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.8v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.2v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: VNN CPU of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 1.1v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor subslot 0/1 power Sensor 0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 12v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 5v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 3.3v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 6 - V: 3.0v of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 11 - RPM: fan0 of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 11 - RPM: fan1 of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:56 AM Hardware sensor Slot 11 - P: pwr of hardware health monitoring on [hostname removed] is Up
4/4/2022 6:55 AM Hardware sensor Power Supply Module 0 of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - P: pwr of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - Temp: Internal of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - Temp: Outlet of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - Temp: CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.8v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: DDR of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 3.3v STBY of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.5v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.0v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: VCC CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 0.75v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - I: 12v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.35v CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.8v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.2v of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: VNN CPU of hardware health monitoring on [hostname 2 removed] is Up
4/4/2022 6:55 AM Hardware sensor Slot 6 - V: 1.1v of hardware health monitoring on [hostname 2 removed] is Up
