Since upgrading to NPM 12.2 page loads are very slow only for Switches, Routers and Firewalls often taking 1-3 minutes to load the page.
Wireless Controllers, DWDM, UPS, VMAN and SRM are all fast with no problems.
We have 5 dedicated fairly high-end Cisco UCS servers 2.6 GHz dual quad-core with HT, 1 main NPM server, 3 APE's, a dedicated MS-SQL 2016 Server with RAID10.
When viewing Switches, Routers and Firewalls, CPU peaks around 35% max the rest of the time it's well below 20%. Main NPM and MS-SQL are on the same physical network segment. Each server has 2x 10 Gb/s NIC for a 20 Gb team.
I have defragmented database indexes. I shut down Activity Tracing that someone forgot and left running, but that did not resolve the problem.
With ProcessExplorer from SysInternals Suite I searched for handles with swdebugMaintenance and found a Database Maintenance process that was still stuck in the background, don't know why I didn't find it with regular Task Manager.
I killed that process, then was able to run Database Maintenance manually and it finally made it past the constipated Hardware Health tables and defragmented the indexes. I rebooted all the servers for good measure since I already had everything shut down. After reboot, I re-enabled hardware sensors and am currently getting about 9 seconds to load all pages including the Switches, Routers and Firewalls.
Thanks, that was very helpful, it seems weird with all of the searches I have done I have never seen Hubble mentioned.
The main holdup seems to be in the database query in Orion.HardwareHealth.HardwareItemBase which takes about 30 seconds. When I run the Database Maintenance manually it just hangs when it tries to process the hardware health table. I actually ran database maintenance before the upgrade and it processed that table with no problem at all. I had already tried Reorganize of the indexes for the table. It would be nice if I could just truncate the table and repopulate it somehow.
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process. Learn more today by joining now.