NCM Causing High CPU Utilization on Cisco Switches

Hi All,

I have a situation where Cisco switches are reporting high CPU utilization, as per cisco TAC it is due to NCM SSH conversation which connect to the device to run commands to display configuration and can perform other jobs.

For testing we disabled NCM polling on the switch for some time and it decreased CPU utilization.


They are suggesting if we can run less intensive jobs on each connection. 

Please suggest if anyone else faced this issue if yes, what you did to overcome it.

Thanks in anticipation.

Parents
  • Update: As per the logs shared by network team, NCM user is logging in and logging out every few minutes. This is causing multiple SSH sessions which is spiking the CPU.

    Mar 12 11:24:46: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: NCM

    Mar 12 11:24:59: %SYS-6-LOGOUT: User NCM

  • We are experiencing locking in the SolarWinds Orion MSSQL database due to processing the NCM_Cisco_Flash table.  Some of the database threads are 3.5 days old.  The NCM_Cisco_Flash table does not have data newer than 3.5 days ago.  There is one current thread that is attempting to UPDATE the table.  This does not appear to be affecting other SolarWinds processing, but as a DBA, it bothers me to see a few screens of SUSPENDED threads when I look at the SQL Server Activity Monitor, and then see some are 300 million milliseconds old.

    Perhaps the high CPU usage when polling NCM is a related issue?

Reply
  • We are experiencing locking in the SolarWinds Orion MSSQL database due to processing the NCM_Cisco_Flash table.  Some of the database threads are 3.5 days old.  The NCM_Cisco_Flash table does not have data newer than 3.5 days ago.  There is one current thread that is attempting to UPDATE the table.  This does not appear to be affecting other SolarWinds processing, but as a DBA, it bothers me to see a few screens of SUSPENDED threads when I look at the SQL Server Activity Monitor, and then see some are 300 million milliseconds old.

    Perhaps the high CPU usage when polling NCM is a related issue?

Children
No Data