Nexus CLI Polling weirdness

I'm having an issue configuring CLI polling for the Nexus Leaf and Spine switches in our ACI fabric.

Using exactly the same username and password, CLI Polling test fails with "Test Failed. Connection Refused by <IP>", but both Putty and NCM connection test works perfectly?

Has anyone come across this issue, or have any suggestions on how to debug this one?

Find more posts tagged with

cisco aci

Accepted answers

All comments

michael.damron

Make sure you have gone through the guide - Set up monitoring for Cisco Nexus devices in NPM

Required privileges and some commands - Success Center

You can also enable CLI logging for some more in-depth troubleshooting.

mark.healey

Michael,

Thank you for your reply.

I've double checked everything is correct, and all the commands work when directly connected via SSH.

I've run CLI tracing and get the following with Keyboard Interactive Authentication:

[39a7f25d] [10/22/2019 09:10:11.589] Device Template: 81 Generic Cisco Nexus - 1.3.6.1.4.1.9.12.3.1.3

[39a7f25d] [10/22/2019 09:10:11.589] Try Acquire Connection...

[39a7f25d] [10/22/2019 09:10:11.605] Connection Acquired

[3abad6a0] [10/22/2019 09:10:11.605] -----------------Net. SWTelnet v7.10.0.166-------------------

[3abad6a0] [10/22/2019 09:10:11.605] UseCustomMorePromptBehaviour: False

[3abad6a0] [10/22/2019 09:10:11.605] MorePromptKey:

[3abad6a0] [10/22/2019 09:10:11.620] Custom MorePrompt Pattern:

[3abad6a0] [10/22/2019 09:10:11.620] Login Attempts: 1

[3abad6a0] [10/22/2019 09:10:11.620] Enable Identifier: #$

[3abad6a0] [10/22/2019 09:10:11.620] UseExactPromptMatch=False

[3abad6a0] [10/22/2019 09:10:11.620] Menu-based mode=False

[3abad6a0] [10/22/2019 09:10:11.620] FreezeLoginForPreCommands mode= False

[3abad6a0] [10/22/2019 09:10:11.620] CommandCharKey = Default

[3abad6a0] [10/22/2019 09:10:11.620] FISP mode= False

[3abad6a0] [10/22/2019 09:10:11.620] EncryptionAlgorithm= Auto

[3abad6a0] [10/22/2019 09:10:11.620] WODSSH Version: 2.7.2.180

[3abad6a0] [10/22/2019 09:10:11.620] UseVirtualPromptForCommands False

[3abad6a0] [10/22/2019 09:10:11.620] Responce Timeout in seconds: 15

[3abad6a0] [10/22/2019 09:10:11.620] Connection Timeout in seconds: 45

[3abad6a0] [10/22/2019 09:10:11.620] Keyboard interactive authentication set

[3abad6a0] [10/22/2019 09:10:11.620] StateChange: Connecting

[3abad6a0] [10/22/2019 09:10:13.636] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:13.636] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:13.636] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:15.651] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:15.651] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:15.651] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:17.651] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:17.651] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:17.651] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:19.667] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:19.667] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:19.667] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:21.683] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:21.683] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:21.683] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:23.698] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:23.698] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:23.698] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:25.698] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:25.698] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:25.698] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:27.714] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:27.714] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:27.714] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:29.730] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:29.730] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:29.730] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:31.730] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:31.730] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:31.730] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:33.745] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:33.745] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:33.745] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:35.761] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:35.761] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:35.761] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:37.776] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:37.776] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:37.776] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:39.792] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:39.792] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:39.792] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:41.808] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:41.808] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:41.808] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:43.823] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:43.823] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:43.823] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:45.839] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:45.839] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:45.839] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:47.855] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:47.855] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:47.855] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:49.870] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:49.870] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:49.870] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:51.886] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:51.886] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:51.886] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:53.901] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:53.901] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:53.901] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:55.917] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:55.917] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:55.917] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:57.933] TimerTick: mstrData=<> State=Connecting

[3abad6a0] [10/22/2019 09:10:57.933] Pending Disconnect = False

[3abad6a0] [10/22/2019 09:10:57.933] TimerTick: not idle, leaving

[3abad6a0] [10/22/2019 09:10:58.167] StateChange: Disconnected

[3abad6a0] [10/22/2019 09:10:58.167] TimerTick: Killing session timer.

[3abad6a0] [10/22/2019 09:10:58.167] Connection Error : WeOnlyDo.Exceptions.SSH.TimeoutException: Timeout occurred due to inactivity.

[3abad6a0] [10/22/2019 09:10:58.167] TimerTick: Killing session timer.

[3abad6a0] [10/22/2019 09:10:58.167] Force Disconnect

[3abad6a0] [10/22/2019 09:10:58.167] ERROR WODSSH: Timeout occurred due to inactivity.

I get the following without Keyboard Interactive Authentication:

[f1b545ab] [10/22/2019 09:11:07.730] Device Template: 81 Generic Cisco Nexus - 1.3.6.1.4.1.9.12.3.1.3

[f1b545ab] [10/22/2019 09:11:07.745] Try Acquire Connection...

[f1b545ab] [10/22/2019 09:11:07.761] Connection Acquired

[9b05a3d] [10/22/2019 09:11:07.761] -----------------Net. SWTelnet v7.10.0.166-------------------

[9b05a3d] [10/22/2019 09:11:07.761] UseCustomMorePromptBehaviour: False

[9b05a3d] [10/22/2019 09:11:07.761] MorePromptKey:

[9b05a3d] [10/22/2019 09:11:07.761] Custom MorePrompt Pattern:

[9b05a3d] [10/22/2019 09:11:07.761] Login Attempts: 1

[9b05a3d] [10/22/2019 09:11:07.761] Enable Identifier: #$

[9b05a3d] [10/22/2019 09:11:07.761] UseExactPromptMatch=False

[9b05a3d] [10/22/2019 09:11:07.761] Menu-based mode=False

[9b05a3d] [10/22/2019 09:11:07.761] FreezeLoginForPreCommands mode= False

[9b05a3d] [10/22/2019 09:11:07.761] CommandCharKey = Default

[9b05a3d] [10/22/2019 09:11:07.761] FISP mode= False

[9b05a3d] [10/22/2019 09:11:07.761] EncryptionAlgorithm= Auto

[9b05a3d] [10/22/2019 09:11:07.761] WODSSH Version: 2.7.2.180

[9b05a3d] [10/22/2019 09:11:07.761] UseVirtualPromptForCommands False

[9b05a3d] [10/22/2019 09:11:07.761] Responce Timeout in seconds: 15

[9b05a3d] [10/22/2019 09:11:07.761] Connection Timeout in seconds: 45

[9b05a3d] [10/22/2019 09:11:07.761] StateChange: Connecting

[9b05a3d] [10/22/2019 09:11:09.776] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:09.776] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:09.776] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:11.777] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:11.777] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:11.777] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:13.792] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:13.792] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:13.792] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:15.808] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:15.808] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:15.808] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:17.823] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:17.823] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:17.823] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:19.839] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:19.839] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:19.839] TimerTick: not idle, leaving

[9b05a3d] [10/22/2019 09:11:21.855] TimerTick: mstrData=<> State=Connecting

[9b05a3d] [10/22/2019 09:11:21.855] Pending Disconnect = False

[9b05a3d] [10/22/2019 09:11:21.855] TimerTick: not idle, leaving

it looks like it cannot understand recognise that it has logged in, and hangs waiting for something.

The NCM trace shows the following lines:

[f82187d3] [10/22/2019 09:17:28.106] -----------------Net. SWTelnet v7.10.0.166-------------------

[f82187d3] [10/22/2019 09:17:28.106] UseCustomMorePromptBehaviour: False

[f82187d3] [10/22/2019 09:17:28.106] MorePromptKey:

[f82187d3] [10/22/2019 09:17:28.106] Custom MorePrompt Pattern:

[f82187d3] [10/22/2019 09:17:28.106] Login Attempts: 1

[f82187d3] [10/22/2019 09:17:28.106] Enable Identifier: #$

[f82187d3] [10/22/2019 09:17:28.122] Device Template: Generic Cisco Nexus

[f82187d3] [10/22/2019 09:17:28.122] UseExactPromptMatch=False

[f82187d3] [10/22/2019 09:17:28.122] Menu-based mode=False

[f82187d3] [10/22/2019 09:17:28.122] FreezeLoginForPreCommands mode= False

[f82187d3] [10/22/2019 09:17:28.122] CommandCharKey = Default

[f82187d3] [10/22/2019 09:17:28.122] FISP mode= False

[f82187d3] [10/22/2019 09:17:28.122] EncryptionAlgorithm= Auto

[f82187d3] [10/22/2019 09:17:28.122] WODSSH Version: 2.7.2.180

[f82187d3] [10/22/2019 09:17:28.122] UseVirtualPromptForCommands False

[f82187d3] [10/22/2019 09:17:28.122] Responce Timeout in seconds: 15

[f82187d3] [10/22/2019 09:17:28.122] Connection Timeout in seconds: 45

[f82187d3] [10/22/2019 09:17:28.122] StateChange: Connecting

[f82187d3] [10/22/2019 09:17:29.497] Got HostFingerPrint: System.Byte[]

[f82187d3] [10/22/2019 09:17:29.856] SWTelnet9 Crypto Information Begin

[f82187d3] [10/22/2019 09:17:29.856] Protocol = SSH2

[f82187d3] [10/22/2019 09:17:29.872] RemoteName = ##.###.##.###

[f82187d3] [10/22/2019 09:17:29.872] SCcipher = AES128ctr

[f82187d3] [10/22/2019 09:17:29.872] CSCipher = AES128ctr

[f82187d3] [10/22/2019 09:17:29.872] Keys = ssh-rsa

[f82187d3] [10/22/2019 09:17:29.872] SWTelnet9 Crypto Information End

[f82187d3] [10/22/2019 09:17:30.075] StateChange: Connected

[f82187d3] [10/22/2019 09:17:30.075] Connected!

Looks like I might need a support ticket?

GoldTipu

Just adding the KB

https://support.solarwinds.com/SuccessCenter/s/article/Troubleshot-CLI-Polling-credential-test