This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

NCM Nightly Config Report Errors - Connection Refused

Hello folks- we are trying to get our nightly config backup working, but we're running into some difficulties. We're running NCM v7.4.1. In the report, the three main problems we're getting back seem to be:

   ERROR: Cannot download Startup config : Cannot Log into Device : bad password.

   ERROR: Cannot download Running config : Connection Refused

   ERROR: Cannot download Running config : Connectivity issues, discarding configuration (or configuration is too short).

The main problem seems to be the connection being refused. We have two NCM profiles, Telnet & SSH- both of which have the same login credentials. I can log on to the switches just fine from Putty using the same credentials in the NCM profiles, and if I go to the node page in SolarWinds I'm able to download both the running and startup configs no issue... We have around 800+ switches managed, and on the report from last night we're getting about 300 back with errors. I've been searching around and it seems there's a lot of people with this problem but not a very specific solution... Could it just be trying to download too many configs at once? If that might be the case it would make sense to me to have different network areas run the config backup at seperate times to avoid congestion. Does anyone have any suggestions on what route I should take? I can provide additional details/screenshots as well. Thanks in advance

Here's a screenshot of our config settings:

NCM Settings.png

  • thad   For me the issue was that the device was somehow loosing the NCM profile.  I had to edit the device and change it to the appropriate profile again.

  • That's strange. I haven't had that issue that I know of... All of the switches I have checked from the list still have their original NCM profiles set. Thanks for the reply

  • I usually find the following to be true:

      ERROR: Cannot download Startup config : Cannot Log into Device : bad password. 

        Issue: Using telnet when it should be ssh or vice versa

      ERROR: Cannot download Running config : Connection Refused

        Issue: Same as above

      ERROR: Cannot download Running config : Connectivity issues, discarding configuration (or configuration is too short).

        Issue: Using a default template or the Cisco IOS when connecting to a WLAN controller or some other non router/switch device. Try changing the templates to the specific device type

  • Have you tried to connect to the device from the poller that is monitoring it? There could be an ACL in place that is preventing the connection and that is why you are getting connection refused. If you can ssh to the device from the poller then there is something else going on with the device in NCM. I have had some devices show up as an unknown device type before which caused issues with NCM and I had to remove and re-add them for it to work right again.

  • jdane07​ I have switched a few switches from Telnet to SSH or vice versa and they still show up on the report the next day. A few that I took note of were not in the report the next day even though I made no changes to them, so that's another strange thing going on.

    @Chris T I have changed all of the switch ACLs to include the solar winds IP. However- if I'm on the poller page, nothing happens when I try to SSH from the webpage; & telnet has been hit/miss. Could this be part of the problem, or is just another random issue? I can go to the configs tab of the node and download the config just fine.

  • It could be another random issue. Have you logged into the physical (or virtual) server that the poller software is on and tested the connection to the device that way? If you can SSH or Telnet from the actual server desktop then the problem is narrowed down to the SolarWinds configuration somewhere.

  • In addition to the other suggestions given already, I would suggest you enable session tracing to help your troubleshoot the issues.  The trace files can be a little hard to parse initially but basically they are showing you exactly what NCM is doing and how it is interpreting the responses it gets from your devices.  That way you can see in debug level detail exactly why you had issues and try to figure out how you can fix them, if you can't figure it out support is going to ask you to generate traces anyway so it doesn't hurt to have them. One handy trick is to search for the lines with <-- because that shows you where NCM was actually sending data to the device.

    See the Success Center article here,  Generate session trace logs

    -Marc Netterfield

        Loop1 Systems: SolarWinds Training and Professional Services

  • Thanks for the suggestion, we tested this and we're still able to Telnet/SSH into the specific switch

  • Hey thanks for the reply, we checked the session trace log and actually found that the config downloader just stopped at 223 nodes left. It didn't show any errors however and just said config download complete. Do you have any idea why it would just stop abruptly?

  • Can you see if all of the devices that are having errors are on the same poller? If the devices are on the same poller and every devices that is managed in NCM on that poller is having issues it may be an issue with the NCM poller software on that poller. Could be a version miss match between the version on the poller and the version on the Core server, or possibly something else. Not sure what could cause that, just fishing for ideas at this point.