NCM Nightly Job Errors but Manual Download Works

[DonnieNarco]

Hello, I am fairly new to NCM and SW, and I have inherited a pretty large database of devices to back up in NCM. Every night the job runs to download running and startup configs on Cisco, Juniper, HP devices. Once the job is complete, I get around 500 errors out of 750 devices, and most will say "Cannot Download Running/Startup: Connection Refused". 

Now I have searched and see the most common issues for this, and usually incorrect credentials and sometimes that is exactly what it is, but I am seeing a lot of devices that are working just fine. For example, if I go into the Node and scroll down to NCM section and click Test, it says Test Successful....then if I go into NCM and go to the device in question, then Configs, then Download Config, it actually works!

Why then does the job fail every night for that device, plus others? It is frustrating to see this and I cannot figure it out. Is this a bug that I need to upgrade to a later revision on? I am running: Orion Platform 2018.4 HF3, IPAM 4.8.1 HF1, NCM 7.9, NPM 12.4, NTA 4.5.0, SAM 6.8.0, NetPath 1.1.4

Thank you for any advice. 

[HerrDoktor]

Do you have an authentication service in the background that maybe limits the amount of logins within a certain timeframe? I also remember having an issue with one of my clients where the backend System was not performing well so the authentication requests against that system were running into timeouts.

[DonnieNarco]

We are using MS NPS as the authentication for this. Do you know how many concurrent connections NCM makes to devices that might overload NPS?

[HerrDoktor]

The usual IT answer “it depends” ;).
By default I think 10 concurrent sessions are opened. However when a Device is “done” it could be that the setting to look at on your authentication system is “number of logins in a certain timeframe”.
From your description it really sounds like you need to look at the MS NPS Server

[DonnieNarco]

Thank you! I will look at he NPS and see if that leads me anywhere. I will report back.

[HerrDoktor]

Did you find anything on the backend authentication system?

[DonnieNarco]

So, I tried a separate job just for one site that had 6 devices that were previously erroring out every night, and this separate job had no issues! We looked at NPS but couldn't really determine if it was giving us problems, as there was nothing in the logs. I am still going through and auditing the rest of our equipment, then I am going to go hard on fixing these errors. I may have to just create a job for every site instead of one large job. I appreciate the help and if I have any further updates I will be sure to share here!