1 of 1 people found this helpful
Yes, this is an interesting one.
I willing to bet the timestamp on your configuration saves is the majority of the issue. I have been able to mitigate this to an extent by running a job that issues the following commands to my Cisco Devices.
copy run start
It does not totally solve the issue, but helps for me.
1 of 1 people found this helpful
The way this feature works is it compares the running configuration that it has last backed up to the startup configuration that it just backed up. If the startup configuration is much older than the running configuration in your backed up configs that is the issue. More than likely you are downloading a copy of your running config nightly and the startup config that is saved within SolarWinds is older. If you schedule a job to run once overnight and get a fresh copy of the startup configuration I bet that would turn a lot of them green again.
You can test this out by clicking on the "Configuration Management" link, under the main "Configs" tab. Then select 10+ devices (enough to see the graph change), and manually initiate downloads for both the running and startup configs.
Once both config types have been downloaded, you should be able to see if that graph changes. If it does change, then you probably just need to adjust your backup schedule, so your configs are downloaded together, or at least closer in time.
And don't forget, you can use the "Script Management" tab, just above the "Download" button, to create a simple "write mem" script, and then use it on your selected devices, as CourtesyIT mentioned. This should help you speed things along, as you can basically do all of your troubleshooting directly within NCM's Config Management page. Just remember to refresh your graph after you run an action, so you can determine which action/task was causing the issue.
kai Or, I suppose you could just simply click on the red part of that chart, which should take you directly into a specific management view, and allow you to do your downloads there. I believe it is basically the same as doing it through the config management view, only this way allows you to specifically sort results that are directly related to config changes.
Either way, it should be a quick way to determine what direction you need to go.
Just click that red part of the graph, then click the conflict filter, take note of the number of conflicts, select a few devices, and re-download both config types. When it refreshes, if that was the issue, the number of conflicted devices should have decreased.
If it did not decrease the number of conflicts, then run your "write mem" script on a few of the devices, and then re-download both config types again.
There are a number of items you may wish to filter out from your configuration comparison criteria, after which you may see significantly improved results.
As CourtesyIT mentioned, time stamps will be different between configs taken at different times. And like him, I have a regular job that writes to memory on all my Cisco devices.
But you can check to see if you're properly ignoring things that shouldn't be compared by going to Admin > Settings > NCM Settings > Comparison Criteria. Use the right syntax to eliminate some common items that can cause the issues. Here are a few screen shots I use as examples:
But since your reports don't show anything actually has changed or is different between startup and running configs, nothing red or yellow, you've got something else going on.
If you're writing to memory regularly, and you're successfully running jobs that backup both the startup-config and the running-config without any failed devices, I'd call Solarwinds Support for some ideas.
I watched the solarwinds tech guy go through all that, even made some changes. But good news, its been resolved now. for some reason, a Reboot of the Solarwinds Server fixed the problem. I cant even tell you how this happened but it worked.
thanks Guys for taking the time!! have a great solarwinds day
Turning it off, and on again... Classic!
Glad to hear they got you all fixed up.
Ive just discovered this after a clean build... about 20 devices.
How do I easily select these 20 devices, then run the wr mem script? I cant do it from this screen, and when I got to config management, where I can run a script, I cant list the devices that have conflicts!
I suppose the o nly way to do this, is select all 20 devices, edit properties, edit one of the comments fields to group them all together, go to conf management, select them and run the script.. such a faff.. could be much easier!
If only all filters/actions were available where devices are listed!
In my environment, every device gets a write memory command every day, unless there is a specific reason not to. Is your environment different than this? If not, just use NCM to write mem on all devices, and your graph should turn 100% green after the next job cycle. That's thinking out of the box.
If the pie doesn't turn 100% green after the next job cycle, hover over the gray failed backup pie section and click on it to see which devices failed to write memory.
You can also find a report that shows which devices failed to successfully execute a job for doing a write memory on them all, and I'd hope you'd have built your job to e-mail you a notification if a failure occurred, and that you set it to list the devices that failed. That one e-mail can be your list of devices to troubleshoot, too.
Troubleshoot them individually--could it be one or more have the wrong credentials supplied? Maybe some simply never were added to a regularly scheduled "write mem" job--it's easy to edit the job to include All devices.
2 of 2 people found this helpful
sk3l3t0r If you want to select all of the devices (20 of them, in your case) that show a config difference, but you want to be able to run a config on that same selection (which takes place on a totally different page), you should be able to do the following:
On your "Running V.S. Startup Config Conflict" pie chart, click the part of the pie chart that indicates the devices with conflict.
Next, select your 20 devices, and simply re-download the running config. (While you may not be directly interested in actually re-downloading those configs at the moment, this is simply going to help you better group your devices...)
Once the config download process begins, you can navigate over to the "Configuration Management" page. (It should actually take you to the "Transfer Status" tab of that page automatically, after you begin the download...)
Now, on the "Configuration Management" page, make sure to remove the "Group By" options, so you can see all of your devices.
Then, sort the table by "Last Action Date", so it lists the most recent at the top.
By this point, the 20 devices you just initiated a download on, should now be showing at the top of the list.
Finally, just select those devices, and click on "Execute Script".
I hope that helps.