cancel
Showing results for 
Search instead for 
Did you mean: 

Network Configuration Management and Automation

Level 10

Based on some of the responses from this last post, it is good to see that there are some that take backing up their network configurations serious. I would like to build on that last post and discuss some ideas around network configuration management, specifically solutions and automation to handle some of the tasks required. I see that several stated that they use Solarwinds NCM, which I personally do use in my environment. Solarwinds NCM is extremely easy to setup and configure to “save your bacon”, as one comment stated. NCM is a perfect example of a solution, which has the ability to track changes, roll back changes, reporting and auditing; as well as many other use cases of the product. There are also many open source products that I have used previously which include routerconfigs plugin for Cacti, simple TFTP jobs setup on each router/switch to backup nightly and numerous other solutions including rConfig which is another open source product that is a very solid solution. However, you may have the solution in place but how do you handle making sure that each and every network switch/router is in a consistent state? Or is configured to have nightly scheduled backups of their configs? Do you still do this manually? Or do you use automation tools such as Ansible, Chef or Puppet to stamp out configurations? I have personally began the journey of using Ansible to build playbooks to start stamping out configurations in a consistent manner as well as for creating dynamic configurations using templates. This is also another great way to start building a solution around the fact that when a network device fails, you have a solid way to start rebuilding a failed device from a somewhat consistent state. I would definitely still leverage a nightly backup to restore configurations which may have changed over time since deployment of an automation tool but hopefully as changes are made, your automation deployment configurations are also modified to reflect these changes.

19 Comments
clubjuggle
Level 13

I find the Compliance Reporting feature within NCM to be tremendously useful in determining whether routers are configured consistently with company standards.

Comparing configurations to baselines can be useful as well, but in the environment I came from, authorized changes happened too often for that to be as useful as I would have liked.

Jfrazier
Level 18

While I do not use the NCM product itself, I have worked at shops where it was used to back up configs on a nightly basis.  I am not aware of tools used to automate configurations in use at either of those shops.  I do know that the config backups were utilized from time to time to fix or resolve issues...

clubjuggle
Level 13

We had that situation a lot at my last employer.

Jfrazier
Level 18

Apparently it saved someones bacon..not mine but I am all about saving bacon.

jhandberg
Level 13

Before everything started moving to the web interface, I had build several execute scripts in the NCM console that I could easily upload to any switch for our standard settings like:

Several security settings like console and tty settings, ssh, management vlan ACLs, password encryption, logging as well as QoS settings and other scripts that would bring any new or blanked switch up to our standard base configuration.

I have an NCM compliance policy I can then run against the switch to make sure nothing was missed and it has every setting we want configured correctly and is ready to go.  Then it is just customizing access port settings (mainly VLAN settings and port descriptions) to get any spare switch into service. 

I suppose I should work on getting these available in the web interface, and maybe more dynamic.  There is always something new to learn.

cahunt
Level 17

I like how you brought up Bacon, and did not get Tangent-ed into Sandvich Chatter (Cheddar)

cahunt
Level 17

We rely heavily on the configuration changes displays and back ups / reference to the last config saved before a change.

   Solarwinds NCM : Saving Bacon Daily!   


And with the increasing cost of swine, you can't go wrong!

cahunt
Level 17

Our scripted job for back ups sends an email with back up status for each node.
We get two emails, one for start up, one for running - And the Failed Nodes are always at the top grouped together.
    -> For us it exposes an issue or the node is one of those (sometimes connected to the network, and the name reflects it so we don't worry).

clubjuggle
Level 13

Maybe we need a Solarwinds-branded one of these in the Thwack store:

51KQ0TFovmL.jpg

bluefunelemental
Level 15

hmmm

"/^[b][a][c][o][n]/i"

cahunt
Level 17

Color that Orange and Slap a SW Logo on it, I will buy it in the THWACK! Store! 

jkump
Level 15

I too have spent time tuning and tweaking the compliance reports to ensure that all configurations are consistent and standardized across all the platforms relying on NCM to perform the nightly backups.  In the event, that major changes are needed during the production, then I will actually schedule a manual backup in addition to the nightly.  I have found NCM as a great tool to save my bacon when we had a fiber interconnect switch failure occur.

clubjuggle
Level 13

As an alternative, you might consider enabling real-time change notification. As part of the notification process, NCM automatically downloads an updated copy of the running configuration whenever a change is detected.

The steps to do so are listed on this help page: SolarWinds Online Help

If you want the automated backup when a change is detected, but not a notification, you can set up the download action but skip the steps to set up the email notification on the alert.
jgrobinette050
Level 10

We use NCM for backing up configs.(Network Devices), we have several compliance reports with remedies for some of them that can be triggered.

We need to get our hands on more templates if that is available online. Automation is the word of the day here, so the more we can demonstrate getting templates, downloading them, modifying them to meet the needs here and running them as a job, we will win. My customer is looking at BBNA, they claim more out of the box automation.

They need to do real baselining here, that is the topic and word of the day as well. We see a baseline job that comes out of the box, but that scares us. Our lab needs beefed up before we tackle a full scale baseline or if that job is not harmful then we may proceed in using that.

Also, any insight into incorporating servers into NCM, imaging or snagging just the OS directories?

jkump
Level 15

That would be handy but our Change Management approval process has the changes discussed and approved before hand and then it is just a matter of actually making the changes and proving that they were made.

Good thoughts though!

clubjuggle
Level 13

That's good policy, but real-time change detection is helpful for proving that unauthorized changes are not being made without following the proper procedure, and knowing who made a change if a violation occurs. It's an extra level of assurance for yourselves and your regulators.

quiglem
Level 7

NCM is the best part of Orion for me. Being able to track changes, compare configs, and review setup for devices that are down, makes this the premier app.

jkump
Level 15

I see your point.  Since I have Radius/Tacacs+ login logging to each device, i will have to investigate setting up the real-time configuration changes.  They would certainly keep things tidy for audits.  Thanks!

d09h
Level 16

We have a write mem job running every day.  Also a batch file that counts the number of configs in the filesystem for the current date.  That count is appended to a running list of number of configs by date.  This batch file is on each poller so we watch the number of configs on each filesystem for any deviations.