cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

NCM 7.6 Sneak Peek - Firmware Upgrades

Level 15

There has been a lot of focus on security lately in the news and rightfully soSeemingly each week there’s news of companies being hacked, data being stolen, and mass DDoS attacks.  With the amount of news on this topic I sometimes wonder if companies are actually taking the steps to protect themselves.  Granted, taking the proper steps to protect your network can be time consuming and tedious work, of which most engineers don’t have time for.  Well times are a-changing and now with SolarWinds Network Configuration Manager 7.6 and the new Firmware Upgrade feature---Everyone Has Time for That!

Before I start to dig into the new features of NCM 7.6, let’s back up and talk about a previous version of NCM (7.4) where SolarWinds introduced the Firmware Vulnerabilities feature.   This feature leverages the National Vulnerability Database to notify NCM users when they’re running firmware that potentially has a serious vulnerability.

pastedImage_2.png

Network Configuration Manager Vulnerability Summary

pastedImage_25.png

Vulnerability Details

I’ve received a lot of really positive feedback about this feature but the obvious question that always comes up after I show it to customers is; “Can SolarWinds fix this for me?”.  Historically I would have said, Yes, using the Network Configuration Manager and the amazing scripting technologies you can upgrade your firmware.  Well, now I’m pleased to say I can answer that question differently.  Using the new Firmware Upgrade Wizard in NCM 7.6 you can upgrade one or many of your Cisco IOS devices.

According to Cisco documentation there are 11 steps needed to complete a firmware upgrade on your Cisco IOS devices.  While 11 steps don’t sound too bad there are actually several sub-steps which drag this process out to over 40 tasks users must complete to upgrade a SINGLE device.  Seriously, who has time for that?  We here at SolarWinds decided we could save our users time and the misery of completing these upgrades by simplifying this process while adding a bit of automation.

pastedImage_35.png

NCM Firmware Upgrade Wizard

The new Firmware Upgrade feature in NCM contains a 3-step process of upgrading your devices.  During this process, we will collect a wealth of data about the devices you want to upgrade, including several important settings and of course we will ensure there is enough free space to successfully transfer your new image.  In addition, we will automatically backup running and startup configuration files and do a comparison after the upgrade has completed.  We’ve taken the necessary steps to make this process as smooth and safe as possible.  

pastedImage_44.png

After you’ve verified all of the settings and options you can then proceed to run the upgrade immediately or schedule it for a later date.  You can always keep track of the upgrade process on the Firmware Upgrade Operations page.  Hopefully you’ll agree that this is a much-improved process to the standard method of upgrading your Cisco IOS devices.  Ready to give this a try?  You can find the Release Candidate in your customer portal if you’re under active maintenance for NCM.  Otherwise you’ll have to wait until the official release of NCM 7.6.

Everyone Has Time for That!

31 Comments
MVP
MVP

Nice new features. I'm hoping to move to NCM at some time in the future.

MVP
MVP

tallyrich​ NCM is easily worth the money... It has saved us time and time again. Once you get all of the basic chores, and regular maintenance tasks setup, which is quick and painless, then you find all of the little hidden gems. Very powerful tool.

Level 10

This will be a GREAT feature - IF You have solved the issue in the present release: The report only looks at major releases and not minor releases, which ends up in a lot of false positives... Did You look into this?

Level 15

Are you talking about the firmware vulnerabilities reporting?  If so the match is based on the IOS version we have and what the NVD has put in the CVE.  Sometimes they CVEs are very generic and specify things like Cisco IOS, which make its difficult to match to a specific version of IOS.  This is less than ideal, so we have two options at present, ignore generic CVEs or notify users and allow them to decide if it applies to them.  Is their a particular side of the fence you prefer? 

We recently finished rolling out NCM/NTA in support of installing RADIUS across our landscape. Of the 300+ switches we found that 86 of them were running IOS versions that did not support SSH. NCM 7.6 could not have come at a better time.

Level 8

We are starting to tune NCM to create our own custom policies and auto remediate, and it has been a huge value off this alone.  The IOS upgrade feature was the one main function I thought it was missing, so this is great news and I really hope it shows up in the final release.  jeff.stewart​ I agree with you that it is really much closer to a 40 step process to do an IOS upgrade and each engineer seems to have a bit different process.  Looking forward to more detail as this comes along.

Level 11

Sounds wonderful, but please tell me that will have full coverage in the SDK from day 1!

In the modern age if a feature like this that does not have great API coverage and still depends on human time, we'll very likely just need to find another way of automating the upgrades.

Level 10

Yes it is the Firmware Vulnerabilities check that has to be adjusted a bit....

Example:

We have 12 nodes which are on the list with a possible issue (Vulnerability Summary for CVE-2005-2451).

CVE-2005-2451 details: Cisco IOS 12.0 through 12.4 and IOS XR before 3.2, with IPv6 enabled, allows remote attackers on a local network segment to cause a denial of service (device reload) and possibly execute arbitrary code via a crafted IPv6 packet.

BUT when You go to one of the nodes - the IOS details are as follows:

Cisco IOS Software, 2800 Software (C2800NM-S​PSERVICESK​9-M), Version 12.4(9)T7, RELEASE SOFTWARE (fc3) Technical Support: http://www.c​isco.com/t​echsupport Copyright (c) 1986-2008 by Cisco Systems, Inc. Compiled Thu 10-Jan-08 16:35 by prod_rel_t​eam

That means that the tool is not taking into account that a node could have an IOS version newer than 12.4 but older than 12.5. The software on the node in the example is from 2008 and the vulnerability is from 2005.

So I would like a tool that is looking on the minor releases also when it is reporting on which nodes could have an issue regarding vulnerabilities...

Level 10

This is great news if it works!

We've been trying to use NCM 7.5.1 for vulnerability patching but it's nowhere near good enough so we've started looking for other products, there doesn' seem to be anything out there that does everything we want, the main issues we have with NCM 7.5.1 is all the false positives we got due to the following 3 reasons.

would be great to know if these will still be an issue in the next release.

1) Solarwinds doesn't seem to be able to get the exact model of the switch, we even got vulnerabilities coming up for 3750 switches that were for access points just because there is also a range of cisco access points that start with 37xx

2) I'd also echo the comments made by orionshark around the IOS versions, nothing more to say on that apart from what is already posted above.

3) Will the new NCM take into account the config on the devices, we had vulnerabilities coming up for features that we don't have enabled - NCM has a copy of the config, can't it check this and decide we are not vulnerable to it.

If in future someone enables that feature it would then be picked up on the next scan.

Thanks for the continued work on this, hoping we can stick with it!

cheers,

worto.

EDIT - oh and a little off topic to the rest of my post but does it support NX-OS for our Nexus kit?

Level 10

I haven't installed this yet so I haven't verified this but it's 2017 and we're still writing Cisco-only features into a network management tool?  Come on...

Level 8

will there be an extensive list of device templates available or will a person need to create their own templates for example an 892 router

MVP
MVP

While it has not been a difficult task to utilise the NCM Execute Script or Config Change Templates capabilities to perform Firmware upgrades for a long time, it is rare that I come across customers actually using NCM (or indeed any other CM tools) to perform this task.

I am interested to see if this changes with this new feature, as it certainly simplifies the process, but does it take away the fear admins have for performing upgrades?

swf5002@psu.edu​ - while you can consider it to be disappointing that the out of the box Firmware upgrade templates are only including Cisco, the framework allows any vendor/device to be supported. Indeed, SolarWinds have added the ability to create and share new templates, so get the ball rolling..

Mark Roberts

Prosperon - UK SolarWinds Partners

Installation | Consultancy | Training | Licenses

facebook_icon.jpglinkedin.pngblogger.pngtwitter-icon.jpg 

Level 10

m_roberts Perhaps the framework is there, but I've now installed it and I again I see multiple roadblocks because of the Cisco-centricities.  For instance the firmware repository itself seems only to accept .bin files!  The documentation says "by default" but I can find no option to modify it.

binonly.png

On top of that, the repository itself requires a network share location, unlike other network locations in this same software package (namely the config-backup location, which can be a local drive).  There are access denied messages when you set it to a local drive.  From NcmBusinessLayerPlugin.log:

pastedImage_1.png

If you try to trick it by making the local folder a network share and entering that path as a network share into the repository field your credentials fail to work!

At the time I made my original post I hadn't installed it yet but the documentation made it look like there wasn't even a way to do different types of devices.  Now that I've tried it I can confirm that indeed at least for anything that doesn't use a .bin file you cannot use this piece of the product.

Level 7

We are in almost the exact same boat, have 72 devices that have to be upgraded for the same reason. Now I just have to convince my boss to hold off a bit longer so I can use this tool instead of doing it all manually.

Level 13

Sounds really great but will it work for Cisco devices only?

Level 10

It only works with vendors whose devices have firmware files with the .bin extension.  I don't' know of any other than Cisco at this point that do that to test but I can say positively it will not work as-is for Dell, Juniper, or Aruba.

Level 11

Firmware upgrades is a nice feature (in theory), but there are two major drawbacks from my point of view:

- Image repository can only be local or on a Windows share (with some effort you could probably get it to work off Linux..but still); it would be nice to use other protocols as well (FTP, SCP). We have quite a diverse ecosystem and various firmware images can occupy quite a bit of disk space and it would be nice to store them on a dedicated server (an FTP server for example).

- You can only define one software repository. Quite a limitation when you are a company operating worldwide...

Granted, it may be a bit raw and may improve over time, but it may be unusable to us at this point.

Level 8

I have been doing a bit of testing with this feature, one thing i have noticed is that when i run it to multiple devices it updates them consecutively and not concurrently is there a way to have the multiple devices upgrading at the same time?

Level 8

any plans to add the Cisco ASAs?

Level 14

Thanks for this. I was able to finally get to the point where it would actually recognize the files in my repository. Unfortunately, like you, I'm not testing this out on a Cisco device and now my upgrade process is stuck because it's waiting for a question mark (?) prompt in the CLI that this device is not giving. I really hope this feature is built upon to overcome these limitations.

Awesome! but +1 on this^

Level 10

Looks good, but having issues with mixed 3750 and 3750v2 stacks (mix of S and E images).

Also, Solarwinds doesnt seem to distinquish 3750 and 3750v2 switches so upgrading v15 ios to 3750v2s is more difficult.

Level 8

I've been waiting a loooong time for this feature so thank you. Tested a few times on single devices and worked great, when selecting multiple devices it hangs after the second device (3rd max) and doesn't continue onto the next device to upgrade.

Level 9

What about for Dell switches?  I know that they are not as popular as Cisco but we use them for our Layer 2 only closet switches and would like to see more added in NCM for them.

Level 10

No they are not supported.  Only .bin files are accepted as firmware types in the repository manager.

Level 10

you can customize the upgrade templates to make them work for ASAs which is what we are doing.

Level 9

There's a way:

In the Advanced Configuration, which can be found under:

http://<server>/Orion/Admin/AdvancedConfiguration/Global.aspx

you can specify the extension. But it seems only one extension at a time works. So still not very nice to the busy administrator.

Level 7

I have tried the Firmware Upgrade Operation on two different routers but get the same error on both:

Verifying backed up image...

verify /md5 bootflash:isr4300-universalk9.03.16.01a.S.155-3.S1a-ext.SPA.bin 3d8a665b53db527471e4a2b8b0e0ca52 ${SuccessRegEx:Verified}

ERROR Process could not continue because execution results do not match expected pattern: Verified

_______________________________________________________________________________

Upgrade Error on node SAVRTP01. Operation interrupted

Has anyone else seen this or know what it means?

Level 12

Totally agree if this vulnerability reporting is to be useful it needs to take into account minor releases. Other than that a big fan of NCM

Level 12

What if you've to do a large number in one evening?!!!!

Level 12

Did you get anywhere with this?

About the Author
Former SolarWinds Customer, THWACK MVP & Product Manager. Currently leading the Americas Sales Engineering team.