Open for Voting

NCM - Perform a Masive firmware upgrade operation

​We need an option that allows Confirm the same upgrade procedure to all selected Nodes. in order to expedite the deployment of updates.

  • and it sould be possible that when one update out of 100 device failded, the rest of the updateprocess is going on, and ist not stoping on the device that may be temporary down.... 

  • I understand now.  You want upgrades to be simultaneous instead of sequential.  It's a useful idea.

    My bet is that Solarwinds will need to make some modifications to allow it.  I've had good luck moving IOS images to seven Cisco devices simultaneously by leveraging the multi-threading capabilities of Solarwinds' SFTP or TFTP servers, but not more than that.  Trying to do twenty simultaneous file transfers of the same IOS image using those servers fails.  Fixing that, or providing a work around, would be required.

    Something that scales higher and moves faster is what you're looking for.  We use a Linux scripting library to do multiple configuration changes or IOS upgrades in my environment.  It's pretty bullet-proof and our reliability factor with it is good.

    We can deploy or remove ISE on all ports on a large number of switches with it, which is a time saver.  We have separate jobs to push out IOS to many devices, change their boot variables and save them, and a last job to reboot them all.   By breaking those steps into separate jobs, and by using good planning, anyone can use Solarwinds or another solution (like our Linux scripting solution) to push out new firmware to a large number of switches several days in advance, then change all of their boot variables to use the new code, and finally to upgrade/reboot them all at the same time once we know they all have received the code successfully and had their boot parameters adjusted correctly.  

    On the other hand, my environment is a 7x24 critical health care system serving several million customer experiences each year.  Having all our networks down at the same time for an upgrade isn't allowed.  We take it one building floor at a time, or one building at a time, since we must accommodate the needs of patients and their providers.

    It would be nice to be able to reboot them all and upgrade everything at once, but if something were to be incorrectly configured or deployed, we'd have a hundred hospitals, clinics, and business offices down until my team of 8 Network Analysts could manually visit them all, and that would take days--which would be unacceptable.  For my environment it's best to do small numbers of device upgrades at one time, then confirm success, talk with end users and Technical Support staff at the upgraded site, and be certain all went as planned before moving on to a different site's upgrades.

    Swift packets to you and your customers!

    Rick Schroeder

  • I think the NCM job does only one device firmware upgrade at a time. Imagine a situation when large setups need to upgrade devices to fix vulnerabilities, it may take ages to cover the firmware upgrades of 10000 devices, if each is going to take about an hour

    Is this Bulk Upgrade feature planned in the next release ?

           

  • I'm interested in learning more about the process that's failing for you.  Where exactly is it halting, where in the process do you need the bulk approval performed.  

    Screen shots may help me better understand your need.

    My team uses Ansible to make large configuration changes to hundreds of switches, tens of thousands of switchports, in one job.

    But we can also use NCM's Upgrade tool to upgrade many switches in one job, and it has a place for custom scripting and the necessary confirming to the switches that we really mean what we scripted.

    Perhaps even easier is building a new NCM Job that duplicates all of the CLI commands you'd need to make config changes or upgrades and reboots.  Just point it at the nodes you need upgraded and let it do its work--even unattended via a schedule.

    Swift Packets!

    Rick Schroeder