13 Replies Latest reply on May 10, 2019 8:10 AM by rjrothwell

    Firmware Upgrades for stacked Cisco switches

    keles

      I would like to ask how do you handle Firmware upgrades for stacked switches. I found a workaround which is now satisfying in my case. My workaround is quite simple:

      As you have the problem, that you cannot upgrade multiple stacked switches with the .bin-file within this firmware upgrade process, I´ve created a pseudo .bin-file to satisfy the SolarWinds process. This pseudo .bin-file is uploaded to the (master) switch by upgrading. The real upgrading process is triggered by the "archive download-sw" command using the .tar-file. Here is a screenshot how a template could look like:

       

       

       

       

      So, my question now is, do you know if there is an official solution from SolarWinds for upgrading stacked switches?

        • Re: Firmware Upgrades for stacked Cisco switches
          beadlek

          We have the same issue.

           

          Please could you give some more information on your ingenious workaround;

           

          I have a .tar file but what did you do to create the ciscpseudoios.bin file, i.e. did you simply rename the extension?

           

          'Upgrade firmware image command'; I see the /overwrite http://IP.address.of.location.of.the.file/xxx/xxx but what is this referring to?

           

          any help is really appreciated.

            • Re: Firmware Upgrades for stacked Cisco switches
              keles

              Hello beadlek

               

              first of all I want to apologize, because of my very late response to your question. I am using the ciscopseudoios.bin, to be able to use this solarwinds operation. As you know, you can just update stacked cisco switches with .tar file and not with the .bin file. Solarwinds just supports .bin files for the upgrade operation. I just thought, ok, I will create an empty .bin file and afterwards in the upgrade command I just push the new IOS with the archive download-sw command. After some testing on different devices, I figured out that this way is working properly (for me). So the real upgrade process is hidden in the archive download-sw command. the http path is just the ftp share, where I stored the .tar file, in order to be more flexible while upgrading.

              Meanwhile I am using this method for almost all devices, where the archive download-sw command is working.

                • Re: Firmware Upgrades for stacked Cisco switches
                  llemieux

                  Can you explain how you created the "ciscopseudoios.bin" file? Did you just name a text file with a .bin extension?

                    • Re: Firmware Upgrades for stacked Cisco switches
                      llemieux

                      Okay so I just did exactly that. Created a text file with jibberish and gave it a ".bin" extension.

                       

                      So I have the following template being used for this, mimicking keles image above:
                      As mentioned in the original post, the most important part of this is the "archive download-sw" command. I am using the "/leave-old-sw" switch to leave the current IOS on the device (since I have the space in flash) and the "/imageonly" switch to help speed the process up since we do not manage switches via web browser.

                       

                      What SolarWinds will do when running this template against the selected node, is first backup the current IOS to SolarWinds TFTP-root location, this is the "TFTP-Root" located on the C:\ of your Orion Server.

                       

                      Once the current IOS is backed up, it will move the backup .bin file it creates to the "Image Repository" location you specified in the firmware upgrade settings in NCM.

                       

                      SolarWinds will then move the CiscoPseudo.bin file I created into that TFTP-root directory since it thinks this is the IOS I am using, BUT HERE IS THE MOST IMPORTANT PART:

                       

                      Make sure you place the ACTUAL IOS image in the TFTP-Root directory BEFORE running the upgrade process as your command, within the template, points to this file instead of the "CIscoPseudo":

                       

                      Again, I'll repeat, SolarWinds will move the "CiscoPseudo.bin" file into this directory because it thinks this is the IOS image we are using and can only work with .bin files. The ACTUAL IOS is the "c2960s-universalk9-tar.150-2.SE11.tar" which I MANUALLY placed within this directory. The "archive download-sw" command specified in the template is the actual command that will be initiated on the switch. That "CiscoPseudo.bin" file will just continue to sit there unused.  

                       

                      I tested this out on a pair of unused switches I had configured in stack and it worked perfectly. As always, NEVER TEST ON PRODUCTION. Thanks keles!

                      1 of 1 people found this helpful
                • Re: Firmware Upgrades for stacked Cisco switches
                  marnell

                  Do I need to run any scripts to upgrade my firmware of a cisco 3560 8port swith or once I have NCM push to the system it will reload and update it?

                  • Re: Firmware Upgrades for stacked Cisco switches
                    bmallon

                    Are you kidding?

                    SolarWinds hardly supports switch stacks and FEX's. (Apparently the technology is too new)

                     

                    It's truly a shame SolarWinds hasn't tackled this issue head on. Instead, they rely on us to make our own custom pollers for this kind of thing.

                    • Re: Firmware Upgrades for stacked Cisco switches
                      elien

                      Hello Keles,

                       

                      Is it possible if you place the template files here ? I am in the processing of upgrading our cisco 2960 X stack as well. The picture is great, its just that i am not to familiar on tar files and empty bin files. A detail instructions will be awesome.

                       

                      Thanks

                      • Re: Firmware Upgrades for stacked Cisco switches
                        marnell

                        Does anyone have a video to see the process? I am trying to see if I can get a test lab switch to a stack switch ISO upgrade.

                        • Re: Firmware Upgrades for stacked Cisco switches
                          tclawrence2014

                          I am having the same issue.  I'm trying to do this process with Catalyst 3850/9300 stacks in install mode.  Commands for this are roughly:

                           

                          1 - copy scp://$xxxxxxxxx:password@x.x.x.x/image.bin flash:

                          2 – verify /md5 flash:image.bin (whatever hash value it gets from Cisco)

                          3 -- request platform software package install switch all file flash:image.bin auto-copy

                          4 -"reload" will reload the whole stack

                           

                          For whatever reason, when I customize the fields to include these commands it ends up spitting out the monolithic or bundled mode config..

                           

                          • Re: Firmware Upgrades for stacked Cisco switches
                            christopher.t.jones123

                            If you go to the Advanced Configuration page for SolarWinds you can adjust the file types that the firmware repository can see

                             

                            Add file extensions to NCM Firmware upgrade repo

                            • Re: Firmware Upgrades for stacked Cisco switches
                              rjrothwell

                              So here is my work around for our Cisco 2960x Stacks. We do not use the Firmware Upgrade through Orion due to the lack of Stack Support. I created a script and just ran that during the day, to stage for the upgrade to run as a job at night/early morning. This only spikes the CPU by less than 10% so it is not service impacting. I created a copy script based on the number of switches within that stack.

                               

                              copy tftp://ServerIP/c2960x/c2960x-universalk9-mz.152-4.E7.bin flash1:/

                              copy tftp://ServerIP/c2960x/c2960x-universalk9-mz.152-4.E7.bin flash2:/

                              copy tftp://ServerIP/c2960x/c2960x-universalk9-mz.152-4.E7.bin flash3:/

                              dir flash1:/ ${SuccessRegEx:c2960x-universalk9-mz.152-4.E7.bin}

                              verify /md5 flash1:/c2960x-universalk9-mz.152-4.E7.bin c8148471111b33794391521dadb47ed6 ${SuccessRegEx:Verified}

                              dir flash2:/ ${SuccessRegEx:c2960x-universalk9-mz.152-4.E7.bin}

                              verify /md5 flash2:/c2960x-universalk9-mz.152-4.E7.bin c8148471111b33794391521dadb47ed6 ${SuccessRegEx:Verified}

                              dir flash3:/ ${SuccessRegEx:c2960x-universalk9-mz.152-4.E7.bin}

                              verify /md5 flash3:/c2960x-universalk9-mz.152-4.E7.bin c8148471111b33794391521dadb47ed6 ${SuccessRegEx:Verified}

                               

                              You must copy the BIN file to each of the switch's flash. This is for a 3 Stack Switch. By adding the ${SuccessRegEx:....}, will cause the script to error out if the check fails. I found it took on average of 7 minutes per switch to copy the flash and Verify the MD5 Hash.

                               

                              Now for the time of the upgrade, I would run a job with this script at 4:00 am.

                               

                              config terminal

                              no boot system

                              boot system flash:/c2960x-universalk9-mz.152-4.E7.bin

                              end

                              write memory

                               

                              Next I run the reboot Job on the switches at 5:00 am.

                               

                              If there is a storm coming like there in North FL/South GA area, I would suggest holding off on the upgrade. If the ROM needs to be flashed during the first boot of that OS and power is lost to the switch, you could be coming into work with a stack of bricked switches.

                               

                              I remember reaching in the relase notes for the move recent version of config manager, one of the new features is better stack support.

                               

                              I have not had any success with the ${SuccessRegEx:....} with the Cisco 4500s in VSS, but it does save the output of the script under the Transfer Status and Show Script Results. You can always check it after the copy and Verify before the actual upgrade job.