Using Compliance Reporting to Verify Juniper Alternate Slice Version

Version 1

    About a year ago I started a quest to figure out how to achieve this goal.  Since I've made some major headway I decided to share it out with the community in case it comes in handy.  There's probably better ways to do this so please comment if you have improvements!  Also, this only applies to EX (standalone and in VCs) and SRX devices; QFXes use virtual machines so it doesn't apply to them, and I don't have any other Juniper devices to verify whether or not this works there.

     

    The problem is that when you update firmware on a Juniper device the primary partition gets updated automatically but the backup partition does not.  You must run request system snapshot slice alternate to copy the active partition over so that if you have a failover of the primary the backup is running the same version.  This is particularly important in virtual-chassis setups where mismatched firmware can cause major problems.

     

    The only way to see the versions of backup vs. primary is from operational mode; you run "show system snapshot media internal".  There's no way to pull this information from SNMP.  So we need a system whereby we can run that command, pull in the resulting output, and search the versions on primary and backup, then report when it's different than expected.

     

    ====================

     

     

    Step one is to create a custom configuration type and then modify or create a new device template for NCM so that it knows what command to run.

     

    Under Settings->NCM Settings->Config Settings, add a new config type and record the name exactly:

    1.png

    Under Settings->NCM Settings->Device Template Management, copy the Juniper device template and give it a name you can remember.  Add a command name with the config type you created previously, and for the value you want to put inside the quotes "system snapshot media internal".  This way, when it goes to download the config it will run the DownloadConfig command which is "show ${ConfigType}".  That will in turn run "show system snapshot media internal" on the device.  Save your template and assign it to your Juniper nodes.

         2.png

    After you've confirmed you're able to download that config from a device, let's move on to step two.

         3.png4.png    

    ==============================

     

    In step two we have to search and compare the configs that are generated.  Compliance reporting is best for this.  This part will be fairly difficult because depending on EX, EX in a VC, or SRX the output of the show command is different, and also because the order in which primary and backup are displayed varies randomly.  In addition, because compliance reports only search one line at a time, we have to make separate rules for backup and primary slices and then compare them to a known desired version, rather than just comparing them to themselves and verifying they're the same.  Add all that up and it equals many rules we need to specify.  I'll go through an SRX example and an EX / VC example and you can iterate based on whatever versions you want to look at.

     

         In Settings->NCM Settings->Manage Rules, create a new rule.  I've named mine with what type of device, what version I'm looking at, and whether I'm looking at backup or primary partition.

              5.png

    You want advanced config search enabled, alerting when the string is NOT found, and searching in a config block using regex.  Here's a screenshot of the settings I'm using.  Note all you need to replace here is the version you're looking for, but make sure you include "ex-" because we're only looking for the jbase version.  All others should be the same as that.

              6.png

    I've verified this search will work regardless of where the backup "stanza" falls in the config output.  It just checks to make sure the jbase version is whatever you set.  Again remember to include "ex-" because jbase is the only line that reports its version that way.  Here's an example config to show what I'm talking about (jbase bolded):

    fpc0:
    --------------------------------------------------------------------------
    Information for snapshot on       internal (/dev/da0s1a) (primary)
    Creation date: Sep 4 13:30:42 2015
    JUNOS version on snapshot:
      jbase  : ex-12.2R10.2
      jkernel-ex-2200: 12.3R10.2
      jcrypto-ex: 12.3R10.2
      jdocs-ex: 12.3R10.2
      jswitch-ex: 12.3R10.2
      jpfe-ex22x: 12.3R10.2
      jroute-ex: 12.3R10.2
      jweb-ex: 12.3R10.2
      fips-mode-arm: 12.3R10.2
     Information for snapshot on       internal (/dev/da0s2a) (backup)
    Creation date: Sep 4 14:59:50 2015
    JUNOS version on snapshot:
      jbase  : ex-12.3R10.2
      jkernel-ex-2200: 12.3R10.2
      jcrypto-ex: 12.3R10.2
      jdocs-ex: 12.3R10.2
      jswitch-ex: 12.3R10.2
      jpfe-ex22x: 12.3R10.2
      jroute-ex: 12.3R10.2
      jweb-ex: 12.3R10.2
      fips-mode-arm: 12.3R10.2
    

     

    Next, copy this rule and change the word primary in the config block start regex to search the primary config.  Ideally when you're searching for a version this rule should never trigger but it's nice to have, and if you apply the policy to every juniper device could help you find whether a device is not running the version of code you think it is.

     

    The SRX rules vary a bit because again the output of the show command is different.  Use these settings for an SRX backup partition:

    7.png

    Copy and replace backup with primary for the primary partition.  Adjust version numbers as desired.

     

    Now you just need to create policies and add in your rules as desired.  In my case I created different policies based on the version I wanted to look at, and then added the backup and primary rule for that given version to it.  I filtered my devices based on OS version because I specifically wanted to just check that the backup version was the same as the primary for a given JunOS version, but you can do whatever you'd like there.  Make sure your policies run against the Junipersnapshot config type only.

    8.png

    After you have your policies, create a report and add all of them in.  Almost done!

    9.png

    ==================

     

    Step three is testing.  Download a Junipersnapshot config for all devices you assigned that config and device type to.  Then, in compliance reports, highlight your report and run "Update selected".  Run it and you should get results:

    10.png

     

    ============

    Step four: Profit!  Let me know what you think!