30 Replies Latest reply on Jun 5, 2018 9:29 AM by john.b

    SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)

    john.b

      I am having a problem with Solarwinds NPM 'discovering' new Cisco 3650 switches. I have used the same configuration on them as I have on our Cisco 2960X switches but when I run Add Node, select SNMP v2c, enter the community string and run the test, it fails.

       

      The same switch can be pinged.

       

      Here is a sample of the SNMP config I use on the switches.

       

      snmp-server community xxxxxx RO

      snmp-server community xxxxxx RW

      snmp-server location GRHHOS

      snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart

      snmp-server enable traps envmon fan shutdown supply temperature status

      snmp-server enable traps flash insertion removal

      snmp-server enable traps config

      snmp-server enable traps bridge newroot topologychange

      snmp-server enable traps stpx inconsistency root-inconsistency loop-inconsistency

      snmp-server host xxx.xxx.xxx.xxx xxxxxx

       

      My question is, is there something else that is required for this IOS version for SNMP to work ?

        • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
          john.b

          Further to the above, the version of IoS used is

          SW Image - CAT3K_CAA-UNIVERSALK9

          SW Version - 16.3.1

          Switch Model - WS-C3650-48FQM

          • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
            zennifer

            do a ? after snmp-server host xxx.xxx.xxx.xxx xxxxxx   ... you may need to add v2c and the community string again ....  keep doing the ? until you have completed the host, version, and string

            • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
              CourtesyIT

              Consider moving to SNMPv3.

               

               

              snmp-server user [UserName] [GroupName] v3 auth sha [Keystring] priv aes 128 [AnotherKeyString] access SNMP-Access

              !

              snmp-server group [GroupName] v3 auth access SNMP-Access

              snmp-server group [GroupName] v3 priv read [GroupName]VIEW access SNMP-Access

              snmp-server view [GroupName]VIEW internet included

               

               

              ip access-list standard SNMP-Access

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              permit [SW Server/Poller IP]

              remark --- Review Solarwinds for Device identification 

              deny any log

              • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                CourtesyIT

                Keep in mind [GroupName]VIEW is one word. 

                • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                  zennifer

                  Is there any type of access list that you need to include your polling ip address?

                  Do you need to identify the interface or VLAN to push/pull the traps?

                  Can your Orion server ping the Cisco Switch?

                   

                  Here is a config from my past on a 3750 - has an access list, and defines the port to send the traps.

                   

                  Hope this helps.  

                   

                  logging history errors

                  logging trap warnings

                  logging source-interface Loopback0

                  logging 10.85.111.234

                  access-list 99 permit 10.85.111.244

                  access-list 99 permit 10.85.11.234

                  access-list 99 permit 10.85.111.245

                  access-list 99 permit 10.85.111.230

                  access-list 99 permit 10.85.111.231

                  access-list 99 permit 10.85.111.232 0.0.0.7

                  access-list 99 permit 10.85.111.240 0.0.0.3

                   

                  snmp-server community N3TsurAntr RO 99

                  snmp-server community N3TsurAntrw RW 99

                  snmp-server trap-source Loopback0

                  snmp-server packetsize 4096

                   

                  snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart

                  snmp-server enable traps transceiver all

                  snmp-server enable traps tty

                  snmp-server enable traps eigrp

                  snmp-server enable traps ospf state-change

                  snmp-server enable traps ospf errors

                  snmp-server enable traps ospf retransmit

                  snmp-server enable traps ospf lsa

                  snmp-server enable traps ospf cisco-specific state-change nssa-trans-change

                  snmp-server enable traps ospf cisco-specific state-change shamlink interface-old

                  snmp-server enable traps ospf cisco-specific state-change shamlink neighbor

                  snmp-server enable traps ospf cisco-specific errors

                  snmp-server enable traps ospf cisco-specific retransmit

                  snmp-server enable traps ospf cisco-specific lsa

                  snmp-server enable traps cluster

                  snmp-server enable traps fru-ctrl

                  snmp-server enable traps entity

                  snmp-server enable traps cpu threshold

                  snmp-server enable traps power-ethernet group 1-9

                  snmp-server enable traps power-ethernet police

                  snmp-server enable traps vtp

                  snmp-server enable traps vlancreate

                  snmp-server enable traps vlandelete

                  snmp-server enable traps flash insertion removal

                  snmp-server enable traps port-security

                  snmp-server enable traps auth-framework sec-violation

                  snmp-server enable traps dot1x auth-fail-vlan guest-vlan no-auth-fail-vlan no-guest-vlan

                  snmp-server enable traps envmon fan shutdown supply temperature status

                  snmp-server enable traps stackwise

                  snmp-server enable traps license

                  snmp-server enable traps bgp

                  snmp-server enable traps cef resource-failure peer-state-change peer-fib-state-change inconsistency

                  snmp-server enable traps config-copy

                  snmp-server enable traps config

                  snmp-server enable traps config-ctid

                  snmp-server enable traps event-manager

                  snmp-server enable traps hsrp

                  snmp-server enable traps ipmulticast

                  snmp-server enable traps isis

                  snmp-server enable traps msdp

                  snmp-server enable traps pim neighbor-change rp-mapping-change invalid-pim-message

                  snmp-server enable traps energywise

                  snmp-server enable traps bridge newroot topologychange

                  snmp-server enable traps stpx inconsistency root-inconsistency loop-inconsistency

                  snmp-server enable traps syslog

                  snmp-server enable traps rtr

                  snmp-server enable traps mac-notification change move threshold

                  snmp-server enable traps vlan-membership

                  snmp-server enable traps errdisable

                  snmp-server host 10.85.111.230 version 2c public

                  snmp-server host 10.85.111.231 version 2c public

                  snmp-server host 10.85.111.232 version 2c public

                  snmp-server host 10.85.111.233 version 2c public

                  snmp-server host 10.85.111.234 version 2c N3TsurAntr

                  • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                    CourtesyIT

                    You are correct. You do not need SNMP v3 to make it work.  Get v2c working then test with v3. 

                    • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                      john.b

                      Hi All

                       

                      The config I put in the original post works fine on C2960X-FPS, C3750G, C2950G, C3560-8PC, with a variety of Ios versions but doesn't work on these C3650-FQM.

                       

                      Does anyone have experience of these switches ?

                      • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                        CourtesyIT

                        I do not have any FQM version switches.  But curious about your strings.  Did you try less than 12 characters and only Letters and Numbers, with no specials. 

                        • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                          zennifer

                          So ... you had me .. why is this being difficult for you ... IOS version 16.3.x  is not your normal IOS!!!

                           

                          Please refer to the below link:

                          Software Configuration Guide, Cisco IOS XE Denali 16.3.x (Catalyst 3650 Switches) - Configuring Simple Network Manageme…

                           

                          This IOS software is very powerful, and I am pretty sure that you will have to define an access list.   The basic config ... you have ... but you need to point it to the ORION server.... listed in "Summary Steps".   Pretty sure that if you refer to the article,  it will definitely get you pointed in the right direction. 

                          Keep us posted!

                           

                           

                          Configuring Community Strings

                           

                          You use the SNMP community string to define the relationship between the SNMP manager and the agent. The community string acts like a password to permit access to the agent on the device. Optionally, you can specify one or more of these characteristics associated with the string:

                          • An access list of IP addresses of the SNMP managers that are permitted to use the community string to gain access to the agent
                          • A MIB view, which defines the subset of all MIB objects accessible to the given community
                          • Read and write or read-only permission for the MIB objects accessible to the community

                          Follow these steps to configure a community string on the device.

                          SUMMARY STEPS

                          1.    enable

                          2.    configure terminal

                          3.    snmp-server community string [view view-name] [ro | rw] [access-list-number]

                          4.    access-list access-list-number {deny | permit} source [source-wildcard]

                          5.    end

                          6.    show running-config

                          7.    copy running-config startup-config

                           

                          ******************************  Steps to send it to the ORION server! ***************************************************

                          SUMMARY STEPS

                          1.    enable

                          2.    configure terminal

                          3.    snmp-server engineID remote ip-address engineid-string

                          4.    snmp-server user username group-name {remote host [ udp-port port]} {v1 [access access-list] | v2c [access access-list] | v3 [encrypted] [access access-list] [auth {md5 | sha} auth-password] }

                          5.    snmp-server group group-name {v1 | v2c | v3 {auth | noauth | priv}} [read readview] [writewriteview] [notify notifyview] [access access-list]

                          6.    snmp-server host host-addr [informs | traps] [version {1 | 2c | 3 {auth | noauth | priv}}]community-string [notification-type]

                          7.    snmp-server enable traps notification-types

                          8.    snmp-server trap-source interface-id

                          9.    snmp-server queue-length length

                          10.    snmp-server trap-timeout seconds

                           

                           

                            • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                              designerfx

                              zennifer It's about 100x more simple than that, so I suggest that OP ( john.b  ) return to these basics specifically for SNMPv2 and then go from there. Here's output from my 3650 on the same IOS as I'm SSH'd in, in a lab env and I'm polling using Orion NPM 12.1. Polling with SNMPv2c and "enable 64 bit counters" checked.

                               

                              Total config to get Orion to poll the 3650 everything in bold needs to be defined + setting appropriate traps.

                               

                              #sh run | s 50

                              access-list 50 permit $SYSLOGSERVER

                              access-list 50 permit $ORIONAPE

                              access-list 50 permit $ORIONSERVER

                               

                              #sh run | include snmp-server

                              snmp-server community $STRING RO 50

                              snmp-server community $STRING RW 50

                              snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart

                              snmp-server enable traps vtp

                              snmp-server enable traps vlancreate

                              snmp-server enable traps vlandelete

                              snmp-server enable traps port-security

                              snmp-server enable traps envmon fan shutdown supply temperature

                              snmp-server enable traps flash insertion removal

                              snmp-server enable traps entity

                              snmp-server enable traps ipsla

                              snmp-server enable traps config-copy

                              snmp-server enable traps config

                              snmp-server enable traps hsrp

                              snmp-server enable traps bridge newroot topologychange

                              snmp-server enable traps stpx inconsistency root-inconsistency loop-inconsistency

                              snmp-server enable traps syslog

                              snmp-server enable traps vlan-membership

                              snmp-server enable traps mac-notification change move threshold

                              snmp-server host $ORIONSERVER $STRING

                              snmp-server host $ORIONAPE $STRING

                              snmp-server host $SYSLOGSERVER  $STRING

                            • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                              CourtesyIT

                              john.b,  This may be a shot in the dark, but I recently had an issue with one of my ASR1002 Routers.  Would not answer any of the snmp configurations previously back to my Orion Server.   I ended up doing this to get it to respond.

                               

                              1.  Select Node; Edit Properties

                              2.  Set the SNMP credentials

                              3.  Define a specific Device Template:  Cisco IOS

                              4.  Submit

                              5.  Log on to the Orion Server

                              6.  Set UAC Slider to the bottom

                              7.  Run As Administrator:  Configuration Wizard

                              8.  Finish CW

                              9.  Test Node again, and it started working correctly. 

                              1 of 1 people found this helpful
                              • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                                mthughes

                                Hi,

                                 

                                There is no difference between the IOS and IOS-XE configuration for the simple SNMP version 2c config. Maybe there is a bug in the IOS-XE code your switch is running. The commands shown for the Denali version of IOS-XE do not look any different to me either.

                                 

                                I would remove all SNMP commands from your switch and apply only the most basic SNMP configuration for polling SNMP from the Cisco switch (no trap configuration). I use the following on all my IOS and IOS-XE Cisco equipment,

                                 

                                 

                                 

                                snmp-server community COMM_STRING RO SNMP_READ_ONLY

                                 

                                 

                                 

                                ip access-list standard SNMP_READ_ONLY

                                remark Production SolarWinds Server

                                permit 01.02.03.04

                                I am running IOS-XE version 03.03.05SE on 3850 switches - it is the cat3k_caa-universalk9 SW Image.  This is a package install.

                                 

                                XXXXXXX-SW#show version | in image    

                                System image file is "flash:packages.conf"

                                 

                                XXXXXXX-SW#more packages.conf

                                #! /usr/binos/bin/packages_conf.sh


                                iso   rp 0 0   rp_base       cat3k_caa-base.SPA.03.03.05SE.pkg

                                iso   rp 0 0   rp_infra       cat3k_caa-infra.SPA.03.03.05SE.pkg

                                iso   rp 0 0   rp_platform       cat3k_caa-platform.SPA.03.03.05SE.pkg

                                iso   rp 0 0   rp_iosd       cat3k_caa-iosd-universalk9.SPA.150-1.EZ5.pkg

                                iso   rp 0 0   rp_wcm       cat3k_caa-wcm-ldpe.SPA.10.1.150.0.pkg

                                iso   rp 0 0   drivers       cat3k_caa-drivers.SPA.03.03.05SE.pkg

                                Double check that you do not have any ACLs that could be blocking traffic, that your source IP for your SolarWinds server is not being NATd somewhere in the network before reaching the switch and if everything looks good and it still does not work, reload the switch if you can.

                                  • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                                    john.b

                                    Thanks for the suggestion, which I have now tried. Removed all current SNMP config and applied the basics as per your example. I deleted the switch from NPM.

                                     

                                    I then added it again and ran the test for snmp v2 and it passed. Clicked Next to list resources and once again it just hung on the spinning 'doing something' circle. When it eventually timed out I went back a page, re-run the test and, as before, it failed.

                                     

                                    The 3650's are running this version

                                     

                                    iso   rp 0 0   rp_base        cat3k_caa-rpbase.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_core        cat3k_caa-rpcore.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_daemons     cat3k_caa-rpcore.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_iosd        cat3k_caa-rpcore.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_wcm         cat3k_caa-wcm.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_webui       cat3k_caa-webui.16.03.01.SPA.pkg
                                    iso   rp 0 0   srdriver       cat3k_caa-srdriver.16.03.01.SPA.pkg
                                    iso   rp 0 0   rp_security    cat3k_caa-rpcore.16.03.01.SPA.pkg

                                    iso   fp 0 0   fp             cat3k_caa-rpcore.16.03.01.SPA.pkg

                                     

                                    We also have Cisco 3850's running the XE Ios, version below, and SNMP works fine with them.

                                     

                                    iso   rp 0 0   rp_base       cat3k_caa-base.SPA.03.07.04E.pkg

                                    iso   rp 0 0   rp_infra       cat3k_caa-infra.SPA.03.07.04E.pkg

                                    iso   rp 0 0   rp_platform       cat3k_caa-platform.SPA.03.07.04E.pkg

                                    iso   rp 0 0   rp_iosd       cat3k_caa-iosd-universalk9.SPA.152-3.E4.pkg

                                    iso   rp 0 0   rp_wcm       cat3k_caa-wcm.SPA.10.3.141.0.pkg

                                    iso   rp 0 0   drivers       cat3k_caa-drivers.SPA.03.07.04E.pkg

                                      • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                                        zennifer

                                        Do you have logging enabled on that switch?   Does the log present anything?

                                         

                                        It passes authentication, then it fails authentication.... it is starting to sound like a "buggy" switch or software... you may want to upgrade / downgrade the IOS software.  You should not  be struggling with this configuration!!!   I am amazed ... have you called Cisco TAC?   It sure sounds like an issue with the switch.

                                         

                                        Keep us posted!

                                        • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                                          mthughes

                                          Hi John,

                                           

                                            I would suggest that you try to do an snmpwalk against your switch. It that works, it will show there is some issue with your SolarWinds server (at least for these switches). If it doesn't work then you will at least know it is an issue on the switch/IOS side.  You can run snmpwalk from any Mac or Linux server. I use the following to get the full snmp support mib with the OID translated to names and with their full numerical OIDs (which you need in SolarWinds to create a UnDP).

                                           

                                          snmpwalk -v 2c -c snmpstring 01.02.03.04

                                          snmpwalk -v 2c -c snmpstring 01.02.03.04 -Ofn .1

                                          Make sure to add the computer's source IP to your snmp ACL.

                                           

                                          It could be something as simple as the SolarWinds poller not correctly identifying the switch's system OID and/or not having it in the system.

                                          Maybe the SolarWinds logs will give you a better idea of the issue once you figure out if this is an issue on your switch or not.

                                           

                                          I am not sure which log file you need to view for the poller discovery portion. It is most likely in here (C:\ProgramData\SolarWinds\Logs\Orion) though. You can initiate the polling again and then sort them by date to find the correct log.

                                           

                                          I hope this helps.

                                      • Re: SNMP on Cisco IOS XE Release 3SE (Catalyst 3650 Switches)
                                        john.b

                                        Thank you all for your suggestions, the problem is now fixed.

                                         

                                        The fix was, an upgrade of the IoS from 16.03.01 to 16.03.06

                                         

                                        Once installed, it now logs correctly with Solarwinds.