5 Replies Latest reply on May 31, 2008 7:03 AM by zmerch

    Switch Port Mapper Problems

      I was pretty excited to see the switch port mapper program, however, it doesn't seem to work on the majority of my switches. An example is a closet with two 2950T switches. One Gig port on each switch goes back to the backbone switch, the other Gig ports are connected to each other but placed in blocking by spanning tree for redundancy. Every time I try to run the switch port mapper I get this:

      Run-time error '457':
      This key is already associated with an element of this collection.

      If I shut down the redundant Gig port on the switch, I don't get this error anymore, instead switch port mapper stops responding part of the way through the process and my switch is dead for about 60 seconds. It does not reboot, it simply stops responding to network traffic, and then, just as suddenly, starts again.

      Anyone else had similar problems with switch port mapper? It seems to run fine on switches that I don't have deployed a redundant spanning tree design. I've tried it from multiple machines, so I don't think the problem is my install of SolarWinds.

      RS
        • Re: Switch Port Mapper Problems
          I was pretty excited to see the switch port mapper program, however, it doesn't seem to work on the majority of my switches....

          Run-time error '457':
          This key is already associated with an element of this collection.
          RS


          I found out why this is happening (as I also have an older version of the Port Mapper software and searched for this error) - Certain switches can return multiple exact MIBs while still continuing through the SNMPwalk - but the array type that SolarWinds used cannot handle multiple like entries in their index; and instead of checking to see if the index already exists, ignoring that entry & continuing it's abending.

          Linux SNMP tools can help diagnose the problem if you have access to a linux box with them - there are also freeware tools for Win32 - google is your friend here. This is what I did:

          snmpwalk -On -c mycommunityhere -v 2c -Ct -Cc 10.123.456.789 > AllDellSwitchInfo.txt

          [[ If you leave off the -Cc switch, the SNMPwalk command will barf on the first duplicate (or non-increasing) MIB... but if you left it in, you can use linux to find the offending lines pretty easily:  ]]

          awk '{print $1}' < AllDellSwitchInfo.txt | uniq -d

          These are the repeated MIBs on my Dell PowerConnect switch:

          .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.153.18 = Hex-STRING: 00 16 C7 2C 99 12
          .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.185.191 = Hex-STRING: 00 16 C7 2C B9 BF

          There were 3 instances of each of these MIBs in the full SNMPWalk.

          Hope this helps!
          Roger "Merch" Merchberger 

            • Re: Switch Port Mapper Problems

               Roger - Is it returning different ports for each mac or the same port multiple times?  Do those ports resolve to the same interface or different physical interfaces?

               

              Thanks,

              Greg 

                • Re: Switch Port Mapper Problems

                  It's tough for me to tell - I know enough about SNMP & SolarWinds to fix things when they break, but I don't know enough to know MIBs by heart or anything... I widened some of the search parameters, and this is what I found: 

                  root@linux:/var/log# grep -i '0.16.C7' AllDellInfo.txt
                  .1.3.6.1.2.1.4.22.1.2.1.10.102.1.254 = STRING: 0:16:c7:2c:b9:bf
                  .1.3.6.1.2.1.4.22.1.2.1.10.102.7.1 = STRING: 0:16:c7:2c:b9:bf
                  .1.3.6.1.2.1.4.22.1.2.1.10.102.10.1 = STRING: 0:16:c7:2c:b9:bf
                  .1.3.6.1.2.1.4.22.1.2.1.10.102.11.1 = STRING: 0:16:c7:2c:b9:bf
                  .1.3.6.1.2.1.17.2.15.1.8.49 = Hex-STRING: 80 01 00 16 C7 2C 99 00
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.153.18 = Hex-STRING: 00 16 C7 2C 99 12
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.153.18 = Hex-STRING: 00 16 C7 2C 99 12
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.153.18 = Hex-STRING: 00 16 C7 2C 99 12
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.185.191 = Hex-STRING: 00 16 C7 2C B9 BF
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.185.191 = Hex-STRING: 00 16 C7 2C B9 BF
                  .1.3.6.1.2.1.17.4.3.1.1.0.22.199.44.185.191 = Hex-STRING: 00 16 C7 2C B9 BF

                  To someone who doesn't know much about these things, I'd have to surmise that it's 'secondary data' being relayed in the later repeated MIBs, so I don't think it's an "IP->MAC" or vice-versa issue here. If I can forge a guess of "Most IP address based MIBs will be the last 4 octets thereof" we certainly don't have anything in the 199.44.x.x type address, and you can see earlier that the MAC address is sitting squarely in the 10.x.x.x Class A which is where it should be. ;-)

                  I did an SNMPwalk of just '.1.3.6.1.2.1.4.22' and it returned all the MAC addresses & IPs on the switch and it didn't return any duplicate data (and did return pretty much exactly what I was looking for... yay!), so with that I also don't think it's a port-reporting problem.

                  I pretty much figured that it's just a bug in Dell's firmware, but when I googled, folks said there were issues with Ciscos & other equipment, too. I realize that there shouldn't be duplicate MIBs... but I learned long ago that shoulds's and is's can vary greatly. ;-)

                  I can make the whole SNMPwalk file available to you if it will help you. It's about 1Meg uncompressed, but the .gzip is only 82K.

                  Thanks, and hope this helps!
                  Roger "Merch" Merchberger 

                    • Re: Switch Port Mapper Problems

                      Thanks Roger - Can you please walk the following MIB?

                       
                      1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18

                       
                      This will be the ports column of the forwarding table. It relates the MAC to the port to send it out on.

                      Thanks,

                      Greg 

                        • Re: Switch Port Mapper Problems

                          Thanks Roger - Can you please walk the following MIB?
                          1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18

                          Sorry for the late response - I have an odd workweek, and the 2-day delay was my "weekend."
                          Here's the SNMPwalk:

                          snmpwalk -On -c ssmtcro -v 2c -Ct -Cc 10.102.20.13 .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18
                          .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18 = INTEGER: 49
                          .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18 = INTEGER: 49
                          .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18 = INTEGER: 49
                          Total traversal time = 0.031000 seconds

                          When I take out the "keep going no matter what flag" of -Cc, this is what I get:

                          snmpwalk -On -c ssmtcro -v 2c -Ct 10.102.20.13 .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18
                          .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18 = INTEGER: 49
                          Error: OID not increasing: .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18
                           >= .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18

                          (not surprising, but not as "destructive" as the manpage would lead one to believe... i.e. endless loops, etc.) but an SNMPget on the MIB works fine / doesn't error out:

                          snmpget -On -c ssmtcro -v 2c 10.102.20.13 .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18
                          .1.3.6.1.2.1.17.4.3.1.2.0.22.199.44.153.18 = INTEGER: 49

                          To hazard a guess, maybe Dell did their testing with an array of MIBs (which they would obviously known ahead of time and looped through them with SNMPget & never came across the bug?

                          Sad part is - if these things used 6809s, I could disassemble the firmware & patch it... yea, I'm *old*. ;-)

                          Thanks a lot for the help!
                          Roger "Merch" Merchberger