UDT timeout on Catalyst 9k switches

I have had this issue for over a year now

UDT never populates any information for devices connected on 9300 switches, on Fuji (16.9.x) or above - I have approximately 80 units behaving exactly the same way

Earlier versions were fine

Tried many different versions of firmware (currently on 16.9.5, but behaviour is the same in Gibraltar 16.12.x)

This problem is getting worse as we swap EOL switches out for 9300 - around 10-15% of our estate has no UDT information

Solarwinds blame Cisco - Cisco blame Solarwinds.

Using UDT Compatibility Checker - vmMembershipSummaryTable returns all the correct VLANs, 

However, it then receives a 'Not Supported' response when querying the 'dot1TpFdbTable' and therefore just times out on when querying all the dot1d tables - suggesting it is a Cisco problem.

But getting them to acknowledge it is painful

Anyone else having issues? I cant be the only person in the world using 9300 switches and UDT? 

  • So, its "normal" for UDT compatibility checker to return "Unsupported" on some tables like that.   On my 3850's there are some predefined VLANS that it picks up (1002, 1003, 1004, 1005) that the compatibility checker returns unsupported on.   Plus, since we remove VLAN 1, it tries to scan that and fails.

    If there are other VLANS defined and its failing scanning those, then you have a problem.   There are some other things I'd try though, such as trying different SNMP versions?   Have you tried creating a view and making sure that MIB is exposed?   Have you tried doing an SNMPWalk to see what you get?   Make sure any SNMP stuff you do, you do both with the default VLAN and with a specific VLAN.  You can specify the VLAN in SNMPv2 by doing "community@VLAN", so if your community string was foobar and the VLAN you wanted was 321, try polling with foobar@321 as the community string.    If you continue to get nothing, try looking for MAC addresses, which is what I think this returns, in a full SNMP walk of the switch to see if Cisco changed what MIB to use...

    I have had a case open with Solarwinds open, for I think what ended up being this MIB, where if you don't have VLAN 1 active, that it will refuse to scan any other VLANs, even though they're present and active.   The fix was to change a value in a config file on the Solarwinds server to tell it to do a full inventory instead of an abbreviated one.   Just changing that value didn't work on its own, also had to do a "List Resources", manual refresh and would have to make sure the VLAN box was checked in List Resources and refresh.   If you think this might be it, let SW support know that it was in a message from them on Nov 11th regarding case 00388484, I'd rather them walk you through it.   You would have to make this change potentially every time you run config wizard also.

  • Thanks for your reply

    I have several vlans defined and it is failing on all of them, unfortunately
    From a working switch, (in this case, a C3850x) the relevant OID is .1.3.6.1.2.1.17.4.3

    2020-03-13_13-00-13.jpg

    I took your advice and did an SNMPWalk on one of my C9300 devices, looks like those MIBs have been completely wiped out of existence!

    2020-03-13_12-51-26.jpg

    Whilst I accept that on the face of it, it looks like a Cisco issue - it simply isn't acceptable for Solarwinds to sell me a product that doesn't work and then expect me to chase the vendor to fix it. It didn't exactly take me long to work out where the problem lies, but as a customer I have very little ability to do anything about it. I am expected to weild enough power with Cisco to get them to reinstate the functionality?
    This issue has existed across at least the last 3 versions of Fuji (16.9.3 > 16.9.5) and I have seen it in Gibraltar (16.12.1) and I have suffered with it for a year

    I have purchased a product from Solarwinds that is supposed to support Cisco products, but for the last year hasn't worked on a rapidly growing percentage of my estate (10-15% at the moment) and Solarwinds don't really seem to care or want to own this issue, instead referring me to Cisco

    This is only going to get worse as people replace infrastructure - C3750 are EOL, C3850 have an EOL announcement on them, Cisco's strategy is to move to a DNA future based on Cat9K - and Solarwinds products don't support them

    Might be a good idea for them to read this thread and take notice?

  • If the device does not provide the OIDs solarwinds needs, then it is what it is.  However, I know that on the latest version, they do have CLI Polling for Layer 3.  I think it's a matter of time it would also be adapted to Layer 2 Polling.

  • In case someone else comes across this thread,   I have Cat 9300's running 17.3.3 and UDT is working on that version.

  • I am just wondering what if anyone has come up with an answer to this? I am running Cisco 9K switches more and more and none of the layer 2 data is populating in UDT. It is very frustrating.

  • Add my organization to this list as well.  We are moving from Cisco 2960s  &  2960x switches to the Cisco 9200R with CAT9k_Lite_IOSXE 17.5.3 and the UDT information is disappearing from our landscape more and more each day.  This is very frustrating since for many many years this information has been readily available and suddenly it's not....

     

  • We are now running 17.6.5 and there is still no Host or IP data populating in UDT.

  • We ran into this same issue as well, we resolved it by moving to SNMPv3 for layer 2/3 polling and filter out the old purpose-use VLANs that come with a Cisco switch (1002-1005) which was causing a timeout for the polling on the backend.  I have it working on 17.6.4, havent tried 17.6.5 yet

  • We ran into this same issue as well, we resolved it by moving to SNMPv3 for layer 2/3 polling and filter out the old purpose-use VLANs that come with a Cisco switch (1002-1005) which was causing a timeout for the polling on the backend.  I have it working on 17.6.4, havent tried 17.6.5 yet

  • Here is an example of a working SNMPv3 config for a 9200/9200L

    no snmp-server system-shutdown
    no snmp-server trap-timeout 180

    snmp-server packetsize 1400
    snmp-server queue-length 20

    snmp-server ifindex persist

    !

    ip access-list standard SNMP

     10 permit <SNMPSERVERIP>

     20 deny any log

    !
    snmp-server contact <CONTACTINFO>
    !

    snmp-server view SNMPv3SWView iso included
    snmp-server group SNMPv3SWGroup v3 priv read SNMPv3SWView write SNMPv3SWView

    snmp-server group SNMPv3SWGroup v3 priv context vlan match prefix

    snmp-server host <SNMPSERVERIP> informs version 3 priv SNMPv3SolarWinds

    !

    snmp-server user SNMPv3SolarWinds SNMPv3SWGroup v3 auth SHA "<SNMPv3PASSWORD>" priv AES 128 "<SNMPv3PASSWORD>" access SNMP