25 Replies Latest reply: Mar 25, 2015 10:31 AM by mrm@vmware.com RSS

    SNMPv3 with ESXi?




      Putting this in the NPM group, since it is an SNMP issue....


      Please don't ask me why (long story), but I am trying to get NPM to poll an ESXi server.  Anyone get this to work?  I have the ESXi 5.1 system configured for SNMPv3, however when I run a test connect from NPM it fails.  The ESXi is spitting out a syslog message that indicates NPM has requested authNoPriv[2}, and I have it set for both authorization and privacy, so the resulting connection is not supported.... I have both the authorization and privacy fields configured in NPM, so I am not sure why it is sending that kind of request.  As a test, I dropped the ESXi back to auth only, and it quit giving errors, but it also still fails with no errors at the NPM end.


      If anyone has the magic beans for the configuration on the ESXi side I might be missing, I'd appreciate them.  I have tried this on two different ESXi boxes, and get the same results.


      Note that I only need polling at this time, not traps.


      I'll probably open a ticket on this as well.





        • Re: SNMPv3 with ESXi?

          Here's some further documentation on SNMPv3 in ESXi 5.1: VMware vSphere 5.1

            • Re: SNMPv3 with ESXi?

              (OK, I'll try this edit again...)  Thanks for the input..  I have spent a lot of time in the VMWare sites/instructions including a couple of folks that posted step-by-step instructions for enabling V3 (that matched).  From what I can tell, I have it functioning. Have you gotten it to work with NPM?  I have the settings looking OK and locally tested, but still getting a connect failure  from NPM.  I opened a ticket.


              I was hoping to find out if anyone had this actually working with NPM.  Then I'd know which way to push my energies.  ESXi is getting authnopriv requests from NPM when it should be getting authpriv requests.  When I drop the priv requirement at the ESXi client, it is fine with the request but NPM says "test failed".  No clue what NPM is looking for at that point.  I know  ESXi really would like an Engine ID defined from NPM up front, but have not see any way to get that, and in any case ESXi is not detecting any problems with the request at that point. Sooo....






            • Re: SNMPv3 with ESXi?

              Has anyone found a fix for this?  I really would like to get this setup so here's hoping someone found something.

                • Re: SNMPv3 with ESXi?

                  It's been yet another month and this still isn't working right. I just checked and verified that SNMPv3 does not work right on ESXi 5.5, either.

                    • Re: SNMPv3 with ESXi?

                      I actually heard back from support regarding my case, and here's what they said:


                      Here is developments  summary of the problem found during comparison of net-snmp snmpwalk and SolarWinds SNMP library behavior:

                      net-snmp implementation of SNMPv3 handshake:
                      1. send "get-request" with msgFlags set to Reportable, msgAuthoritativeEngineID <MISSING>, msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime sets to 0
                      2. Device responds with "report" message with msgAuthoritativeEngineID set to own value and msgAuthoritativeEngineBoots, msgAuthoritativeEngineTime sets to 0
                      3. net-snmp send "get-request" with
                      msgAuthoritativeEngineID set to value obtained previously and msgAuthoritativeEngineBoots, msgAuthoritativeEngineTime sets also to values obtained previously. None check for values which are wrong is placed here.

                      SolarWinds SNMP library implementation of SNMPv3 handshake:
                      1. send "get-request" with msgFlags set to Reportable, msgAuthoritativeEngineID <MISSING>, msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime sets to 0
                      2. Device responds with "report" message with msgAuthoritativeEngineID set to own value and msgAuthoritativeEngineBoots, msgAuthoritativeEngineTime sets to 0 -- this is root cause of problem
                      3. SNMP library send "get-request" with msgFlags set to Reportable and Authenticated only
                      msgAuthoritativeEngineID set to previously obtained value, msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime sets to 0
                      - it's because device responds with msgAuthoritativeEngineBoots, msgAuthoritativeEngineTime sets to 0, which is wrong value
                      4. device should responds with "report" with variable binding set to (usmStatsNotInTimeWindows) and set msgAuthoritativeEngineBoots, msgAuthoritativeEngineTime sets to some reasonable value. But device in reality responds with (usmStatsUnsupportedSecurityLevel) and msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime is still sets to 0

                      SolarWinds implementation fail with adding ESXi device because of difference in behavior in step 3,4. We check msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime value and refuse to communicate with such device until obtain real device time value. We are doing it because of security concern.

                      I believe that Solarwinds approach is correct, because msgAuthoritativeEngineTime must be set by initiator of connection and device must increase this counter over time as described in RFC2574 http://www.ietf.org/rfc/rfc2574.txt otherwise is vulnerable to replay attack.


                      Development would like you to open a ticket about the replay attack security issue with ESXi support.


                      So it sounds to me like Solarwinds won't be doing anything, as they believe this is an incorrect implementation of SNMPv3 behavior within ESX.

                  • Re: SNMPv3 with ESXi?


                    I have NPM polling my ESXi 5.1 server using SNMPv3, but it is auth-only. Whenever I tried to use AES128 for priv, it failed. Here are the steps that I used:


                    esxcli system snmp set -r               # this resets all SNMP on the VM host

                    esxcli system snmp set -a=MD5     # I use MD5; have not tried SHA1

                    esxcli system snmp hash -r -A <authPassword>     # creates the auth-hash needed when configuring the user

                    esxcli system snmp -u=<username>/<auth-hash>/-/auth          # creates the snmpv3 user

                    esxcli system snmp -e=true          # enables snmp


                    Still trying to figure out why priv doesn't work, but at least this got NPM to poll it.


                    Good luck,


                    • Re: SNMPv3 with ESXi?

                      If you are running Orion NPM on a Microsoft 2K3 server there is a known issue with communication between ESXi 5.5 and Windows 2003 due to ESXi 5.5 using 2048 bit security certs which Windows 2003 doesn't support.

                      Microsoft has a KB which addresses this functionality:



                      I was getting "Test Failed. Cannot login with selected vCenter or ESX credential". Hope this helps.


                      Case #708443


                      • Re: SNMPv3 with ESXi?

                        There's an issue with the agent in ESXi not updating engine boots/time flags (B=, T=),

                        I expect a KB article real soon now from vmware that would explain the fix needed.

                        But if you're having problems with SNMPv3 here's some basic tips with ESXi

                        esxcli system snmp set --engineid 0x0102030405060708  # a unique value for each system

                        #pick cipher for authentication

                        esxcli system snmp set --authentication SHA1"

                        # avoid MD5 is no longer cryptographically safe to use, instead use SHA1 and also vote for approval of SHA2 now! send email to opsawg@ietf.com

                        # supporting working group last call on draft-ietf-opsawg-hmac-sha-2-usm-snmp-03

                        # pick cipher for crypto if needed else can just use authentication to get started

                        esxcli system snmp set --privacy AES128

                        # and at some point IETF needs to use longer keys too which hasn't been standardized

                        # Next put two passwords in a file or provide them on the command line and use --raw-secret, its less secure to put paswords on cmd line so I'm told)

                        esxcli system snmp hash --auth-hash authpassword --priv-hash privpassword  [--raw-secret]

                        # define the user

                        esxcli system snmp set --users janedoe/a/b/priv" where a and b are the localized hash generated from your password, cipher and engine id.

                        # configure where to send traps if needed using this id

                        esxcli system snmp set --v3targets

                        # turn on the agent (pidof snmpd -> shows running process)

                        esxcli system snmp set --enable true

                        Type: esxcli system snmp get

                        to see your config




                        To send a warmStart trap on demand

                        esxcli system snmp test


                        To verify your v3 user's configuration works

                        esxcli system snmp test  -u janedoe


                        To see what's going on between an snmpv3 manager and agent for outbound traps, in another ESXi shell:


                        tcpdump-uw -vv -i vmk0 -T snmp udp and port 162

                          tcpdump-uw: listening on vmk0, link-type EN10MB (Ethernet), capture size 96 bytes

                                21:53:46.264332 IP truncated-ip - 90 bytes missing! (tos 0x0, ttl 64, id 17994, offset 0, flags [none], proto UDP (17), length 172)

                          promb-2s-dhcp95-91.eng.vmware.com.51285 > mrm-pc.eng.vmware.com.1162:  { SNMPv3 { F=ap } { USM B=0 T=0 U=janedoe [|snmp]} { ScopedPDU [|snmp]} }

                        Notice the B=0 and T=0, usually T=0 must increment but that's the rub its not right now....