This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Unable to poll VMware (it was after 5.5U3 on some and 6.0U1)

Anybody have having/had issue polling their vCenter?  I opened a case (963304) with support three weeks ago, but they haven't made any progress and are trying to get me to adjust my retention.

We removed and re-added one of my vCenters, and it won't list any host under it.

If I add an vSphere host itself, it won't list any guests.

We tried multiple different user accounts, all test out fine, but no matter how long we leave it, the guests are not listed.

We used a web browser to look at the MOB directly and it works.

Virtualization Manager is able to poll it without any issue (for testing, we removed the integration)

Here is what I am running: Orion Platform 2015.1.3, WPM 2.2.1, NCM 7.4, NPM 11.5.3, DPA 10.0.1, QoE 2.0, IVIM 2.1.1, SAM 6.2.3

I have re-installed (and repaired, and marked for compatibility):

SolarWinds Integrated Virtual Infrastructure Monitor

JobEngine

JobEngine.v2

.Net 4.5

Upgraded

  NPM 11.5.3 (caused me to need a buddy drop because I could not list resources after that)

  APM 6.2.3

  IVIM 2.1.1 (I download it, but one of the previous upgrades did it for me)

I don't know what else to do, an suggestions?

Hey, aLTeReGo​, if this is the wrong place, can you move it (or tell me where/who is the PM, or have a moderator do it)? Thank you!

  • Hi, this looks like a known issue. In those versions VMware changed number of allowed elements in queries. This happens only when the polling is done through Orion, Virtualization Manager doesn't have this issue. There are actually two workarounds which you can try and might be able to help.

    1. There is a KB article by VMware which describes setting that can be changed to increase number of allowed elements in queries: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2107096
    2. You can modify a configuration file on SolarWinds side, this will cause that we will execute more smaller queries. To do so:
      1. Backup and modify SolarWinds.VIM.Poller.dll.config (by default in: c:\Program Files (x86)\SolarWinds\Orion\VIM\SolarWinds.VIM.Pollers.dll.config)
      2. Change the value of VMwarePerformanceCounter.ChunkSize to something smaller (for example 3, but you will need to experiment a bit with what should be the correct value)
      3. After you save the change restart Orion services

    Hopefully one of these should help.

  • Changed the vmware side to 128 (was at 64), and Check Orion and it is/was set at 30.  The article also said there would be errors about the query size being rejected in the vpxd log but there aren't any.

    I was hoping that was going to be it.  Plus the test server I am pointing at is only two hosts and ten guests.

  • Strange, can you check logs in c:\ProgramData\SolarWinds\Logs\VIM\Jobs for following: System.Web.Services.Protocols.SoapException: This operation is restricted by the administrator - 'vpxd.stats.maxQueryMetrics'.? In case you can see such errors there, it should be caused by this query metrics limitation. If that is the case, try to decrease the chunk size. The chunk size is not equivalent to the amount of metrics used, so even if chunk size is 30 it can still try to use more than 128 query metrics (so I'd recommend to start with small number an then you can try increase it step by step to see where is the threshold for your environment).

    If you don't see the issue there, please open a case with support.

  • there are no "This operation is restricted by the administrator", but there are:

    System.Web.Services.Protocols.SoapException: A specified parameter was not correct.

    System.Web.Services.Protocols.SoapException: A general system error occurred: Operation was canceled

    System.Web.Services.Protocols.SoapException: A general system error occurred: An existing connection was forcibly closed by the remote host

  • I changed the VMware values to unlimited, but it still is not working.

    I found a another error message as I was looking around:

    "Error while connecting to VMware device - The request failed with HTTP status 503: Service Unavailable (Failed to connect to endpoint: [class Vmacore::Http::LocalServiceSpec:00000000006816D0] _serverNamespace = /sdk _isRedirect = false _port = 8085)."

    One web search turned up "make sure vpxd is listening" and using netstat I found:

    Proto  Local Address          Foreign Address        State

    TCP    0.0.0.0:80             0.0.0.0:0              LISTENING       2180

    TCP    0.0.0.0:443            0.0.0.0:0              LISTENING       2180

    TCP    127.0.0.1:8085         0.0.0.0:0              LISTENING       5364

    Image Name                     PID Session Name        Session#    Mem Usage

    rhttpproxy.exe                2180 Services                   0     17,244 K

    vpxd.exe                      5364 Services                   0    953,052 K

    I can manually go to https://x.x.x.x/mob and move around.

    Any other ideas?

  • Thank you support!!!

    So it wasn't the upgrade, but it was that a proxy got configured (I'm not sure how):

    Disable Default Proxy Server for Orion Virtualization Monitor Connections to VMware vCenter

    To check to see if a proxy is configured:

    Netsh Winhttp show proxy

    I probably could have just removed the proxy too:

    Netsh Winhttp reset proxy

    We also had some odd remnants hanging around in some of the VIM_ tables we had to remove.

    Thank you Michael!