VCSA and PSC CPU/Memory monitoring

[pboydit]

We have set up the nodes for our VCSA (vCenter Server Appliance) and PSC (vCenter Platform Services Controller) in SolarWinds for monitoring.  However, the average CPU and memory statistics for both show 100%.  If you look at the resource consumption in vSphere neither the CPU or memory statistics are anywhere close to that.  According to vSphere performance monitoring our VCSA CPU average is 19.3% and the memory is 11% and the PSC CPU average is 2.8% and the memory is 13.7%.

We have rebooted both guests, removed and added back the resource from the node, and unmanaged/remanaged each node.  In SolarWinds the monitors for both guests and their resources are showing 100%.

Has anyone experienced this before?  If so, what did you do to get it fixed?

Current environment

VCSA and PSC are VMware Linux OS

ESX hosts are at 6.0.0, build 4600944

Orion Platform 2016.1.5300, SAM 6.2.4

[crzyr3d]

Any update on this issue?  The ability to monitor without the use of MIB's?

[shack]

With regards to your PSC's are you just polling them via SNMP or passing creds for VMWare?

Do you know what level permission is needed to pass creds for VMWare or if it provides any additional information?

[stevenstadel]

Any update on accurate OIDs for Memory and CPU for vSphere 6.0 Appliances?

[cczech]

Here's the MIB article from VMware.

https://kb.vmware.com/s/article/1013445

and this article was updated around the same time per the previous post.

https://kb.vmware.com/s/article/2054359

[styrker801]

We're also experiencing the same problem, we have 3 VCSA and 3 PSC  (6.0u3, but had issues since 6.0) in the environment and they are all reporting the same high CPU utilization.

From my findings and I'm not sure if it's VMware or Solarwinds, but VMware's MIB shows the OID for CPU utilization is: 1.3.6.1.2.1.25.3.3.1.2 (VMware's published OIDs can be found in VMware's KB Article)

So if I run a SNMP walk against that OID, I get the following response:

HOST-RESOURCES-MIB

hrProcessorLoad

column

1.3.6.1.2.1.25.3.3.1.2

.1.3.6.1.2.1.25.3.3.1.2.1 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.2 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.3 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.4 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.5 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.6 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.7 = INTEGER: 100

.1.3.6.1.2.1.25.3.3.1.2.8 = INTEGER: 100

Since this particular VCSA has 8 vCPU, it returns 8 values, all at 100.

If I use the Solarwinds poller checker tool, I can validate that it is using the OID value specified by VMware, see attached screenshot.

So I don't know if Solarwinds is getting the wrong value from VMware or VMware is reporting the incorrect value to Solarwinds; but for now we have just disabled CPU/Memory utilization monitor on these devices.

[kevinsolarwindsvman]

You nailed it.

I've seen quite a few of these cases come through support, where the software is reporting VCSA machines as having 100% CPU and Memory. Through poller checker and snmpwalk these devices are reporting to us this information even though the device isn't actually using that much of it's resources.

Currently the only potential solution I've been able to suggest to other customers is that if VMware has another OID that we can use or if you can find an OID your device  will support that returns more correct values, we can use those to create a new poller using Manage Pollers on the site. This would replace our default poller and hopefully start to make it look more correct. Unfortunately I haven't had any customers get back to me with any success at this,  so we are left at a bit of stand still.

[novell]

Very new and novice to this to pardon my ignorance. What is OID (object ID?) and where do I get the poller checker and snmpwalk tools? And also if anyone found a resolution to this issue.

Thanks,

[farida.kanchwala]

Hi Novell,

Yes OID is Object ID. You can find the poller checker within your SolarWinds installation files. Check this out https://support.solarwinds.com/success_center/Network_performance_Monitor_(NPM)/Use_the_Poller_Check..._tool

https://support.solarwinds.com/success_center/Network_perfomance_Monitor_(NPM)/SolarWinds_SNMP_Walk_A_New_Tool_For_Collecting_SNMP_MIB_Walks

[farida.kanchwala]

Hi,

Has anyone had any latest updates on this. We have recently upgraded to NPM 12.0.1 and SAM 6.3. We are having similar issues with all 8 CPU reporting 100% on VCSA 6.0. I run SNMP walk against the device OID and I get  "No more variables left in this MIB view (It is past the end of the MIB tree)!

[styrker801]

I installed a vCenter 6.5 in a test lab, and the VCSA or the PSC no longer shows 100%.  My belief is it's an issue/bug with vCenter 6.0.

[damien.solodow]

What VCSA versions?

We have VCSA 6.5.0b, NPM 12.1.0 and SAM 6.2.4 and it's reporting correct CPU and memory load.

[crzyr3d]

We have this version VMware-vCe​nter-Serve​r-Applianc​e 6.5.0.2000​0 embedded build 8307201 VMware, Inc x86_64.  I'm still getting the way off stat.  Are you monitroing by snmp v2 or v3?

[skipdog77]

We face the same issue.  All four vcenter 6 appliances show 100% for memory and CPU load.  Sometimes we can remove the monitoring and re-add it and for 1-2 poll cycles it shows the correct amount.  After the second or third poll cycle it returns to 100%.

[stevenwhunt]

Have you opened a support case for this issue?

[pboydit]

We have opened a support case.

Thanks

[solarwinds_noob]

Same here, having the same issue.  Were you able to get a resolution?

[crzyr3d]

Were you able to get this resolved?

[abhough88]

Were you able to get this resolved?  We are having the same issue.

Thanks

Adam

[novell]

we ran the following command on the vCenter Server and Orion reported the actual utilization:

sync; echo 3>/proc/sys/vm/drop_caches