WMI Intermittent Access Denied

Question

I have recently setup SAM and am using application templates to collect various information via WMI. My servers will randomly go into an unknown state. On one of my templates I turned on debugging and I'm finding that I get entries like the following:

2012-06-05 22:27:54,794 [STP SmartThreadPool Thread #1] [C(null)] ERROR SolarWinds.APM.Probes.ProbeExecutors.WmiWindowsServiceBatchExecutor - ManagementException caught. MamagementStatus code: AccessDenied

System.Management.ManagementException: Access denied

at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)

at System.Management.ManagementObjectCollection.ManagementObjectEnumerator.MoveNext()

at SolarWinds.APM.Probes.ProbeExecutors.WmiWindowsServiceBatchExecutor.GetProcessInfoViaWmi(String ipAddress, UserNamePasswordCredential credential, IEnumerable`1 services)

at SolarWinds.APM.Probes.ProbeExecutors.WmiWindowsServiceBatchExecutor.ExecuteGroup(List`1 serviceProbeInfos)

But then the next poll (or the last one) will function just fine. Randomly choosing one server I see that it went through the cycle:

Failed

Success

Failed

Success

Failed

Success

Failed

Success

Failed

Success

etc

All of the troubleshooting data I've found has to do with configuring the firewall, permissions the account needs, etc. However it succeeds roughly 1/3 to 2/3rd of the time (different servers have different rates of success) so I'm not sure where to start troubleshooting.

aLTeReGo · Accepted Answer

Under some conditions I've seen limits hit with the number of WMI querys that can be executed simultaneously against a single host. The result looked very similar to the one you describe here. It's extremely unlikely, and I'm not sure anyone has ever reported an issue where this was found to be the cause. This is just something I've observed in the lab when doing crazy things with WMI. It is important to note that Microsoft has implemented some cap to the number of WMI queries that can be executed simultaneously, so if you have another monitoring solution or something that may be heavily utilizing WMI it may be colliding with SAM. Again, not likely, but I thought I'd throw that out there in case something sprang to mind.

Just for giggles have you tried changing the fetching method of the Windows Service component monitors in the template from WMI to RPC to see if that's more reliable in your environment?