Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Processor Load Data Does Not Match System Provided Proc Use Info

This morning a member of our team was looking at processor load over time on a windows 2008 R2 host and noted that what he saw in SolarWinds NPM did not match what he was seeing when he looked at the system's load through Task Manager and later Perfmon on the system The data in NPM was consistently higher. He asked me about this and I did a bit of digging and found a disturbing trend in the source data within SolarWinds.

In short - NPM appears to be recording the same value for min and max CPU (and hence average CPU) in the CPUMultiLoad_Detail, CPUMultiLoad, and CPULoad tables (my guess would be that the latter is derived from the former). I am guessing that this is the result of one of the following:

Stupid-Human-Trick 1 - I'm looking at the wrong tables in the database (but the data I see in the table aligns with what I am seeing in the graphs which are out of line with the graphs that we are seeing in the other sources - so I doubt this is the case)
Stupid-Human-Trick 2 - I made a mistake on installation which resulted in this averaging before population (possible - although I was pretty careful during the portion of the configuration in which actual decisions were being made).
Fixed Bug - There is a known issue in the software - apply the update... (note - I did do a bit of digging and did not find this listed
Active Bug - There is a known bug that is not resolved
Other?

My money is on the second or third option - but searching for a few did not provide any information on this symptom. Data and system version information below...

Thanks in advance...

DATA / SYSTEM INFORMATION

For example - the data for the node in question for the last couple of windows from each table follows:

FROM CPUMultiLoad

NodeID	TimeStampUTC	CPUIndex	MinLoad	MaxLoad	AvgLoad	Weight
326	2015-01-12 22:13:54.363	0	12	12	12	600
326	2015-01-12 22:13:54.363	1	12	12	12	600
326	2015-01-12 22:13:54.363	2	0	0	0	600
326	2015-01-12 22:13:54.363	3	6	6	6	600
326	2015-01-12 22:13:54.363	4	0	0	0	600
326	2015-01-12 22:13:54.363	5	18	18	18	600
326	2015-01-12 22:13:54.363	6	6	6	6	600
326	2015-01-12 22:13:54.363	7	18	18	18	600
326	2015-01-12 22:03:54.363	0	6	6	6	600
326	2015-01-12 22:03:54.363	1	93	93	93	600
326	2015-01-12 22:03:54.363	2	93	93	93	600
326	2015-01-12 22:03:54.363	3	31	31	31	600
326	2015-01-12 22:03:54.363	4	6	6	6	600
326	2015-01-12 22:03:54.363	5	37	37	37	600
326	2015-01-12 22:03:54.363	6	93	93	93	600
326	2015-01-12 22:03:54.363	7	100	100	100	600

FROM CPUMultiLoad Detail

NodeID	TimeStampUTC	CPUIndex	MinLoad	MaxLoad	AvgLoad
326	2015-01-12 23:03:54.397	0	0	0	0
326	2015-01-12 23:03:54.397	1	37	37	37
326	2015-01-12 23:03:54.397	2	93	93	93
326	2015-01-12 23:03:54.397	3	93	93	93
326	2015-01-12 23:03:54.397	4	12	12	12
326	2015-01-12 23:03:54.397	5	93	93	93
326	2015-01-12 23:03:54.397	6	93	93	93
326	2015-01-12 23:03:54.397	7	49	49	49
326	2015-01-12 22:53:54.397	0	0	0	0
326	2015-01-12 22:53:54.397	1	0	0	0
326	2015-01-12 22:53:54.397	2	47	47	47
326	2015-01-12 22:53:54.397	3	64	64	64
326	2015-01-12 22:53:54.397	4	0	0	0
326	2015-01-12 22:53:54.397	5	88	88	88
326	2015-01-12 22:53:54.397	6	58	58	58
326	2015-01-12 22:53:54.397	7	41	41	41

FROM CPULoad

NodeID	DateTime	MinLoad	MaxLoad	AvgLoad	TotalMemory	MinMemoryUsed	MaxMemoryUsed	AvgMemoryUsed	AvgPercentMemoryUsed	Archive	Weight
326	2015-01-12 15:13:54.363	9	9	9	6.012908E+10	3.840328E+09	3.840328E+09	3.840328E+09	6.386806	0	600
326	2015-01-12 15:03:54.363	57	57	57	6.012908E+10	3.835929E+09	3.835929E+09	3.835929E+09	6.37949	0	600

SO - The values in the MinLoad, MaxLoad, and AvgLoad fields in the CPULoad table are each the same, and each the rounded average of the full set of values for all processors in the node for the same window in CPUMultiLoad. I have to assume that there is an issue with my configuration that is causing this behavior (or at least I HOPE that this is not the intended behavior as it certainly is not "good data" regarding min/max/avg CPU load since the last poll.

Am I looking at the wrong data in the database? What causes this?

Environment Details:

FROM SETTINGS: Orion Platform Details (redacted a couple of items not germane to the question - like the hostname)

Orion Platform Details
Server
Web Server
Operating System	Windows Server 2008 R2
OS Version	6.1.7601.65536
Service Pack	Service Pack 1
Network Elements
Total Count
Nodes
Interfaces
Volumes
Version Info
AccountLimitationBuilder.exe	2014.01
AlertingEngine.exe	2014.1.0.0
AlertManager.exe	2014.01
AntiXSSLibrary.dll	4.2.0.0
Antlr3.Runtime.dll	3.1.0.39271
Antlr3.Utility.dll	0.1.0.39272
BasicAlertsManager.exe	2014.01
ConfigurationWizard.exe	2014.1.4.1466
Database-Maint.exe	2014.1.4.1466
DatabaseManager.exe	2014.1.4.1466
EO.Pdf.dll	5.0.34.2
ExportToPDFCmd.Exe	2014.1.4.1466
HtmlSanitizationLibrary.dll	4.2.0.0
ICSharpCode.SharpZipLib.dll	0.85.5.452
Infragistics2.Shared.v7.3.dll	7.3.20073.38
InstallHelper.RunOnce.exe	2014.1.4.1466
Interop.cdosys.dll	1.0.0.0
Interop.COMAdmin.dll	1.0.0.0
Interop.NetFWTypeLib.dll	1.0.0.0
Interop.olelib.dll	1.61.0.0
Interop.OrionSWScheduler8.dll	1.0.0.0
Interop.SHDocVw.dll	1.1.0.0
Interop.TaskScheduler.dll	1.0.0.0
Kiwi_LogViewer_2.1.0_Win32.setup.exe	Unknown
LicenseManager.exe	3.0.0.528
log4net.dll	1.2.10.0
LogAdjuster.exe	2014.1.4.1466
makecert.exe	5.131.3790.0 (srv03_rtm.030324-2048)
Microsoft.Web.Services3.dll	3.0.5305.0
MomConnectorFrameworkV2.dll	5.0.2749.0
NetPerfMonEventLog.dll	6.0.0.0
nsoftware.IPWorksSSNMP.dll	9.0.5077.0
OptimizeWebsite.exe	2014.1.4.1466
OrionAlertEngine.exe	2014.01
OrionNPMMapEngine.dll	2014.01
OrionNPMWeb.dll	2014.01
OrionPermissionChecker.exe	2014.1.4.1466
OrionReportWriter.exe	2014.01
OrionSWDiscoveryServices9.dll	2014.01
OrionSWLogger.dll	2014.01
OrionSWScheduler8.dll	2014.01
OrionWeb.dll	2014.1.4.1466
OrionWebBackup.exe	2014.01
phantomjs.exe	1.9.2
PollerCheckerTool.exe	2014.1.4.1466
PrintLauncher.exe	2014.1.4.1466
RegAsm.exe	4.0.30319.1 (RTMRel.030319-0100)
SciLexer.dll	3.0.4
SciLexer64.dll	3.0.4
ScintillaNet.dll	2.5.0.0
SmartThreadPool.dll	2.0.0.0
SnmpWalk.exe	2011.2.0.0
SolarWinds.BusinessLayerHost.Contract.dll	2014.1.4.1466
SolarWinds.BusinessLayerHost.exe	2014.1.4.1466
SolarWinds.Cirrus.IS.Client.dll	7.3.3.897
SolarWinds.Collector.Contract.dll	2.9.0.72
SolarWinds.Common.dll	2014.1.4.1466
Solarwinds.Compression.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.Common.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.IIS7.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.IIS7.Installer.exe	2014.1.4.1466
SolarWinds.ConfigurationWizard.Permissions.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.Plugin.APM.dll	6.1.1.3047
SolarWinds.ConfigurationWizard.Plugin.Common.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.Plugin.IpSla.dll	4.1.0.1568
SolarWinds.ConfigurationWizard.Plugin.NPM.dll	10.7.4.810
SolarWinds.ConfigurationWizard.Plugin.OIP.dll	2.1.113.0
SolarWinds.ConfigurationWizard.Plugin.Orion.dll	2014.1.4.1466
SolarWinds.ConfigurationWizard.Plugin.VIM.dll	1.9.0.69
SolarWinds.DatabaseManager.Strings.dll	2014.1.4.1466
SolarWinds.DataGap.AnalysisTool.exe	2014.1.4.1466
SolarWinds.Diagnostics.DBResponse.exe	2014.1.4.1466
SolarWinds.FipsManager.exe	2014.1.4.1466
SolarWinds.InformationService.Contract2.dll	2014.1.0.45
SolarWinds.Internationalization.dll	2014.1.4.1466
SolarWinds.JobEngine.Contract.dll	1.6.0.28
SolarWinds.JobEngine.Contract2.dll	2.7.0.30
SolarWinds.Licensing.Analysis.dll	3.0.0.528
SolarWinds.Licensing.Diagnostics.exe	3.0.0.528
SolarWinds.Licensing.Framework.dll	3.0.0.528
SolarWinds.Logging.dll	2014.1.4.1466
SolarWinds.Management.dll	2014.1.4.1466
SolarWinds.NCM.AlertAction.dll	7.3.3.897
SolarWinds.NCM.BusinessLayerPlugin.dll	1.0.1.0
SolarWinds.NCM.Contracts.dll	7.3.3.897
SolarWinds.NCM.RTNForwarder.exe	7.3.3.897
SolarWinds.Net.ICMP.dll	2014.1.4.1466
SolarWinds.Net.SNMP.dll	2014.1.4.1466
SolarWinds.Net.WMI.dll	2014.1.4.1466
SolarWinds.NTM.Common.dll	1.0.0.1
SolarWinds.NTM.Data.dll	1.0.0.1
SolarWinds.NTM.Discovery.dll	1.0.0.1
SolarWinds.NTM.Topology.dll	1.0.0.1
SolarWinds.Orion.Collector.dll	2014.1.4.1466
SolarWinds.Orion.Collector.MessagePlugin.dll	2014.1.4.1466
SolarWinds.Orion.Collector.Plugins.dll	2014.1.4.1466
SolarWinds.Orion.Common.dll	2014.1.4.1466
SolarWinds.Orion.CommonHelper.Interop.dll	1.0.0.0
SolarWinds.Orion.Core.Actions.dll	2014.1.4.1466
SolarWinds.Orion.Core.Alerting.dll	2014.1.4.1466
SolarWinds.Orion.Core.Auditing.dll	2014.1.4.1466
SolarWinds.Orion.Core.BusinessLayer.dll	2014.1.4.1466
SolarWinds.Orion.Core.Collector.dll	2014.1.4.1466
SolarWinds.Orion.Core.Collector.MessageSender.dll	2014.1.4.1466
SolarWinds.Orion.Core.Common.dll	2014.1.4.1466
Solarwinds.Orion.Core.Data.dll	2014.1.4.1466
Solarwinds.Orion.Core.Dependencies.dll	2014.1.4.1466
SolarWinds.Orion.Core.Discovery.dll	2014.1.4.1466
SolarWinds.Orion.Core.Jobs.dll	2014.1.4.1466
SolarWinds.Orion.Core.Jobs2.dll	2014.1.4.1466
SolarWinds.Orion.Core.MessageActions.dll	2014.1.4.1466
SolarWinds.Orion.Core.Models.dll	2014.1.4.1466
SolarWinds.Orion.Core.Pollers.dll	2014.1.4.1466
SolarWinds.Orion.Core.Reporting.dll	2014.1.4.1466
SolarWinds.Orion.Core.SharedCredentials.dll	2014.1.4.1466
SolarWinds.Orion.Core.Strings.dll	2014.1
SolarWinds.Orion.Core.SwisViaCOM.dll	1.0.0.0
SolarWinds.Orion.Core.Topology.dll	2014.1.4.1466
SolarWinds.Orion.Discovery.Contract.dll	2014.1.4.1466
SolarWinds.Orion.Discovery.Framework.dll	2014.1
SolarWinds.Orion.Discovery.Job.dll	2014.1
SolarWinds.Orion.FeatureManager.Interop.dll	1.0.0.0
SolarWinds.Orion.I18n.Interop.dll	1.0.0.0
SolarWinds.Orion.MacroProcessor.dll	2014.1.0.0
SolarWinds.Orion.NCM.Security.dll	7.3.3.897
SolarWinds.Orion.Packages.dll	1.0.0.55
SolarWinds.Orion.Pollers.Framework.dll	2014.1.4.1466
SolarWinds.Orion.Security.dll	2014.1.4.1466
SolarWinds.Orion.Topology.Calculator.exe	2014.1.4.1466
SolarWinds.Orion.Topology.Framework.dll	2014.1.4.1466
SolarWinds.OrionScheduler2.dll	2014.1.4.1466
SolarWinds.Packaging.Contract.dll	1.0.0.55
SolarWinds.Packaging.Core.dll	1.0.0.55
SolarWinds.PollerChecker.Contract.dll	2014.1.4.1466
SolarWinds.PollerChecker.dll	2014.1.4.1466
SolarWinds.PollerChecker.Plugin.Core.dll	2014.1.4.1466
SolarWinds.PollerChecker.Plugin.PollerChecker.dll	2014.1.4.1466
SolarWinds.Reporting.dll	2014.1.4.1466
SolarWinds.Security.Cryptography.X509Certificates.dll	Unknown
SolarWinds.Syslog.dll	2013.2.0.1
SolarWinds.Syslog.Strings.dll	2013.2.0.1
SolarWinds.WebLauncher.exe	2014.1.4.1466
SolarwindsDiagnostics.exe	2014.1.4.1466
StopNetPerfMon.exe	2014.1.4.1466
SWAlertActions8.dll	2014.01
SWAlertEngine8.dll	2014.01
SWSubClass.dll	2014.01
SWSysLogActions.dll	2014.01
SWTrapActions.dll	2014.01
SWTrapService.exe	2014.1.4.1466
SyslogService.exe	2014.1.4.1466
SyslogViewer.exe	2014.01
taskschd.dll	6.0.6001.18000 (longhorn_rtm.080118-1840)
tdbgpp8.dll	8.0.20034.14
TDBWSnk6.dll	5.00.0020
TestAlerts.exe	2014.01
todgub8.dll	8.0.20034.18
TrapViewer.exe	2014.01
Vim25Service.dll	0.0.0.0
Vim25Service.XmlSerializers.dll	0.0.0.0
VimService.dll	0.0.0.0
VimService.XmlSerializers.dll	0.0.0.0