Honestly, I'm still not sure why this is a feature request rather than a bug fix but, that is all that Support is offering. We all understand the implications of noisy alerts and there are a few articles, videos and forum post about ways to mitigate those concerns. CPU being one of the worst offenders, tying a CPU utilization metric to another metric like Processor Queue Length is fundamental and the alert manager allows you to do this in the alert trigger condition itself. That is, in a normal trigger condition you can choose the object type Component and pick the trigger for Node CPU and Component Processor Queue Length. Likewise, you can pick them in the action. Using the OOB alert definition "High CPU Utilization with Top 10 Processes" since I want the TOP 10 Processes in the alert action Email, I constructed this alert thusly:
...
With the email from picked variables
${N=Alerting;M=AlertName} on ${N=SwisEntity;M=Application.Node.Caption}
CPU Consumption: ${N=SwisEntity;M=Application.Node.CPULoad}
Processor Queue Length: ${N=SwisEntity;M=ComponentAlert.StatisticData}
The top 10 processes running at the time of this poll are listed below:
1: ${N=Alerting;M=Notes}
2: ${N=Alerting;M=Notes}
3: ${N=SwisEntity;M=ComponentAlert.UserNotes}
TEST SQL CALL for processes
${SQL: SELECT Notes from alertStatusView where alertDefID = '${N=Alerting;M=AlertDefID}'}
the result being:
TEST NORMAL - SVR: HIGH Server CPU Utilization on XXXXXXXXXX
CPU Consumption: 6 %
Processor Queue Length: 1
The top 10 processes running at the time of this poll are listed below:
1:
2:
3:
TEST SQL CALL for processes
So I can get the Node.CPU and Component.Processor Queue Length but, I cannot get the result from:
SolarWinds.APM.RealTimeProcessPoller.exe -n=${NodeID} -alert=${N=Alerting;M=AlertDefID} -alertId=${N=Alerting;M=AlertID}
which, I know works from the OOB alert..and yes, the ExecuteExternalProgram has been approved in the Db Manager for every flavor I have attempted.
If I could, this problem would be solved.
The problem with the OOB Alert is that since its object is Node, a SAM component is not available as a triggering condition but, it does return the processes:
The CPU on ${Caption} is currently running at ${CPULoad}. The top 10 processes running at the time of this poll are listed below:
${N=Alerting;M=Notes}
For more information click the link below.
${NodeDetailsURL}
TEST SQL CALL for processes
${SQL: SELECT Notes from alertStatusView where alertDefID = '${N=Alerting;M=AlertDefID}'}
Returns
The CPU on XXXXXXXX is currently running at 5 %. The top 10 processes running at the time of this poll are listed below:
Name Process ID CPU
WmiPrvSE.exe 4508 5.4 %
SolarWinds.InformationService.ServiceV3.exe 5316 2.83 %
lsass.exe 908 2.36 %
SolarWinds.BusinessLayerHost.exe 12036 1.14 %
erl.exe 2820 0.84 %
SolarWinds.BusinessLayerHost.exe 12772 0.56 %
SolarWinds.BusinessLayerHost.exe 10636 0.28 %
MsMpEng.exe 2628 0.23 %
svchost.exe 148 0.21 %
System 4 0.15 %
For more information click the link below.
https://XXXXXXXX:443/Orion/View.aspx?NetObject=N:5002
TEST SQL CALL for processes
Name Process ID CPU
WmiPrvSE.exe 4508 5.4 %
SolarWinds.InformationService.ServiceV3.exe 5316 2.83 %
lsass.exe 908 2.36 %
SolarWinds.BusinessLayerHost.exe 12036 1.14 %
erl.exe 2820 0.84 %
SolarWinds.BusinessLayerHost.exe 12772 0.56 %
SolarWinds.BusinessLayerHost.exe 10636 0.28 %
MsMpEng.exe 2628 0.23 %
svchost.exe 148 0.21 %
System 4 0.15 %
This is effectively useless since a CPU metric alone is a very noisy alert as discussed previously.
Now comes the Complex Alert Condition. This allows me access to all the variables in the condition picker and the email alert action but does not populate the content of the varibles
...
...
Email Action
${N=Alerting;M=AlertName} on ${N=SwisEntity;M=SysName}
CPU Consumption: ${N=SwisEntity;M=CPULoad;F=Percent}
Processor Queue Length: ${N=SwisEntity;M=ComponentAlert.StatisticData}
The top 10 processes running at the time of this poll are listed below:
1: ${N=Alerting;M=Notes}${N=Alerting;M=Notes}
Poller: ${N=SwisEntity;M=Engine.DisplayName}
Node ID: ${NodeID}
AlertID: ${N=Alerting;M=AlertID}
AlerttObjID: ${N=Alerting;M=AlertObjectID}
AlertActiveID: ${N=Alerting;M=AlertActiveID}
SolarWinds.APM.RealTimeProcessPoller.exe -n=${NodeID} -alert=${N=Alerting;M=AlertDefID} -alertId=${N=Alerting;M=AlertID}
Result:
TEST COMPLEX - SVR: HIGH Server CPU Utilization on XXXXXXX
CPU Consumption: 6 %
Processor Queue Length: ${N=SwisEntity;M=ComponentAlert.StatisticData}
The top 10 processes running at the time of this poll are listed below:
1: Name Process ID CPU
WmiPrvSE.exe 4508 5.32 %
SolarWinds.Recommendations.Service.exe 2324 5.16 %
erl.exe 2820 1.07 %
SolarWinds.InformationService.ServiceV3.exe 5316 0.45 %
lsass.exe 908 0.26 %
svchost.exe 148 0.23 %
MsMpEng.exe 2628 0.23 %
SolarWinds.Orion.LogMgmt.PollingService.exe 2416 0.2 %
SolarWinds.Orion.LogMgmt.SyslogService.exe 2384 0.13 %
SolarWinds.BusinessLayerHost.exe 12772 0.13 %Name Process ID CPU
WmiPrvSE.exe 4508 5.32 %
SolarWinds.Recommendations.Service.exe 2324 5.16 %
erl.exe 2820 1.07 %
SolarWinds.InformationService.ServiceV3.exe 5316 0.45 %
lsass.exe 908 0.26 %
svchost.exe 148 0.23 %
MsMpEng.exe 2628 0.23 %
SolarWinds.Orion.LogMgmt.PollingService.exe 2416 0.2 %
SolarWinds.Orion.LogMgmt.SyslogService.exe 2384 0.13 %
SolarWinds.BusinessLayerHost.exe 12772 0.13 %
Poller: XXXXXXXXX
Node ID: 5002
AlertID: 610
AlerttObjID: 199638
AlertActiveID: 3391448
SolarWinds.APM.RealTimeProcessPoller.exe -n=5002 -alert=8869c7b9-5237-4786-8c0f-c8ea62641e32 -alertId=610
Cleary, I get the notes but now I don't get any values from the component.
The ASK: If I can pick it, I should be able to use it in an alert action whether it's simple condition where the stack is Component --> Node or complex where the Primary object is Node and the Secondary is anything.
And, yes I have flipped the Primary and secondary conditions which yeilds not Top 10 CPU processes neither ${N=Alerting;M=Notes} nor ${SQL: SELECT Notes from alertStatusView where alertDefID = '${N=Alerting;M=AlertDefID}'}
