Powershell Script Monitor "Not Defined"

Hi Team,

I am attempting to create a powershell script monitor component but am receiving the error “Not Defined” when testing the script against my target machine. The script in question is below, and basically just checks the filecount in a folder.

[int]$fileCount = ( Get-ChildItem D:\backups | Measure-Object ).Count;

if ($fileCount -lt 5) {

Write-Host "Statistics.status: 100"

Write-Host "Message.status: Down"

exit 2

}

else {

Write-Host "Statistics.status: 0"

Write-Host "Message.status: Up"

exit 0

}

As this was not working (output was "Not Defined") I simplified the script to below, but this still does not work.

write-host "Statistic.status: 100”

write-host "Message.status: Down”

I am using the "Windows Powershell Monitor" component with the following settings.

Execution Mode - Remote Host

Use HTTPS - No

URL Prefix - wsman

Port - 5895

Run the script under specified account - No

Script Output #1

Unique ID - status

DisplayName - status

Warning - Greater than 0

Down - Greater than 50

I should be getting a output of Down according to the above configuration.

I have tested WinRM and powershell is able to be executed remotely from the Orion server to the target machine and it works.

Orion Server

==========

PS C:\Windows\system32> Enable-PSRemoting -Force

WinRM is already set up to receive requests on this computer.

WinRM is already set up for remote management on this computer.

PS C:\Windows\system32> Test-WsMan **hostname**

wsmid : http://schemas.dmtf.org/wbem/wsman/identity/1/wsmanidentity.xsd

ProtocolVersion : http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd

ProductVendor   : Microsoft Corporation

ProductVersion  : OS: 0.0.0 SP: 0.0 Stack: 2.0

PS C:\Windows\system32> Invoke-Command -ComputerName **hostname** -ScriptBlock { echo $null >> D:\backups\touch.txt } -credential**user**

PS C:\Windows\system32> Invoke-Command -ComputerName **IP** -ScriptBlock { echo £null >> D:\backups\touch.txt } -credential **user**

When either of the two commands above are run(to the hostname or IP) I can see a file generated in D:\backups\touch.txt. From Wireshark on the target device I can see this traffic hitting TCP port 5985, the WinRM port.

However when I test the script through the component GUI targeting the device in question, I do not see any traffic hit the target device at all on port 5985 and I simply receive an output error of “Not Defined”. It appears as if the script isn’t even bring ran on the remote device. I have updated and confirmed the credentials being used through the SAM credential manager is correct (the same ones tested with the Invoke-Command above).

Any advice and suggestions would be welcome. Many thanks!

  • Where are you testing it from.

    It sounds like you are using a template that already has outputs defined?

  • I would suggest you copy a template that already has Powershell component.

    Delete all of the pre-defined  outputs (normally only 1 on a template)

    pastedImage_0.png

    Then Edit Script, Paste in your code assign credential (instead of the ones in the script - create in Credentials Library in SW)

    I think that may help

  • Hi Ian,

    Many thanks for the reply. I gave this a go, I copied an existing template using a powershell component and edited it but get the same results.

    Originally I was getting the problem from a brand new template so there should not have been any dormant configuration from copying/using an existing template.

    I am testing the script on the edit script page, I am selecting the device in question and and clicking on "get script output", which results in "not defined".

    I can't even see any traffic hitting the target device on port 5985 when I am testing this, which I find a bit unusual.

    cheers

  • What appears in the APM debug log ...... It may be that the SW account that is submitting this either does not have the set execution policy set?

  • Sadly Just tested on a new one in my install and got this: working ok?

    pastedImage_0.png

    So it must be elsewhere:  I turned on Debug logging and the working version did this output

    2018-02-05 14:58:19,666 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Logging.ApmJobContextInfo - Reseting thread log data

    2018-02-05 14:58:19,588 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test starting - ComponentId: -1; NodeId: 670; NodeName: XXXXXXXXXXXXXX; ApplicationId: 694; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.4.0.1020, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: d7df043b-0066-48b5-be81-35bf83c22e2c;

    2018-02-05 14:58:19,604 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.MonitorJob - Current process: SWJobEngineWorker2 (PID 32596)

    2018-02-05 14:58:19,604 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.MonitorJob - Using new credentials DOM\Acc

    2018-02-05 14:58:19,604 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.ProbeBase`1 - Remaining Time: 290.9843745 sec.

    2018-02-05 14:58:19,604 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Creating local runspace

    2018-02-05 14:58:19,666 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - PS Runspace opened [id:80c81797-5ba9-453a-85fe-d359c17395ba,uri:http://VVV.NNN.XXX.MMM:5985/wsman]

    2018-02-05 14:58:19,666 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Processing macros: 19 variables processed

    2018-02-05 14:58:19,666 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.PowerShellProbe -

    Begin PowerShell Execute Result: ================================

    Target: VVV.NNN.XXX.MMM

    Script:      ====================================================

    write-host "Statistic.status: 100”

    write-host "Message.status: Down”

    Output:      ====================================================

    Statistic.status: 100

    Message.status: Down

    Errors:      ====================================================

    Result:      ====================================================

    Component Evidence Type: DynamicEvidence

    Component Type: WindowsPowerShell

    Actual Outcome: Undefined

    Registered Errors:

    Dynamic Monitor Result Info:

    Outcome based on statistic thresholds: Undefined

    End PowerShell Execute Result ===================================

    2018-02-05 14:58:19,666 [STP SmartThreadPool Thread #3] [C-1] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test finished - ComponentId: -1; NodeId: 670; NodeName: XXXXXXXXXXXXXX; ApplicationId: 694; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.4.0.1020, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: d7df043b-0066-48b5-be81-35bf83c22e2c;

    ---------------------------------------------------------

    Is there any application log produced?

  • Hey Ian,

    After a bit of further digging through the debug log, I noticed that the WinRM requests were coming from a second poller, the device in question was not being polled by our primary Orion servers (which was the one I trusted in the WinRM configuration). silly me forgot to add/trust the IP of the second poller on the target device. Once I did this, it worked as expected.

    cheers for your advice, I hadn't thought about turning debugging on!

    thanks

  • Same probleme here.

    Using invoke-command from the Orion Server, the execution is complete on remote node.

    From Orion server to Boba node :

    PS C:\Windows\system32>  Invoke-Command -ComputerName boba -ScriptBlock { echo $null >> c:\solar\touch.txt } -credential

    dom_utac\admsolar

    => file is created on remote node

    PS C:\Windows\system32>  Test-WsMan boba

    wsmid           : http://schemas.dmtf.org/wbem/wsman/identity/1/wsmanidentity.xsd

    ProtocolVersion : http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd

    ProductVendor   : Microsoft Corporation

    ProductVersion  : OS: 0.0.0 SP: 0.0 Stack: 3.0

    the script is very simple in the monitor template :

    write-host "statistic.var1:99"

    write-host "message.var1:aa bbb ccc"

    But using the windows powershell monitor, I encounter the output result : Not defined

    What can I check ?

    here is the debug log http or https :

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Logging.ApmJobContextInfo - Reseting thread log data

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test starting - ComponentId: 14867; NodeId: 24; NodeName: BOBA; ApplicationId: 279; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.8.0.9502, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: 158fd6c1-b46e-4bd2-9603-5eb4c71fba1f;

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Current process: SWJobEngineWorker2x64 (PID 15852)

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Using new credentials dom_utac\admsolar

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.ProbeBase`1 - Remaining Time: 291 sec.

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Impersonating execution user:admsolar, domain:dom_utac agentExecutionMode:False

    2019-07-31 11:15:13,086 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.NetworkUser - Init called: userName = (admsolar), domainOrMachine = (dom_utac), logonType = (NewCredentials), LogonProvider.DefaultProvider = (DefaultProvider)

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Creating remote runspace to http://172.16.1.51:5985/wsman with schema http://schemas.microsoft.com/powershell/Microsoft.PowerShell

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] ERROR SolarWinds.APM.Probes.PowerShellProbe - System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server 172.16.1.51 failed with the following error message : The WinRM client cannot process the request. Default authentication may be used with an IP address under the following conditions: the transport is HTTPS or the destination is in the TrustedHosts list, and explicit credentials are provided. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. For more information on how to set TrustedHosts run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.AsyncResult.EndInvoke()

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at System.Management.Automation.RemoteRunspace.Open()

       at SolarWinds.APM.Probes.Script.PowerShellScriptHelper.PrepareRunSpace(PowerShellProbeSettings settings, JobExecutionMode executionMode, UserNamePasswordCredential cred, Boolean skipCertChecks)

       at SolarWinds.APM.Probes.Script.PowerShellScriptHelper.RunScript(PowerShellProbeSettings settings, ProbeInformation probeInformation, ScriptMacroParser macroParser)

       at SolarWinds.APM.Probes.PowerShellProbe.ProbeInternal(ProbeInformation probeInfo, DynamicMonitorResult result, Func`1 powershellScriptHelperFactory)

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.PowerShellProbe -

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test finished - ComponentId: 14867; NodeId: 24; NodeName: BOBA; ApplicationId: 279; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.8.0.9502, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: 158fd6c1-b46e-4bd2-9603-5eb4c71fba1f;

    2019-07-31 11:15:00,851 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Logging.ApmJobContextInfo - Reseting thread log data

    2019-07-31 11:15:00,789 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test starting - ComponentId: 14867; NodeId: 24; NodeName: BOBA; ApplicationId: 279; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.8.0.9502, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: d0a93f6f-cb6b-46dc-8fa3-4eb5284f9b7a;

    2019-07-31 11:15:00,789 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Current process: SWJobEngineWorker2x64 (PID 15852)

    2019-07-31 11:15:00,789 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Using new credentials dom_utac\admsolar

    2019-07-31 11:15:00,789 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.ProbeBase`1 - Remaining Time: 291 sec.

    2019-07-31 11:15:00,804 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Impersonating execution user:admsolar, domain:dom_utac agentExecutionMode:False

    2019-07-31 11:15:00,804 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.NetworkUser - Init called: userName = (admsolar), domainOrMachine = (dom_utac), logonType = (NewCredentials), LogonProvider.DefaultProvider = (DefaultProvider)

    2019-07-31 11:15:00,820 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.Script.PowerShellScriptHelper - Creating remote runspace to https://172.16.1.51:5985/wsman with schema http://schemas.microsoft.com/powershell/Microsoft.PowerShell

    2019-07-31 11:15:00,851 [STP SmartThreadPool Thread #0] [C14867] ERROR SolarWinds.APM.Probes.PowerShellProbe - System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server 172.16.1.51 failed with the following error message : The server certificate on the destination computer (172.16.1.51:5985) has the following errors:      

    Encountered an internal error in the SSL library. For more information, see the about_Remote_Troubleshooting Help topic.

       at System.Management.Automation.Runspaces.AsyncResult.EndInvoke()

       at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

       at System.Management.Automation.RemoteRunspace.Open()

       at SolarWinds.APM.Probes.Script.PowerShellScriptHelper.PrepareRunSpace(PowerShellProbeSettings settings, JobExecutionMode executionMode, UserNamePasswordCredential cred, Boolean skipCertChecks)

       at SolarWinds.APM.Probes.Script.PowerShellScriptHelper.RunScript(PowerShellProbeSettings settings, ProbeInformation probeInformation, ScriptMacroParser macroParser)

       at SolarWinds.APM.Probes.PowerShellProbe.ProbeInternal(ProbeInformation probeInfo, DynamicMonitorResult result, Func`1 powershellScriptHelperFactory)

    2019-07-31 11:15:00,851 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.PowerShellProbe -

    2019-07-31 11:15:00,851 [STP SmartThreadPool Thread #0] [C14867] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test finished - ComponentId: 14867; NodeId: 24; NodeName: BOBA; ApplicationId: 279; ComponentName: Windows PowerShell Monitor; ComponentType: SolarWinds.APM.Probes.PowerShellProbe, SolarWinds.APM.Probes, Version=6.8.0.9502, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: d0a93f6f-cb6b-46dc-8fa3-4eb5284f9b7a;

  • 2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] ERROR SolarWinds.APM.Probes.PowerShellProbe - System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server 172.16.1.51 failed with the following error message : The WinRM client cannot process the request. Default authentication may be used with an IP address under the following conditions: the transport is HTTPS or the destination is in the TrustedHosts list, and explicit credentials are provided. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. For more information on how to set TrustedHosts run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

    This tells me WinRM isn't configured to allow connections from the poller on "boba". You'll have to configure it before Invoke-Command will work.

  • thulsey  wrote:

    2019-07-31 11:15:13,117 [STP SmartThreadPool Thread #0] [C14867] ERROR SolarWinds.APM.Probes.PowerShellProbe - System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to remote server 172.16.1.51 failed with the following error message : The WinRM client cannot process the request. Default authentication may be used with an IP address under the following conditions: the transport is HTTPS or the destination is in the TrustedHosts list, and explicit credentials are provided. Use winrm.cmd to configure TrustedHosts. Note that computers in the TrustedHosts list might not be authenticated. For more information on how to set TrustedHosts run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting Help topic.

    This tells me WinRM isn't configured to allow connections from the poller on "boba". You'll have to configure it before Invoke-Command will work.

    alaingo​ In this document:  Troubleshoot PowerShell issues There's a section on enabling winrm and verifying that it is enabled Use PowerShell in SAM