Retrieve Web Data using CURL

Working a requirement to retrieve data using a URI via http script monitor.

In this case license expiration data for a program called Nexus which I believe is a developer support tool.

In my shop we do a lot of URI monitoring, mostly using Powershell script monitors that execute the Invoke-WebRequest command.

In this case the engineer I'm working with provided a CURL command for access as listed below (note that username/password, ANTI-CSRF-TOKEN, and specific URI are samples, not actual):

curl -u admin:admin123 -X 'GET' \
'https:/.../license' \
-H 'accept: application/json' \
-H 'NX-ANTI-CSRF-TOKEN: 0.123456789123456789' \
-H 'X-Nexus-UI: true'

I have a couple of questions about this:

in my world plain text passwords are a big no-no. I want to keep my username/password in the Solarwinds SAM credential store. If so, how do I provide this credential to the CURL command?
Or... can I just use the CURL command in a regular PowerShell script monitor, where I know how to retrieve and apply credentials from Solarwind?
Or... can I / should I just forget about CURL and translate the command into Invoke-WebRequest (I'm pretty uncertain about how to render the '-H 'accept: application/json' \
-H 'NX-ANTI-CSRF-TOKEN: 0.123456789123456789' \
-H 'X-Nexus-UI: true' ' part).

Thanks in advance for any thoughts / advice from the community.

Find more posts tagged with

webpage_monitor

sam

http monitor

Accepted answers

adam.beedell

Aah shoulda said you were doing it for doing cert monitoring, loads of those around

Much of the time, though probably not for this, i'd default to invoke-restmethod over invoke-webrequest, the former's better when receiving structured data and the latter's better when you need a cookie or something more web-ey.

There is a block of code you can add to dodge bad certificate options in 5.1/invoke-webrequest which avoids the need for the ps7 invoke-webrequest switch.

I think it's possible to spawn Ps7 from SAM because I had a script once for forcing 64 bit from 32 bit before I found the switch, but there was another thread with a guy on here who tested what I reccomended on that topic and didnt have success, ps7 support seems to be coming though, probably best to wait for that.

That said most of the PS7 features can be constructed in 5.1, they're just more easily available in 7 (with the exception of linuxey stuff mostly)

All comments

yaquaholic

Hi Robert,

I'd translate the curl to PowerShell for staters (https://curlconverter.com/), then check the manual on how to utilise the stored credentials.

I hope it helps,

Rich

adam.beedell

Yaqua's link is better, but-

1) $credential = ${CREDENTIAL}, then you can extract the user/pass if you need to
2) You kinda can if you really want to, cURL's probably installed - @ UX guys, a cURL component would save a lot of time.
3) Yeah that's what I do, -h is headers and you do this:
$headers = @{
Authorization = $bearer
}
$result = Invoke-RestMethod -Method GET -Uri $URL -headers $headers

robert777

Thank you for all the replies!

I've posted my solution for the problem. The code has comments which detail things like why I could not use invoke-webrequest in this situation.

In this case the script pulls license expiration data from the Sonatype Nexus application. However, I believe the structures have value for any requirement where you need to pull information from a web API back into Solarwinds.

Note that I have obfuscated customer data to sample data in order to protect the innocent.

Anyone in the community feel free to question or comment. There may be more elegant solutions to this particular problem. Definitely share. Code below:

# {creds} is a Solarwinds variable that references the credential from the Solarwinds SAM store

# credential is selected in the drop down of the component monitor

$b = ${creds}

$c = Get-Credential -Credential $b

# section below uses powershell commands to extract the credential from the SAM credential store into text.

# full disclosue: I found this code on Thwack. I didn't write it myself.

# if someone can identify the applicable Thwack post I will give credit where it is due.

Function SecureStringToString($value)

{

[System.IntPtr] $bstr = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($value);

try

{

[System.Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr);

}

finally

{

[System.Runtime.InteropServices.Marshal]::FreeBSTR($bstr);

}

[string] $username = $c.Username

[string] $password = SecureStringToString $c.Password

$securepass = $password | ConvertTo-SecureString -AsPlainText -Force

# note below the Curl command requires username and password in plain text.

# because the above code decrypts the password from the credential store on the fly the plain text password only exists in

# system memory for an instant - not user-viewable.

$userpass = $username + ':' + $password

# Note if you use just 'curl' the system will call powershell invoke-webrequest which has 'curl' as an alias.

# in this case I need actual 'curl' because the target system does not have the same

# chain of trust as my Solarwinds system, so I need to suppress the revocation check.

# powershell has this switch but only in versions after 5. Windows server 2016/2019 has Powershell 5.1

# built-in. Per previous Thwack thread there is apparently no way to tell Solarwinds to use a different

# verison of powershell other than the built-in 5.1. Therefore we need to use curl.exe not powershell invoke-webrequest.

# note that URIs and switches are sample data not real data from my customer (Opsec).

$result = curl.exe -s -X GET https://nexus-sample-http-endpoint.local/service/rest/v1/system/license --ssl-no-revoke -H 'accept: application/json' -H 'NX-ANTI-CSRF-TOKEN: 0.123456789123456789'-H 'X-Nexus-UI: true' -u $userpass

# the below series of commands parses the JSON return value into a single statistic value for

# the number of remaining days on the license.

$result = [string]$result

$expiration = [regex]::Match($result, 'expirationDate" : "(.*?)"').Groups[1].value

$index = $expiration.IndexOf("T") # I only care about the date, not the time. Only want what's to the left of the 'T'

$expirationshortdate = $expiration.Substring(0,$index)

$CurrentDate = Get-Date

$ExpirationDate = Get-Date -Date $expirationshortdate

$DaysRemaining = (New-TimeSpan -Start $CurrentDate -End $ExpirationDate).Days

# these are the variables that return to Solarwinds.

$ReturnValueMessage = " time remaining on license. "

$ReturnValueStatistic = $DaysRemaining

# Return values to Solarwinds

write-host "Message.software_license_expiration: $ReturnValueMessage"

write-host "Statistic.software_license_expiration: $ReturnValueStatistic"

exit 0;

adam.beedell

There is a block of code you can add to dodge bad certificate options in 5.1/invoke-webrequest which avoids the need for the ps7 invoke-webrequest switch.

That said most of the PS7 features can be constructed in 5.1, they're just more easily available in 7 (with the exception of linuxey stuff mostly)