Integration with PagerDuty

PagerDuty has developed and provided a very nice integration guide for SolarWinds.

However, in my experience, it goes a little too far by requiring extra custom properties and VB script execution via scheduled task in order to post the JSON payloads to the PagerDuty site. Below are a couple of examples you can use with the HTTP POST action from the web alert engine. Just post these messages as the payload to https://events.pagerduty.com/generic/2010-04-15/create_event.json

ALERT TRIGGER:

{
"_comment": "https://events.pagerduty.com/generic/2010-04-15/create_event.json",
"service_key": "ENTER CLIENT'S SERVICE KEY FROM PAGERDUTY HERE",
"incident_key": "THIS IS THE STRING FOR YOUR INCIDENT TITLE",
"event_type": "trigger",
"description": "THIS IS THE BODY OF YOUR MESSAGE DETAILS",
"client": "SolarWinds",
"client_url": "THIS IS WHERE YOU CAN PUT A DETAILS URL BACK TO SOLARWINDS",
}

ALERT RESET:

{
"_comment": "https://events.pagerduty.com/generic/2010-04-15/create_event.json",
"service_key": "ENTER CLIENT'S SERVICE KEY FROM PAGERDUTY HERE",
"incident_key": "INCIDENT TITLE MUST MATCH THE TRIGGER EXACTLY",
"event_type": "resolve",
"description": "THIS IS THE BODY OF YOUR MESSAGE DETAILS"
}

Enjoy!

Find more posts tagged with

Loop1

pagerduty

loop1 systems

Accepted answers

All comments

ajwill

how do you have a guide on how to implement this?

Thanks

jpr7311

You simply create a "Send a GET or POST request...." as a trigger condition for any of your alerts, and copy the url into the url field using HTTP Post, and then the rest of the text into the body. The only thing you should have to change is the SERVICE KEY, and you get the service key from your PagerDuty site. We had to remove any blank lines when copying and pasting the code and also, of course, leave out the _comment line(common sense, I know, but when it's the end of a business day, that goes out the window).

Thanks, Zach!

cyates

Does anyone know if there is a way to verify that the HTTP POST was successful, or if / how a retry could be setup?

zackm

go to requestb.in and setup a test page, then target that. it will enumerate your payload for you.

t3chno

I've set up an integration using PagerDuty's V2 API

Overview · PagerDuty Developer

HTTP Posts sent to: https://events.pagerduty.com/v2/enqueue

In my case, it was for an Application Down alerts. I have the Routing Key stored as a custom property against the Node, as we monitor multiple services for different clients. The DEDUPE key is used to avoid raising duplicate alerts in PagerDuty:

{

"payload": {

"summary": "${N=SwisEntity;M=Node.DisplayName} - ${N=Alerting;M=AlertName}",

"source": "Solarwinds",

"severity": "info",

"component": "${N=SwisEntity;M=ApplicationAlert.ApplicationName}",

"group": "${N=SwisEntity;M=Node.CustomProperties.Cherwell_Code}",

"class": "deploy",

"custom_details": {

"Description": "${N=SwisEntity;M=ApplicationAlert.ApplicationName} ${N=Alerting;M=AlertDescription}",

"Solarwinds-Severity": "${N=Alerting;M=Severity}",

"Trigger Count": "${N=Alerting;M=AlertTriggerCount}"

}

"routing_key": "${N=SwisEntity;M=Node.CustomProperties.PagerDuty_API_Key}",

"dedup_key": "${N=Alerting;M=AlertActiveID}",

"event_action": "trigger",

"client": "Solarwinds Monitoring Service",

"client_url": "${N=Alerting;M=AlertDetailsUrl}"

}

t3chno

So this week, I've started configuring Disk Space Alerts.

Headache . . .

Took some testing and debugging, but the API will not accept the Back Slash Character \ in the POST Payload.

Having a look now to see if there is a way of getting the drive letter from Solarwinds without the \

t3chno

Thanks to kjc who posted SELECT REPLACE(Caption, '\', '\\') FROM Volumes

the_ben_keen

So we are doing this as well. I downloaded the sample alerts and was able to import all of them except for the Application Down and "Application Warning/Critical".

The crazy thing is I was able to import them on my dev box which is running SAM 6.6 but not PROD which is running 6.7.

I thought I would try to copy and paste the text from the imported alert on my DEV side and it's not working. The text file is created but then just sits in C:\PagerDuty\Queue and doesn't move. I tried to reach out to SolarWinds support and the kid sent me the PagerDuty Integration document on PagerDuty's website that I read over several times.

Anyone else has this issue? If so, did you find a way to fix it?

ppaez

Is there away to retireve the http response from pagerduty? I want to be able to log an error somewhere if the pagerduty events v2 api wasn't able to ingest the http POST from solarwinds.