This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Acknowledge Solarwinds alert by email

Updated 2020-07-05 to support oAuth and Azure AD. New script is here.

Updated 2019-02-26 to support recent changes to the Solarwinds API.

Updated 2019-07-02 to support passing credentials as encrypted credentials files or as clear text username and password on the command line. To keep Task Scheduler configuration simple, I've included instructions for setting the credentials within the script.

This Powershell script allows end users to acknowledge or comment upon a Solarwinds alert via email. The script assumes that your organization uses Exchange as its email platform.

Prerequisites:

  • An email account ("the email account") for the default reply-to address used by your Solarwinds installation. This should be a dedicated account that isn't used by any other applications or users.
  • A Solarwinds individual account ("the Solarwinds account") that can acknowledge alerts. This account must have the following permissions in Solarwinds:
    • Allow alert management rights = yes
    • Allow account to disable actions = yes
    • Allow account to disable alerts = yes
    • Allow account to disable all actions = yes
    • Allow Account to Clear Events, Acknowledge Alerts and Syslogs = yes
  • MS Exchange Web Services Managed API installed on the server that will run the script. Download the script from the MS Download Center.

Configuration:

  1. Log in to Windows as the account that will run the PowerShell script. Create a Windows credentials file for the email account and for the Solarwinds account. For each account, run the PowerShell command
    Get-Credential | Export-Clixml -Path [full path to file]
  2. In your Solarwinds alerts, configure the alert to send email. Make sure that the email body includes this text and variable:
    [AlertObjectID=${N=Alerting;M=AlertObjectID}]
  3. Edit the script as follows.
    1. Edit the line
      $exchangeEmail    = "solarwinds@mydomain.com"
      and replace "solarwinds@mydomain.com" with the address of the email account.

    2. Edit the line
      $SWServer = "solarwinds.mydomain.com"
      and replace "solarwinds.mydomain.com" with the fully-qualified domain name or IP address of your Solarwinds server

    3. If you are using a credentials file for the Exchange user, edit the line
      $ex_credstore = "c:\Users\solarwinds\Exchange_credentials.xml"
      and replace "c:\Users\solarwinds\Exchange_credentials.xml" with the full path to the Exchange credentials file you created.
    4. If you are using a clear text username and password for the Exchange user, edit the line
      $ex_credstore = "c:\Users\solarwinds\Exchange_credentials.xml"
      and replace "c:\Users\solarwinds\Exchange_credentials.xml" with "", then update the values for $ex_username and $ex_password on the next two lines.

    5. If you are using a credentials file for the Solarwinds user,
      $sw_credstore = "c:\Users\solarwinds\Solarwinds_credentials.xml"
      and replace "c:\Users\solarwinds\Solarwinds_credentials.xml" with the full path to the Solarwinds credentials file you created
    6. If you are using a clear text username and password for the Solarwinds user, edit the line
      $sw_credstore = "c:\Users\solarwinds\Solarwinds_credentials.xml"
      and replace "c:\Users\solarwinds\Solarwinds_credentials.xml" with "", then update the values for $sw_username and $sw_password on the next two lines.
  4. Copy the script to your Solarwinds server. You can run the script from the Powershell IDE to test it.
  5. Configure a task manager job to run the script at the desired interval. I run it once per minute to ensure that acknowledgement/comment emails are processed quickly enough to satisfy my end users.

Use:

This script assumes that the first word in the email message is a command verb such as ack or comment. Any text between the command and a return and/or newline character(s) is treated as a comment, and it will be appended to the alert in Solarwinds.

Message syntax:

At minimum, the email message must contain the Alert Object ID, formatted as described below. This element can appear anywhere in the message body.

Alert Object ID: A string of digits, in brackets, formatted like this:

    [AlertObjectID=99999]

For backward compatibility, the message may contain these elements instead of the Alert Object ID.

Alert definition ID: A GUID in brackets, formatted like this:</>

    [AlertDefinition=D1A5279D-B27D-4CD4-A05E-EFDD53D08E8D]

Object ID: A string of digits, in brackets, formatted like this:

    [ObjectID=99999]

Object Type: A string representing a valid Solarwinds object type, formatted like this:

    [ObjectType=APM: Application]

Commands:

The command must be the first word in the message body.

ack:    Acknowledge the alert and append a comment if one is supplied.

comment: Append a comment to the alert

Comments:

Comments are optional. Any text between the command verb and a newline or return character will be treated as a comment. There are no formatting requirements for comments.

Example 1:

This message would acknowledge an alert and append a comment

    Ack  Alert comment from ME

    [AlertObjectID=46673]

Example 2:

This message would also acknowledge an alert and append a comment.

    Ack Kurt is looking into this

    ~~~~~~~~~~~~~~~~~~~~~

    Me, Senior Systems Administrator

    ~~~~~~~~~~~~~~~~~~~~~~

    From: Solarwinds

    Sent: Friday, October 02, 2015 12:33 PM

    To: Me <me@myemail.com>

    Subject: Solarwinds Alert: Oracle database disk utilization KURTSDB on Node kurtsserver.mydomain.com is Down

    When replying, type your text above this line

    ________________________________________

    To acknowledge this alert, click here or reply to this message with "ack" as the first word in the body.

    Alert details:

    Oracle database disk utilization KURTSDB on Node kurtsserver.mydomain.com is Down at Friday, October 2, 2015 12:32 PM.

    When replying, do not delete text below

    ________________________________________

    [AlertObjectID=46673]

Example 3:

Deprecated format for backwards compatibility. This message would also acknowledge an alert and append a comment

    Ack  Third alert comment from ME

    [AlertDefinition=535d1493-a543-4df0-acbf-6b43770aceeb] [ObjectID=673] [ObjectType=APM: Application]

process_alert_reply_email_v2.ps1
  • Ah, very smart - rather than fighting the email format and grep'ing for the strings you need to do the work you force the creator of the email alert action to add some easily-identifiable markers. That's more than reasonable and saves you/us a lot of heartache.

  • This is brilliant and allows us to replace Alert Central which is absolutely horrible. Thank you for sharing.

  • Thanks! If you make any improvements or additions, please update in the comments here!

  • FormerMember
    0 FormerMember over 6 years ago

    How does this replace event alert aggregation that Alert Central does?

  • It doesn't address Alert Central's alert aggregation directly. If you've configured Solarwinds to work with Alert Central, alert aggregation should work exactly as if you acknowledged the alert through the Solarwinds GUI.

  • Interesting solution, thanks for sharing. I just wanted to chip-in and share basics - just for the record - some of you might find it useful as well:

    (1)

    In the alert body use this:

    <a href="https://SolarWindsServer-or-IP/Orion/NetPerfMon/AckAlert.aspx?AlertDefID=${SQL:SELECT AlertObjectID FROM AlertStatusView WHERE '${N=Alerting;M=AcknowledgeUrl}' LIKE '%='+CONVERT(NVARCHAR(MAX),AlertObjectID)}&viaEmail=True"><i>Acknowledge this alert and stop repetitive emails</i></a>

    (2)

    Result:

    pastedImage_5.png

    pastedImage_6.png

    P.S. The reason I am not using "${N=Alerting;M=AcknowledgeUrl}" but instead re-create my own URL is because unfortunately this URL variable consist of FULL URL, which includes server name, whereas very often I just need to access server by IP and over https by default. So, that's why I am re-building the whole URL the way it suits me

    That's it - pretty basic but hope will help you to look at it from another perspective as well

    Obviously "comments" are not in this package emoticons_happy.png

  • Very cool script, ran into a minor issue, when we ack the email it adds the entire email chain as the alert comments.

  • Nice. You should mention that you need to install the Exchange Web API as a Pre Req

    This requires the Exchange Web Services Managed API to be installed on the computer where this script is being ran

    # Download at - http://www.microsoft.com/en-us/download/confirmation.aspx?id=42022

  • This script will to run correctly and "processes" the email without error.  The Alert def object ID and Object type are being picked up by the script  However the alert is not being marked as "acknowledged" and remains unacknowledged.   I can not work out why.

    We are on Exchange 2013 and NPM version 12

  • Good catch. I've added that to the prerequisites.