This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Acknowledge Solarwinds alert by email

Updated 2020-07-05 to support oAuth and Azure AD. New script is here.

Updated 2019-02-26 to support recent changes to the Solarwinds API.

Updated 2019-07-02 to support passing credentials as encrypted credentials files or as clear text username and password on the command line. To keep Task Scheduler configuration simple, I've included instructions for setting the credentials within the script.

This Powershell script allows end users to acknowledge or comment upon a Solarwinds alert via email. The script assumes that your organization uses Exchange as its email platform.

Prerequisites:

  • An email account ("the email account") for the default reply-to address used by your Solarwinds installation. This should be a dedicated account that isn't used by any other applications or users.
  • A Solarwinds individual account ("the Solarwinds account") that can acknowledge alerts. This account must have the following permissions in Solarwinds:
    • Allow alert management rights = yes
    • Allow account to disable actions = yes
    • Allow account to disable alerts = yes
    • Allow account to disable all actions = yes
    • Allow Account to Clear Events, Acknowledge Alerts and Syslogs = yes
  • MS Exchange Web Services Managed API installed on the server that will run the script. Download the script from the MS Download Center.

Configuration:

  1. Log in to Windows as the account that will run the PowerShell script. Create a Windows credentials file for the email account and for the Solarwinds account. For each account, run the PowerShell command
    Get-Credential | Export-Clixml -Path [full path to file]
  2. In your Solarwinds alerts, configure the alert to send email. Make sure that the email body includes this text and variable:
    [AlertObjectID=${N=Alerting;M=AlertObjectID}]
  3. Edit the script as follows.
    1. Edit the line
      $exchangeEmail    = "solarwinds@mydomain.com"
      and replace "solarwinds@mydomain.com" with the address of the email account.

    2. Edit the line
      $SWServer = "solarwinds.mydomain.com"
      and replace "solarwinds.mydomain.com" with the fully-qualified domain name or IP address of your Solarwinds server

    3. If you are using a credentials file for the Exchange user, edit the line
      $ex_credstore = "c:\Users\solarwinds\Exchange_credentials.xml"
      and replace "c:\Users\solarwinds\Exchange_credentials.xml" with the full path to the Exchange credentials file you created.
    4. If you are using a clear text username and password for the Exchange user, edit the line
      $ex_credstore = "c:\Users\solarwinds\Exchange_credentials.xml"
      and replace "c:\Users\solarwinds\Exchange_credentials.xml" with "", then update the values for $ex_username and $ex_password on the next two lines.

    5. If you are using a credentials file for the Solarwinds user,
      $sw_credstore = "c:\Users\solarwinds\Solarwinds_credentials.xml"
      and replace "c:\Users\solarwinds\Solarwinds_credentials.xml" with the full path to the Solarwinds credentials file you created
    6. If you are using a clear text username and password for the Solarwinds user, edit the line
      $sw_credstore = "c:\Users\solarwinds\Solarwinds_credentials.xml"
      and replace "c:\Users\solarwinds\Solarwinds_credentials.xml" with "", then update the values for $sw_username and $sw_password on the next two lines.
  4. Copy the script to your Solarwinds server. You can run the script from the Powershell IDE to test it.
  5. Configure a task manager job to run the script at the desired interval. I run it once per minute to ensure that acknowledgement/comment emails are processed quickly enough to satisfy my end users.

Use:

This script assumes that the first word in the email message is a command verb such as ack or comment. Any text between the command and a return and/or newline character(s) is treated as a comment, and it will be appended to the alert in Solarwinds.

Message syntax:

At minimum, the email message must contain the Alert Object ID, formatted as described below. This element can appear anywhere in the message body.

Alert Object ID: A string of digits, in brackets, formatted like this:

    [AlertObjectID=99999]

For backward compatibility, the message may contain these elements instead of the Alert Object ID.

Alert definition ID: A GUID in brackets, formatted like this:</>

    [AlertDefinition=D1A5279D-B27D-4CD4-A05E-EFDD53D08E8D]

Object ID: A string of digits, in brackets, formatted like this:

    [ObjectID=99999]

Object Type: A string representing a valid Solarwinds object type, formatted like this:

    [ObjectType=APM: Application]

Commands:

The command must be the first word in the message body.

ack:    Acknowledge the alert and append a comment if one is supplied.

comment: Append a comment to the alert

Comments:

Comments are optional. Any text between the command verb and a newline or return character will be treated as a comment. There are no formatting requirements for comments.

Example 1:

This message would acknowledge an alert and append a comment

    Ack  Alert comment from ME

    [AlertObjectID=46673]

Example 2:

This message would also acknowledge an alert and append a comment.

    Ack Kurt is looking into this

    ~~~~~~~~~~~~~~~~~~~~~

    Me, Senior Systems Administrator

    ~~~~~~~~~~~~~~~~~~~~~~

    From: Solarwinds

    Sent: Friday, October 02, 2015 12:33 PM

    To: Me <me@myemail.com>

    Subject: Solarwinds Alert: Oracle database disk utilization KURTSDB on Node kurtsserver.mydomain.com is Down

    When replying, type your text above this line

    ________________________________________

    To acknowledge this alert, click here or reply to this message with "ack" as the first word in the body.

    Alert details:

    Oracle database disk utilization KURTSDB on Node kurtsserver.mydomain.com is Down at Friday, October 2, 2015 12:32 PM.

    When replying, do not delete text below

    ________________________________________

    [AlertObjectID=46673]

Example 3:

Deprecated format for backwards compatibility. This message would also acknowledge an alert and append a comment

    Ack  Third alert comment from ME

    [AlertDefinition=535d1493-a543-4df0-acbf-6b43770aceeb] [ObjectID=673] [ObjectType=APM: Application]

process_alert_reply_email_v2.ps1
  • Get the XML payload created by this portion of the script:

        $xmlString = "<ArrayOfAlertInfo xmlns='http://schemas.solarwinds.com/2008/Orion'>

    <AlertInfo>

        <DefinitionId>$($alertDefId)</DefinitionId>

            <ObjectType>$($objectType)</ObjectType>

            <ObjectId>$($objectId)</ObjectId>

        </AlertInfo>

    </ArrayOfAlertInfo>"

    How does the XML look?

    How about the Solarwinds ID you're using to run the script - can you log in to Solarwinds with it and acknowledge alerts manually?

  • Hi M

    Yes I can definitely login using the creds I provide to the script and it has full privileges.  I should get a chance to look at this tomorrow and let you know. about the XML payload   

    I did have to amend the Exchange version as we are on 2013.  But as I said the script completes, writes the id's to host and  and sends a confirmation email so the issue is with the ack command not being actioned.  I will reply soon  thanks for checking in

  • Hi All, I have been looking for a solution to ack Alerts by email without using AlertCentral, looks like this is exactly what we need, checking on the requirements I see that script assumes we have Exchange Platform, so you know if works on the same way with exchange online? we are already moved to the cloud so just wondering to know if should work before doing all the config and testing for the script.

    Thank you!

  • Does Exchange online have a dedicated URL associated with it? As long as this bit of the script works, you should be OK:

    $s = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2007_SP1)

    $s.Credentials = New-Object Net.NetworkCredential($exchangeUsername, $exchangePassword, $exchangeDomain)

    $s.AutodiscoverUrl($exchangeEmail)

    If autodiscover doesn't work with Exchange online, you should be able to set the Url property manually:

    $s.Url = (your Exchange online URL)

  • hey, thanks for your prompt response, discovery is working on our environment so I don't see a major issue, I will try to get this done today.

    Regards,

  • I am having an issue getting this to work. Autodiscovery is not working within powershell/ews for me (possibly due to our hybrid Office 365 environment). I have made the below change to attempt to resolve this:

    # Discover the URL from the Exchange email address

    #$s.AutodiscoverUrl($exchangeEmail)

    $s.Url = "https://outlook.office365.com/EWS/Exchange.asmx"

    However I am still getting the below error.

    Exception calling "Bind" with "2" argument(s): "The request failed. The underlying connection was closed: An unexpected error occurred on a send."

    At C:\scripts\process_alert_reply_email.ps1:286 char:1

    + $inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($s,[Microsoft.Exchan ...

    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException

        + FullyQualifiedErrorId : ServiceRequestException

    I have been trying for over a month to get this working, but I don't know powershell not to mention EWS. Any assistance would be greatly appreciated.

  • I haven't run into this error before, but it looks like this is usually a problem with SSL. One workaround is here: EWS Api problem connecting Inbox (look for Glen's reply about halfway down the page).

  • Is anyone monitoring this thread anymore? I was wondering if this would work with an O365 deployment?

  • I'm monitoring it, but we aren't using O365 so I can't answer your question.

    jkrenzien​ , did you get this to work with your Office 365 environment?

  • We are doing an Office 365 migration and I was able to get it to work with changing the following line for autodiscover to the generic 365 address with no issues. This is generic for all 365 users so this should work

    # Discover the URL from the Exchange email address\

    $s.url = "https://outlook.office365.com/EWS/Exchange.asmx";

    Also not a huge deal but you may want to change the code to point to the new version of MS Exchange Web Services Managed API which is 2.2 in the script. Easy update to do but wanted to point it out. Great job on this.