15 Replies Latest reply on Aug 23, 2018 3:16 AM by rajasekar

    Slack integration not working

    havox

      We are trying to implement alerting via a Slack channel and are running into problems.  After following the Solarwinds guide, everything works from the command line of the Orion server but we never get a message when it is triggered from an actual alert.  We see the "Send Slack Message" event task run in Orion but the message never shows up in Slack.  What could we be missing?  My guess is that it is a permissions issue when calling the curl.exe.  Is there a certain permission that should be allowed for this to work?

       

      Thanks,

      Jon

        • Re: Slack integration not working
          jm_sysadmin

          If the curl works when you run it, it may be permissions. Try to define a user to run the command:

            • Re: Slack integration not working
              havox

              Tried that and it didn't work.  We got this message Check 'Replace a process level token' account privilege of user which runs Solarwinds Orion Module Engine service.

               

              We did the following to the user account we are trying to use:

               

              - made local admin on Orion box

              - gave Full Control to curl script directory

              - made user the Log On account for the SolarWinds Orion Module Engine service

              - added user to "Replace a process level token" in Local Security Policy on Orion server

              - restarted all Orion services

               

              After adding the user to the Alert and running the Test Credentials, we get the same error as above.  I have used domain\username and username@domain with no success.  Solarwinds Support pretty much gave up and pointed me to Thwack.

               

              Thanks for the help.

                • Re: Slack integration not working
                  jm_sysadmin

                  Can you run anything from the executable action? Something like:

                  "C:\Windows\System32\ipconfig.exe" > %temp%\ipconfig.txt

                   

                  Don't specify a user first.

                  Also, did you clear the downloaded form the internet flag on curl and its resources?

                   

                  Unblock-File

                    • Re: Slack integration not working
                      havox

                      Just writing to the txt file did not work either.

                        • Re: Slack integration not working
                          havox

                          If I edit the Notify Slack Action I created to define a user, then press the Test Credentials button, it fails with the same message: Check 'Replace a process level token' account privilege of user which runs Solarwinds Orion Module Engine service.

                           

                          I have done all the steps (as detailed above) to give the user the proper access and still no go.

                        • Re: Slack integration not working
                          rajasekar

                          Am trying to integrate the solarwinds with slack everything running fine but not receiving the message in slack channel in solarwinds server am getting error like WARNING - The underlying connection was closed: An unexpected error occurred on a send kindly share the steps to resolve this issue.

                            • Re: Slack integration not working
                              jm_sysadmin

                              So I can make a guess, but let start with simple and work our way up.

                              I am basing this on adatole's  guide to slack integration, its likely that is how you got started.

                              The Incomplete Guide to Integrating SolarWinds Orion and Slack

                               

                              On page 14 of the pdf when you run the command in a local shell, not an alert via curl does it work or do you get an error?

                              C:\sw _ tools\curl\bin\curl.exe -X POST --data-urlencode “payload={\”text\”:\”This is a line of text.\nAnd this is another one.\”}” https://YOUR _ SLACK _WEBHOOK _ GOES _ HERE!

                               

                              If that works, and its erring in Powershell, screenshot the whole window you are testing in. If the Powershell sample script works (either or both the samples) but it errors when you use it in an alert, let me know.

                                • Re: Slack integration not working
                                  rajasekar

                                  Am using the path C:\windows\system32\windowspowershell\v1.0\powershell.exe -ExecutionPolicy unrestricted -command "D:\Slack\Slack_Alert_Trigger_Action_Node.ps1 '${N=SwisEntity;M=DisplayName}' '${N=SwisEntity;M=Status;F=Status}' '${N=Alerting;M=AlertDetailsUrl}' '${N=SwisEntity;M=DetailsUrl}' '${N=Alerting;M=AcknowledgeUrl}' '${N=Generic;M=DateTime;F=OriginalValue}'"

                                  in the alert and in the script am using the hookurl

                                    • Re: Slack integration not working
                                      jm_sysadmin

                                      So you didn't say it, but the curl command worked?

                                       

                                      And I didn't see any screenshots, but if you could for the next part it will help me. The code below is mostly not mine I just trimmed it up to make it simple.

                                       

                                      Open Powershell console window.

                                      After updating the webhook, copy the code into the console window:

                                       

                                      $webhook = '<YOURWEBHOOK>'
                                      $slackJSON = @{ }
                                      $slackJSON.channel = '#general'
                                      $slackJSON.username = 'OrionAlert'
                                      $slackJSON.text = 'Text message here to test the send'
                                      $webReq = @{
                                      Uri      = $webhook
                                      ContentType  = 'application/json'
                                      Method      = 'Post'
                                      body = ConvertTo-Json $slackJSON
                                      }

                                      if there are no errors then send the command

                                       

                                      Invoke-WebRequest @webReq

                                       

                                      If it works let me know or screenshot the error. You can also try these two lines:

                                      $tempbody = ConvertTo-Json $slackJSON
                                      Invoke-WebRequest -Uri $webhook -ContentType 'application/json' -Method 'Post' -body $tempbody
                                        • Re: Slack integration not working
                                          rajasekar

                                          Hi,

                                          once i run

                                          1. $webhook = '<YOURWEBHOOK>'
                                          2. $slackJSON = @{ } 
                                          3. $slackJSON.channel = '#general'
                                          4. $slackJSON.username = 'OrionAlert'
                                          5. $slackJSON.text = 'Text message here to test the send'
                                          6. $webReq = @{ 
                                          7. Uri      = $webhook 
                                          8. ContentType  = 'application/json'
                                          9. Method      = 'Post'
                                          10. body = ConvertTo-Json $slackJSON 

                                           

                                          Am not getting any output but after running the Invoke-WebRequest @webReq am getting the output in SLACK.

                                           

                                            • Re: Slack integration not working
                                              jm_sysadmin

                                              Great, that means that a bunch of stuff isn't broken. Do you get WARNING - The underlying connection was closed: An unexpected error occurred on a send  when you run the script from the console and from the alert or just one?

                                                • Re: Slack integration not working
                                                  rajasekar

                                                  When i try to execute the script via solarwinds portal by the external program alert feature by C:\windows\system32\windowspowershell\v1.0\powershell.exe -ExecutionPolicy unrestricted -command "D:\Slack\Slack_Alert_Trigger_Action_Node.ps1 '${N=SwisEntity;M=DisplayName}' '${N=SwisEntity;M=Status;F=Status}' '${N=Alerting;M=AlertDetailsUrl}' '${N=SwisEntity;M=DetailsUrl}' '${N=Alerting;M=AcknowledgeUrl}' '${N=Generic;M=DateTime;F=OriginalValue}'" that time am getting this error in C:\ProgramData\Solarwinds\Logs\Slack_Node_Alert.log

                                                   

                                                  below is the script which am using.

                                                   

                                                   

                                                   

                                                  # This script is used to make an API call into Slack's API to send a message.

                                                  # Created by Thwack user Chad.Every

                                                  # Last modified on: 2016/05/05

                                                  # See https://thwack.solarwinds.com/docs/DOC-188387 for more detail.

                                                  param([string]$ORIONNODENAME, [string]$ORIONNODESTATE, [string]$ALERTDETAILS, [string]$NODEDETAILS, [string]$ACKURL, [string]$ORIONTIME)

                                                   

                                                   

                                                  $SlackWebHook = "XXX"

                                                   

                                                   

                                                  $slackJSON = @{}

                                                  $slackJSON.channel = "#XXX"

                                                  $slackJSON.username = "OrionBot"

                                                  $slackJSON.icon_url = "https://lh3.googleusercontent.com/-SV1-YUgXozE/AAAAAAAAAAI/AAAAAAAABFE/LVzp7vgXZbs/photo.jpg"

                                                   

                                                   

                                                  $LOGFILEPATH="C:\ProgramData\Solarwinds\Logs\Slack_Node_Alert.log"

                                                   

                                                   

                                                  function WriteLog([string]$log,[int]$errorcode)

                                                  {

                                                      if ($errorcode -eq 0) {$LogLevel="INFO"}

                                                      else {$LogLevel="WARNING"}

                                                   

                                                   

                                                      #Write message to log file

                                                      "[$([DateTime]::Now)] - $($LogLevel) - $($log)" | Out-file -filepath $LOGFILEPATH -Append;

                                                   

                                                   

                                                      #If error (warning) occurs, exit the application

                                                      if ($errorcode -eq 1) {exit 1;}

                                                  }

                                                   

                                                   

                                                  Try

                                                      #Set emoji and message format

                                                      $emoji = ""

                                                      $MessageFormat= ""

                                                      switch ($ORIONNODESTATE)

                                                      {

                                                          Unknown {$emoji = ":construction:"}

                                                          Up {$emoji = ":green_heart:"; $MessageFormat="Up";$color="good" }

                                                          Down {$emoji = ":skull:";$color="#222222"}

                                                          Warning {$emoji = ":angry:";$color="warning" }

                                                          Critical {$emoji = ":rage:";$color="danger" }

                                                      }

                                                     

                                                      $ORIONTIME=$ORIONTIME -replace '^(.*?)\s(.*)','$1 @ $2'

                                                      $ORIONNODENAME=$($ORIONNODENAME.ToUpper().Trim())

                                                      $ORIONNODESTATE=$($ORIONNODESTATE.Trim())

                                                     

                                                      if ($MessageFormat -eq "Up")

                                                      {

                                                          $slackJSON.attachments = @(

                                                              @{

                                                                  fallback = "Node: ``$($ORIONNODENAME)`` status changed to *$($ORIONNODESTATE)* $($emoji) `n>Time: $($ORIONTIME)"

                                                                  author_name = "Node: $($ORIONNODENAME)"

                                                                  author_link = $NODEDETAILS

                                                                  color = $color

                                                                  fields = @(

                                                                      @{

                                                                      title = "Status"

                                                                      value = "$($ORIONNODESTATE) $($emoji)"

                                                                      short = "true"

                                                                      }

                                                                      @{

                                                                      title = "Time"

                                                                      value = $ORIONTIME

                                                                      short = "true"

                                                                      }

                                                                  )

                                                               }

                                                          )

                                                      }

                                                      else

                                                      {

                                                          $slackJSON.attachments = @(

                                                              @{

                                                                  fallback = "Node: ``$($ORIONNODENAME)`` status changed to *$($ORIONNODESTATE)* $($emoji) `n>View: <$($ALERTDETAILS)|Alert Details>, <$($NODEDETAILS)|Node Details> or <$($ACKURL)|acknowledge> `n>Time: $($ORIONTIME)"

                                                                  author_name = "Node: $($ORIONNODENAME)"

                                                                  author_link = $NODEDETAILS

                                                                  color = $color

                                                                  fields = @(

                                                                      @{

                                                                      title = "Status"

                                                                      value = "$($ORIONNODESTATE) $($emoji)"

                                                                      short = "true"

                                                                      }

                                                                      @{

                                                                      title = "Time"

                                                                      value = $ORIONTIME

                                                                      short = "true"

                                                                      }

                                                                      @{

                                                                      title = "Alert Links"

                                                                      value = "<$($ALERTDETAILS)|Alert Details>, or <$($ACKURL)|Acknowledge>"

                                                                      short = "false"

                                                                      }

                                                                  )

                                                               }

                                                          )

                                                      }

                                                   

                                                   

                                                      $webReq=@{

                                                          Uri = $SlackWebHook

                                                          ContentType = "application/json"

                                                          Method = "Post"

                                                          body = ConvertTo-Json -Depth 4 $slackJSON

                                                      }

                                                   

                                                   

                                                      Invoke-RestMethod @webReq

                                                   

                                                   

                                                      WriteLog $slackJSON.text 0

                                                  }

                                                  Catch

                                                  {

                                                      #Catch any powershell errors and send them to the log file.

                                                      $Errormsg = $_.Exception.Message

                                                      WriteLog $Errormsg 1

                                                  }

                                                  • Re: Slack integration not working
                                                    rajasekar

                                                    i try via solarwinds alert that time am getting that error.

                                                    • Re: Slack integration not working
                                                      rajasekar

                                                      Am running the script via solar winds alert manually there it shows successful but am not getting any message in slack channel that time am getting the error WARNING - The underlying connection was closed: An unexpected error occurred on a send in log folder.