2 Replies Latest reply on Oct 18, 2018 3:45 PM by j_dennis

    Potential ServiceNow Integration Bug Case # - 00162702

    csaldan1

      Currently, our ServiceNow SolarWinds integration is functioning to an extent but there is one major issue with it.

       

      It changes State values in most cases as it should but I've been able to replicate our issue by doing the following

      I bring node down, creates an incident if there isn't one

      I bring Node back up, sets incident state to monitoring

      I bring the Node back down, sets incident state to active

      I bring Node back up, sets incident state to monitoring

       

      Everything is functioning fine up until this point

      So lets say we have verified functionality and their monitoring period is up

       

      I set the incident state to resolved (value 6) within ServiceNow

      I bring the Node back down, it sends over work notes but it does not send a state value with it

      the payload within the Incident Integrations module shows

      {"sys_id":"<incident sys_id>","work_notes":"Orion alert has been reactivated."}

      with <incident sys_id> being the incidents sys id

       

      I figured that it might be an issue with the orion integration account not being able to update incidents in a resolved state

      to test it, i attempt to bring the node back up, it changes the state just fine on over to monitoring

       

      I figure maybe its an issue with solarwinds attempting to update state immediately following someone manually updating state on the incident (maybe it throws it into an acknowledged state and following that, it doesnt update state)

      Made sure that Do not execute this action if the alert has been acknowledged already (Recommended) was unchecked so that it will continue to update the incident following (even though it seems to update fine on node going back up using the same action)

      Attempted to see if I could get it to change state from resolved immediately following manual resolve using same action

       

      Brought node down, sets state to active

      Manually change state to resolved in ServiceNow

      Brought node back up, sets state to Monitoring successfully

       

      Issue is with Reopen state specifically following a manual change within ServiceNow to Resolved

      Reopen state works otherwise and if i bring node back up and back down, it will set state normally.

      It is only immediately following doing a manual update changing the state to resolved that it does not send over an updated state value

       

      I had thought that maybe it could be the Acknowledge State that is playing with it a bit but i set the Acknowledge State to active from resolved and was still getting the exact same issues

       

      I ran a search within the incident integrations module to see if it ever was able to update the state from resolved to something else using that same action

      query goes something like this

      comment contains state: "6"

      comment does not contain incident_state: "6"

      payload contains Orion alert has been reactivated

       

      I get 10 results but these are for things running different actions

      In the actions, instead of setting incidents to monitoring, they set them to resolved when node goes back up

      When they go back down, it is updating to active just fine meaning that its not an issue going from resolved to active using the Reopen state field

       

      So the issue is narrowed down to,

      following a manual modification of the state field, Reopen state fails to include state key value pair in the message it sends to update the incidents state

       

      Any assistance would be greatly appreciated with this issue!

      We opened a case number with support a few weeks back and recently, they suggested posting here for assistance along with including the case number

      This is a huge issue because when nodes go back down following being resolved, no one is being notified because the state is not changing back to active

        • Re: Potential ServiceNow Integration Bug Case # - 00162702
          rrichter@vcpi.com

          Just for reference, this error is showing up in the ESI.BusinessLayer.Log file. We're not sure if it's relevant or not, but if it is it's worth pointing out that the integration module is mostly working, but they payloads don't include state changes like they should. Does anyone know how we'd go about finding this null value?

           

          2018-09-26 18:05:53,912 [11] ERROR SolarWinds.ESI.Common.StateWatcher.Watcher - Exception occurred when executing detector

          1. System.ArgumentNullException: Value cannot be null.

          Parameter name: serviceNowUserName

             at SolarWinds.SNI.Data.DomainModel.AlertIncident.AcknowledgeAlert(String serviceNowUserName)

             at SolarWinds.SNI.Integration.Detectors.IncidentStateDetector.Detect()

             at SolarWinds.ESI.Common.StateWatcher.Watcher.<>c__DisplayClass12_0.<ProcessingLoop>b__1(Object dc)

          • Re: Potential ServiceNow Integration Bug Case # - 00162702
            j_dennis

            We have been experiencing this issue since the Jakarta release and the subsequent plug-in patch. It appears they removed some code that handled recognizing previously opened incidents.