SCM PowerShell Script with Alerting (Formatting Changes)

I have a PowerShell script that I am running using SCM profiles against all of my servers. It is simply "Get-LocalUser" which gets the local users on the server. 

I have alerting set to create a ticket in Solarwinds Service Desk and send an email when a change in this script output is detected.

For some reason, Solarwinds likes to change the formatting of the text and it then triggers my alert as a false positive. Sometimes it truncates the data with "..." and then 5 minutes later, it doesn't and triggers the alert.

Of course, I have opened a ticket about this and because Solarwinds tech support is absolutely trash, it has taken them a week to tell me to restart alerting services even though I informed that the entire server has been rebooted multiple times since the SCM profile and alert have been created. 

Any idea on why this would be working like this? I have the exact same issue on another profile that checks the local administrator accounts on the server as well.

  • I am providing an update to this post and will continue to do so. 

    After more time of being passed around support only to have to upload diagnostic files over and over again and then not being responded to after multiple attempts to make contact, I was finally told to jump on a Webex session. Of course, we just looked over everything that I described before, nothing new. It appears that the tech I spoke to will be escalating this to the dev department now.

    Will provide more updates as I get them.

  • Latest update: Solarwinds closed my ticket without reason. Last I heard is they were continuing to look into it. Realized today that I haven't heard from them in a while. Went to check the status of the ticket and it is closed. No warning, No "Go Screw Yourself", just closed. 

    I've now escalated this to CSMs and Sales Reps.

  • Surprised that you are getting so little traction but i think its because the people are looking at this as a SW behavior thing to fix rather than just looking at it as a simple powershell problem.

    SCM is just going to scrape whatever powershell displays, in exactly the format that powershell presents it.  Try tacking this to the end of your commands to force powershell to not truncate its output.  I don't know if orion has a max character width that they can display or process, but this should get the output more consistent.

    | Ft -autosize | out-string -width 4096 
  • Appreciate the feedback. What I don't understand is that I can run a PowerShell command a million times over and get the same results. I've never seen PowerShell randomly change uppercase to lowercase and vice versa. This is what makes me believe it is how Orion is handling the formatting. See the example below:

  • That is strange, and maybe you are right that they are doing something weird in processing, but I would be more likely to blame some quirk in powershell profiles or something like that than trying to get into rabbit holes with the Orion devs.  As you have already implied, support isnt going to be able to give you any clarity on this kind of thing, and I can't imagine why the devs would ever write their own data processing layer instead of just scraping the powershell output exactly as-is.

    The way I would approach this would be to specify in my code exactly how I want the outputs to come, such as explicitly having my outputs use .toUpper() and such and see if I could control things that way.  I cannot tell you how many times in my career powershell has done something weird because there is a lot under the covers happening that is not immediately intuitive, but if you tell it exactly what you want it to do instead of relying on default profiles and cmdlet behaviors you can usually solve for those weird bits.

    If you have a script returning explicitly capitalized data and then somehow when it shows in Orion it isn't then thats a pretty clear smoking gun that you can blame the SW devs and have to wait for a hotfix, but sometimes you have to prove things to get traction.

  • I kid you not a after you submitted this post, Solarwinds came back with an answer.

    From the diagnostics it looks like every once in few days the script returns somehow shortened output but I cannot see the outputs. Powershell sometimes decides to truncate the output. It might help to be explicit about the formatting in the script, like this:

    Get-LocalUser | Out-String -Width 160

    or even format the output differently:

    Get-LocalUser | Format-List -Property Name, Enabled, Description

    Can you try these formats to see if these show the difference in the comparison?

    I ended up implementing the second option and sitting on it. So far, I haven't had any false alerting. Coincidence that you posted that and then I get this email???????? You cracked the case!