56 Replies Latest reply on Aug 16, 2019 2:06 AM by epicor.jastalos

    Check All Services Which Set To Automatic Mode Are Running

    alexslv

      Hi All,

       

      This is my first post   OK, straight to the point:

       

      Attached template is based on VBScript and will check status of all services on the Windows based machines. If those services which set to automatic are not running script will increase "Statistic" counter. I have set threshold to CRITICAL = "1", as there is no real reason why AUTOMATIC service should not be running (there are some exceptions of course). The template will also report back names (Service Display Name) of those services which are set to automatic and not running. This is handled by "Massage" variable and will be visible in SAM, so, you can see in SolarWinds SAM all you need straight away.

       

      You can also define exceptions. Well, this can be improved to make it bit more easier to manage. At the moment you will need to modify VBScript itself. Just open it and you will see straight away the line of code which you need to copy-pase. Add additional exception if you need to.

       

      Thank you very much for using it,

       

      Best of luck,

      Alex

        • Re: Check All Services Which Set To Automatic Mode Are Running
          dc_san

          works great, thanks for sharing!!!!!

          1 of 1 people found this helpful
          • Re: Check All Services Which Set To Automatic Mode Are Running
            aLTeReGo

            Great Template alexslv. You might want to consider posting this in the Server & Application Monitor Content Exchange, where it will get greater visibility.

              • Re: Check All Services Which Set To Automatic Mode Are Running
                orioncrack

                Alex,

                 

                Any help making this work for one service? I can't code, I envy you.

                 

                 

                 

                Const wbemFlagReturnImmediately = &h10

                Const wbemFlagForwardOnly = &h20

                 

                Dim MonitoredServer

                Dim arrComputers

                Dim StrComputer

                Dim objItem, objWMIService, colItems

                Dim Statistic

                 

                'SCRIPT INPUT: ${IP}

                MonitoredServer = WScript.Arguments(0)

                 

                 

                For Each strComputer In arrComputers

                Const strSVCName = "erm8"

                arrComputers = Array(MonitoredServer)

                 

                statistic = 0

                 

                For Each strComputer In arrComputers

                    Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")

                    Set colServices = objWMIService.ExecQuery("Select * from Win32_Service WHERE DisplayName = '" & strSVCName & "'")

                       For Each objItem In colServices

                           If objItem.StartMod = "Manual" Then

                             statistic = statistic + 1

                        End If

                    Next

                          

                            End If

                        End If

                      End If

                   Next

                Next

                1 of 1 people found this helpful
                  • Re: Check All Services Which Set To Automatic Mode Are Running
                    alexslv

                    What do you mean "for one service"? This script is specifically designed to handle all of them. If you just need one only - I suggest you use out-of-the-box solution to monitor services - it is called "Windows Service Monitor" when you build an application template and select add new component

                      • Re: Check All Services Which Set To Automatic Mode Are Running
                        orioncrack

                        Right. I'm doing that bu this guy wants to know if someone changes 1 service to manual.

                         

                        I have all the code in the world to do it but can't string it together properly. It's like putting my mom in the cockpit of a Lear jet. I would die to have ninja scripting skills.

                          • Re: Check All Services Which Set To Automatic Mode Are Running
                            alexslv

                            You don't need ninja skills - Google has all you need

                             

                            By the way, this is exactly the idea of this script - if you don't want this script to pick up any services - you just stick them to "manual" mode and job done. In your case however it is not a service you need to be monitoring, but your people. If "someone" has changed it to manual - I guess you need to find out who and why. There might have been a reason of doing so. I can see that in your above attempt you are trying to trigger an alert if your "erm8" service is set to manual... Hmmm... not the best approach. If I were you I would send a blanket email to technical team advising them that under any circumstances should automatic services be switched to manual, unless it has been approved by relevant team/engineers. Also, I am sure windows is logging an event every time you switch service to different mode - I guess you can use this event to trigger an alert on service mode change if needed.

                             

                            If you still want to proceed with your script, try to replace the following line with the below - as it is just one service - simply supply it in query as below:

                             

                            Set colServices = objWMIService.ExecQuery("Select * from Win32_Service WHERE DisplayName = 'erm8' ")

                              • Re: Check All Services Which Set To Automatic Mode Are Running
                                orioncrack

                                Thanks Alex. I will test. You are the OB1 to my Princess Leia.  

                                  • Re: Check All Services Which Set To Automatic Mode Are Running
                                    alexslv

                                    no problem

                                     

                                    By the way, in your script above it seems that you have too many "End If" clauses at the end. For every "For" there should be one "Next" at the end. For every "If" there may be "Then" somewhere in the middle, but there must be one closing "End If". Also you had a typo - "objItem.StartMode" not "objItem.StartMod"...

                                     

                                    So, after testing - here is what you get - working script below:

                                     

                                    =====================================

                                     

                                    Const wbemFlagReturnImmediately = &h10

                                    Const wbemFlagForwardOnly = &h20

                                     

                                    Dim MonitoredServer

                                    Dim arrComputers

                                    Dim StrComputer

                                    Dim objItem, objWMIService, colServices

                                    Dim Statistic, Message

                                     

                                    'SCRIPT INPUT: ${IP}

                                    MonitoredServer = WScript.Arguments(0)

                                    arrComputers = Array(MonitoredServer)

                                     

                                    Statistic = 0

                                    Message = "Hey, Scripting Guy ... erm8 is not Manual, but who knows - maybe someone has disabled it or it does not exist at all ... I need to make sure server access is properly secured and my engineers know what they are doing!"

                                     

                                     

                                    For Each strComputer In arrComputers

                                        Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")

                                        Set colServices = objWMIService.ExecQuery("Select * from Win32_Service WHERE DisplayName = 'erm8'")

                                      For Each objItem In colServices

                                      If objItem.StartMode = "Manual" Then

                                      Statistic = Statistic + 1

                                      Message = "Oh my Gosh, erm8 has been set to Manual again!!! Whoever did it will get a massive slap when I find you!!!"

                                      End If

                                      Next

                                    Next

                                     

                                     

                                    WScript.Echo "Message: " & Message

                                    WScript.Echo "Statistic: " & CInt( Statistic )

                                    1 of 1 people found this helpful
                        • Re: Check All Services Which Set To Automatic Mode Are Running
                          cscoengineer

                          Very nicely done!!   I was able to convert a Nagios monitoring to SW using this script.  The only changes were the addition of exclusions.

                           

                          Nagios:

                          a1.png

                          SolarWinds SAM "Check Automatic Services Are Running.apm-template" with WQL exclusions

                          a2.png

                          Thanks

                          • Re: Check All Services Which Set To Automatic Mode Are Running
                            cscoengineer

                            In addition to the monitoring, I always like to couple it with a report.  This aids in alerting.

                             

                            select

                            n.caption [Node]

                            ,cast(substring(ARD,MultiValueStastics, 11,5) as numeric(5,0)) [Service Down]

                            ,substring(ARD.MultiValueMessages, 37, 500) [ServiceNames]

                            from nodes n with (nolock)

                            inner join APM_AlertsAndReportsData ARD with (nolock) on (ARD.nodeid=n.nodeid)

                            where (ARD.ComponentStatus like 'Crit%' and ARD.ApplicationName like 'Auto%')

                            Order by [Service Down] Desc

                             

                            1 of 1 people found this helpful
                              • Re: Check All Services Which Set To Automatic Mode Are Running
                                alexslv

                                Oou yes, definitely Well done. From my experience, when we first apply this template across all nodes in infrastructure - it picks up sooooo many problems, that we just have to use reporting to start with to go through all of them, before we enable alerts themselves.

                                • Re: Check All Services Which Set To Automatic Mode Are Running
                                  gcow

                                  Thank you for adding the report - very useful.  When I tried it I got "Query not valid" due to a typo.  Fixing that yielded no results as the app name didn't match.  Then the string split was slightly out.  So I tweaked it slightly to match the original template:

                                  select

                                  a.n.caption [Node]

                                  ,cast(substring(ARD.MultiValueStatistics, 11,5) as

                                  numeric(5,0)) [Service Down]

                                  ,substring(ARD.MultiValueMessages, 31, 500) [ServiceNames]

                                  from nodes n with (nolock)

                                  inner join APM_AlertsAndReportsData ARD with (nolock) on

                                  (ARD.nodeid=n.nodeid)

                                  where (ARD.ComponentStatus like 'Crit%' and

                                  ARD.ApplicationName like 'Check Automatic Services Are Running')

                                  Order by [Service Down] Desc

                                  • Re: Check All Services Which Set To Automatic Mode Are Running
                                    nkoconno

                                    This is a great addition.  I'm trying to run get this report created and am encountering the *query not valid* exception.

                                     

                                    Assuming I'm just doing something wrong. I'm trying to create a new report (Web Manager, Manage Reports, Create New Report, Add Content: Custom Table, Advanced Database Query; when using this or the suggestion below from gcow, both show the query not valid exception.

                                     

                                    I found the string in question that should be included in the report via this simple SQL query:

                                    SELECT * FROM [dbo].[APM_AlertsAndReportsData]

                                    where ComponentName='Check Automatic Services Are Running' AND ComponentStatus='Critical'

                                     

                                     

                                     

                                    Desired Value contained in MultiValueMessages

                                     

                                    Can someone please point me in the right direction to get the report working correctly?

                                  • Re: Check All Services Which Set To Automatic Mode Are Running
                                    gtierney

                                    This is exactly what I needed, Thanks Alex!! 

                                     

                                    I ran into a similar issue where our development team was spinning up new services on servers that needed to be monitored.  As you can imagine, this becomes a royal pain as we would need to add the new service to monitoring each time.  What I needed was a monitor that would check any service on a node with a specific prefixed name such as "CORE." and was set to automatic startup. 

                                     

                                     

                                    Below is the VB Script that I used

                                     

                                    ______________________________________________________________________________

                                    Const wbemFlagReturnImmediately = &h10

                                    Const wbemFlagForwardOnly = &h20

                                     

                                    Dim MonitoredServer

                                    Dim arrComputers

                                    Dim StrComputer

                                    Dim objItem, objWMIService, colItems

                                    Dim Statistic

                                    Dim Message

                                     

                                    'SCRIPT INPUT: ${IP}

                                    MonitoredServer = WScript.Arguments(0)

                                     

                                    arrComputers = Array(MonitoredServer)

                                    For Each strComputer In arrComputers

                                        statistic = 0

                                        message = "Stopped services - "

                                     

                                        Set objWMIService = GetObject("winmgmts:" & "{impersonationLevel=impersonate}!\\" & strComputer & "\root\cimv2")

                                     

                                        Set colItems = objWMIService.ExecQuery("SELECT * FROM Win32_Service where name like 'core.%'", "WQL", wbemFlagReturnImmediately + wbemFlagForwardOnly)

                                     

                                        For Each objItem In colItems

                                            If StrComp(objItem.StartMode,"Auto") <> 0 Then

                                                If StrComp(objItem.State,"Running") <> 0 Then

                                                    statistic = statistic + 1

                                                    message = message & " | " & objItem.DisplayName

                                                End If

                                            End If

                                        Next

                                    Next

                                     

                                    If statistic = 0 Then

                                        WScript.Echo "Message: All services are running"

                                        WScript.Echo "Statistic: 0"

                                        Wscript.quit(0)

                                    else

                                        WScript.Echo "Message: " & message

                                        WScript.Echo "Statistic: " & statistic

                                        Wscript.quit(1)

                                    End If

                                    ______________________________________________________________________________

                                     

                                    There are a couple of tweaks I am planning to do with this like adding the service name search prefix as a parameter passed into the script by the calling component check, probably removing the server loop as only one server is being passed into the script at a time and maybe automatically restarting the service but for now, this works great.

                                     

                                    Thanks again!

                                    1 of 1 people found this helpful
                                    • Re: Check All Services Which Set To Automatic Mode Are Running
                                      torwals

                                      Hei,

                                      Sorry beforehand for being a complete noob in scripting in general (especially with vbscript). Our company just started using this gem of a template on a subnumber of our servers. I was wondering what type of value the script is looking after when doing exceptions and where. Under I will link a part of the script after the text  says the exceptions needs to be added and I would love someone just to enter in example exception at the right line, so I can see what is needed. Thank you already for taking your time.

                                       

                                      <Value>This script will check all automatic services and return count number of how many are not running. It is possible to setup exception, but this needs to be done by modifying script itself (you can do it below, here)</Value>

                                                                  <ValueType>String</ValueType>

                                                                  <Key>__UserDescription</Key>

                                                              </d5p1:Value>

                                                          </d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                          <d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                              <d5p1:Key>__UserNotes</d5p1:Key>

                                                              <d5p1:Value>

                                                                  <Required>false</Required>

                                                                  <SettingLevel>Template</SettingLevel>

                                                                  <Value></Value>

                                                                  <ValueType>String</ValueType>

                                                                  <Key>__UserNotes</Key>

                                                              </d5p1:Value>

                                                          </d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                          <d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                              <d5p1:Key>CountAsDifference</d5p1:Key>

                                                              <d5p1:Value>

                                                                  <Required>false</Required>

                                                                  <SettingLevel>Template</SettingLevel>

                                                                  <Value>false</Value>

                                                                  <ValueType>Boolean</ValueType>

                                                                  <Key>CountAsDifference</Key>

                                                              </d5p1:Value>

                                                          </d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                          <d5p1:KeyValueOfstringSettingValueyR_SGpLPx>

                                                              <d5p1:Key>ScriptArguments</d5p1:Key>

                                                              <d5p1:Value>

                                      • Re: Check All Services Which Set To Automatic Mode Are Running
                                        t.imtiyaz

                                        Hi Alex,

                                         

                                        When Testing the script on one of the node i am getting below error. Can you please help me out..

                                         

                                        Unable to impersonate user.This user must be a valid user on the Orion Server/Polling Engine.

                                         

                                        I have selected <inherit credential from node>

                                         

                                        Thanks,

                                        • Re: Check All Services Which Set To Automatic Mode Are Running
                                          faisal.ishfaq

                                          Thanks alot for this, with no knowledge of scripting and hours of search and scratching my head I finally landed on this post which is working as a charm

                                           

                                          Also I would like to thank @gcow for explaining how to put the exception in the script in the comments below as some of the services are sitting there set as automatic however only works on demand and stays in "non-running" state.

                                          • Re: Check All Services Which Set To Automatic Mode Are Running
                                            faisal.ishfaq

                                            I wonder if there is a script similar for Linux running servers please? or you could recommend any.

                                             

                                            Thanks in advance

                                            • Re: Check All Services Which Set To Automatic Mode Are Running
                                              jameslindsay

                                              Once you have loaded and deployed the template to a node, what are the steps to configure an alert to trigger an email?

                                              • Re: Check All Services Which Set To Automatic Mode Are Running
                                                jmayrand

                                                Thanks for this!  Was looking for something exactly like it!

                                                • Re: Check All Services Which Set To Automatic Mode Are Running
                                                  epicor.jastalos

                                                  Hello all - thank you, this is a great script!

                                                   

                                                  How could I start/restart services found NOT running?

                                                   

                                                  Thank you,

                                                  Jan

                                                  • Re: Check All Services Which Set To Automatic Mode Are Running
                                                    marcv

                                                    alexslv

                                                    Was this ever uploaded to content exchange? I can can't find it when i search content exchange using Sam Console?

                                                    -marc

                                                    • Re: Check All Services Which Set To Automatic Mode Are Running
                                                      venubabu

                                                      Hello Alex,

                                                      Firstly thanks for sharing VBscript. It works great per expectations. and I had a query regarding the message out put, if mutliple services are stopped I would like to see one alert per one service so that it makes easier to troubleshoot our team to work on separte incidnets,  Is this something possible modifying the script, I tried several ways in trigger actions but did not work though. Please assist, thanks in advance.

                                                        • Re: Check All Services Which Set To Automatic Mode Are Running
                                                          alexslv

                                                          There is no easy way. The whole idea here is to save on overhead in having multiple components, multiple alerts, reduced scope, etc. In this case it is all simple and straightforward - ONE component, all services, everything is in scope, no inconsistencies ... 100% win-win-win

                                                           

                                                          If I would desperately need to implement something what you are asking - I would do the following:

                                                           

                                                          1. Design SQL script to pull out all message variables from all automatic service components across all servers. Message variable contains all stopped services

                                                          2. Split message per component into separate rows so that you have one service per row (example below)

                                                           

                                                          server1 | abs stopped

                                                          server1 | cdf stopped

                                                          server2 | efg stopped

                                                           

                                                          3. Create standard SQL view based on the above output

                                                          4. Use advanced SQL option for your new alert and lookup for the data in this view. Every row will yield an alert. In this case 2 nodes will result in 3 alerts

                                                           

                                                          Above concept is doable. Most difficult part would be to find SQL guru to design query as per step (2). The rest is simple.

                                                           

                                                          Even if you will accomplish this - I still think one alert per node is enough. More often then not it is just one service failing at a time. If you have multiple failure simultaneously - I bet it still all relates to same issue. So, I would not trigger multiple alerts anyway

                                                           

                                                           

                                                          BONUS:

                                                           

                                                          • One interesting approach is to setup an alert something like below:
                                                            • Trigger on component
                                                              • If component name = "Automatic Services"
                                                              • AND Status = Critical (or warning, depending on your setup)
                                                              • AND message contains "MSSQLSERVER"

                                                           

                                                          This way you will have a dedicated alert for SQL Service across your entire infrastructure (as long as this template is assigned to all servers). It is an excellent way to notify your DBAs. Your can expand this to include more services for your DBA team and obviously create more alerts for other teams as well

                                                           

                                                          Best luck!

                                                           

                                                          With Gratitude,

                                                          Alex Soul

                                                        • Re: Check All Services Which Set To Automatic Mode Are Running
                                                          nkoconno

                                                          Hey, I'm having trouble using a wildcard for the services.  Some services are named differently on each server, example "Sync Host_xxxxxxx"  I tried using

                                                           

                                                          (StrComp(objItem.DisplayName,"Sync Host_%") = 0) OR _

                                                          and

                                                          (StrComp(objItem.DisplayName,"Sync Host_*") = 0) OR _

                                                           

                                                          but neither are working.  Can you please help with what wildcard character would work here for the whitelist?


                                                          Thanks

                                                          Nick

                                                          • Re: Check All Services Which Set To Automatic Mode Are Running
                                                            alexslv

                                                            Recently I had to exclude some services globally, by using a wildcard. jonsc has approached me with exact same question, so, sharing it here with everyone:

                                                             

                                                            For the exact match we are currently using StrComp matching function, example below:

                                                             

                                                            (StrComp(objItem.DisplayName,"Windows Update") = 0)

                                                             

                                                            For the wildcard match you need to use InStr function, example below:

                                                             

                                                            (InStr(objItem.DisplayName, "Sync Host_") > 0)

                                                             

                                                            Happy SolarWinding,

                                                            Alex

                                                            1 of 1 people found this helpful
                                                            • Re: Check All Services Which Set To Automatic Mode Are Running
                                                              epicor.jastalos

                                                              Hi alexslv, all

                                                               

                                                              I've been using this script for a while, but getting this error message on some servers (and app monitor is obviously not assigned during app discovery):

                                                               

                                                              C:\ProgramData\SolarWinds\Orion\temp\d6bb840f-612f-4686-8e74-eb6519268a4a(20, 4) Microsoft VBScript runtime error: Permission denied: 'GetObject'

                                                              Errors: ==============================================

                                                              C:\ProgramData\SolarWinds\Orion\temp\d6bb840f-612f-4686-8e74-eb6519268a4a(20, 4) Microsoft VBScript runtime error: Permission denied: 'GetObject'

                                                               

                                                              Note: Yes, I use admin account.

                                                               

                                                              Could you please advise?

                                                              Thank you.

                                                              Jan

                                                                • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                  alexslv

                                                                  Ensure WMI works on those servers. This is usually the case for such errors

                                                                   

                                                                  >

                                                                    • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                      epicor.jastalos

                                                                      hI alexslv

                                                                       

                                                                      I've identified an issue - because of multiple domains, using vbscript is problematic.

                                                                      A user with admin permissions on both Orion and Target server is required.

                                                                      obviously, I cannot create a domain user that has permissions in all (separate) domains.

                                                                      It would either need to be a local user with the same name and same password on all servers,

                                                                      or a powershell script that can be run remotely - on a target machine.

                                                                       

                                                                      Do we have this script written in PowerShell?

                                                                      Thanks,

                                                                      Jan

                                                                        • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                          alexslv

                                                                          I would suggest to simply create multiple templates for each domain and

                                                                          assign appropriate users for each template. Then you can assign those

                                                                          templates to multiple domains

                                                                            • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                              epicor.jastalos

                                                                              alexslv - that does not make any sense to me. If this was true, it could be achieved also on the application monitor level by changing an account per app monitor. My understanding is when working with Windows Script Monitor = VBSCRIPT, an account with both Orion admin permissions and Target admin server permissions must be used. If Orion server is in a different domain then a target machine, it cannot be achieved by using a domain account.

                                                                               

                                                                              Possibly only a local account with same user&password on Orion and on target server.

                                                                               

                                                                              Details from the Use Windows Script monitors:

                                                                               

                                                                               

                                                                              Credential for Monitoring

                                                                              Select a Windows credential that is both a user who can log on to the Orion server, and has sufficient rights on the target node (which may be the Orion server itself, depending upon your application) to do whatever the script needs to do. For example, if the script does something with WMI, the credentials also need WMI rights on the target node.

                                                                               

                                                                              Please correct me if I am wrong cause I've been struggling with this for a while.

                                                                               

                                                                              Thanks,

                                                                              Jan

                                                                                • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                                  alexslv

                                                                                  Try this - create a local account on the machine in another domain with

                                                                                  exact same user name and password. On the template level see if it is

                                                                                  possible not to specify domain name. This may actually work, but I am not

                                                                                  100% sure on this

                                                                                   

                                                                                  Another option is to ask your domain admins to establish interdomain trust

                                                                                  for authentication

                                                                                   

                                                                                  I guess you are trying to tap into your DMZ zone. I remember I had similar

                                                                                  challenge, but it was long ago, not sure how did I overcome this.

                                                                                   

                                                                                  By the way, this challenge has nothing to do with script. I would advise

                                                                                  you to log a ticket with SolarWinds and seek help from them - they might

                                                                                  have some workarounds in their arsenal

                                                                                   

                                                                                  Share here if you find anything

                                                                                   

                                                                                  With Gratitude

                                                                                  Alex

                                                                                    • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                                      epicor.jastalos

                                                                                      Hi alexslv

                                                                                       

                                                                                      Yes, that's what I've already mentioned - a local account with the same name and same password works, but it raises a security question.

                                                                                       

                                                                                      Domain does not have to be specified, that's why this can be achieved.

                                                                                       

                                                                                      When monitoring plenty of customers, you cannot really afford interdomain trust.

                                                                                       

                                                                                      Anyway, thanks for discussing this, sometimes helps to think loudly to find solutions

                                                                                       

                                                                                      If there was any powershell script with the same functionality, please give me a heads up.

                                                                                       

                                                                                      Thanks,

                                                                                      Jan

                                                                          • Re: Check All Services Which Set To Automatic Mode Are Running
                                                                            erikrichter

                                                                            Fantastic monitor, Alex!!

                                                                             

                                                                            Is there a way to filter out the "Delayed Start" and "Triggered Start" services? I'm pretty green with scripting...