16 Replies Latest reply on Dec 4, 2008 11:21 PM by r0berth1

    SWJobSchedulerSvc crashes with .NET Errors

      The Solar Winds Job Scheduler Service is repeatedly crashing with a .NET 2.0 error and restarting. This occurs about every 30 seconds or so. Here is the .NET error:


       Event Type: Error
      Event Source: .NET Runtime 2.0 Error Reporting
      Event Category: None
      Event ID: 5000
      Date:  10/6/2008
      Time:  4:50:35 PM
      User:  N/A
      Computer: XXXXXXXXXXXXXXX
      Description:
      EventType clr20r3, P1 swjobschedulersvc.exe, P2 1.1.0.238, P3 486c94d4, P4 system.runtime.serialization, P5 3.0.0.0, P6 470e469f, P7 2db, P8 89, P9 vzevnfupilsvc31z3h5gto45urxfkid3, P10 NIL.
      Data:
      0000: 63 00 6c 00 72 00 32 00   c.l.r.2.
      0008: 30 00 72 00 33 00 2c 00   0.r.3.,.
      0010: 20 00 73 00 77 00 6a 00    .s.w.j.
      0018: 6f 00 62 00 73 00 63 00   o.b.s.c.
      0020: 68 00 65 00 64 00 75 00   h.e.d.u.
      0028: 6c 00 65 00 72 00 73 00   l.e.r.s.
      0030: 76 00 63 00 2e 00 65 00   v.c...e.
      0038: 78 00 65 00 2c 00 20 00   x.e.,. .
      0040: 31 00 2e 00 31 00 2e 00   1...1...
      0048: 30 00 2e 00 32 00 33 00   0...2.3.
      0050: 38 00 2c 00 20 00 34 00   8.,. .4.
      0058: 38 00 36 00 63 00 39 00   8.6.c.9.
      0060: 34 00 64 00 34 00 2c 00   4.d.4.,.
      0068: 20 00 73 00 79 00 73 00    .s.y.s.
      0070: 74 00 65 00 6d 00 2e 00   t.e.m...
      0078: 72 00 75 00 6e 00 74 00   r.u.n.t.
      0080: 69 00 6d 00 65 00 2e 00   i.m.e...
      0088: 73 00 65 00 72 00 69 00   s.e.r.i.
      0090: 61 00 6c 00 69 00 7a 00   a.l.i.z.
      0098: 61 00 74 00 69 00 6f 00   a.t.i.o.
      00a0: 6e 00 2c 00 20 00 33 00   n.,. .3.
      00a8: 2e 00 30 00 2e 00 30 00   ..0...0.
      00b0: 2e 00 30 00 2c 00 20 00   ..0.,. .
      00b8: 34 00 37 00 30 00 65 00   4.7.0.e.
      00c0: 34 00 36 00 39 00 66 00   4.6.9.f.
      00c8: 2c 00 20 00 32 00 64 00   ,. .2.d.
      00d0: 62 00 2c 00 20 00 38 00   b.,. .8.
      00d8: 39 00 2c 00 20 00 76 00   9.,. .v.
      00e0: 7a 00 65 00 76 00 6e 00   z.e.v.n.
      00e8: 66 00 75 00 70 00 69 00   f.u.p.i.
      00f0: 6c 00 73 00 76 00 63 00   l.s.v.c.
      00f8: 33 00 31 00 7a 00 33 00   3.1.z.3.
      0100: 68 00 35 00 67 00 74 00   h.5.g.t.
      0108: 6f 00 34 00 35 00 75 00   o.4.5.u.
      0110: 72 00 78 00 66 00 6b 00   r.x.f.k.
      0118: 69 00 64 00 33 00 20 00   i.d.3. .
      0120: 4e 00 49 00 4c 00 0d 00   N.I.L...
      0128: 0a 00                     ..     


      Anyone have any ideas?

        • Re: SWJobSchedulerSvc crashes with .NET Errors
          HemiTruck

          do you have .NET 3.0 loaded on the machine?


          unless you are running pre v8 you should be running .NET 3.0

          • Re: SWJobSchedulerSvc crashes with .NET Errors
            JasonKV

            I just worked with Solarwinds on this exact problem last week.


            They were describing my problem as a conflict that arises when APM and NPM are installed on the same machine. They described two solutions.


             1. Upgrade APM 2.0 SP3 and NPM V9.0 SP1 and the problem shoudl be fixed is it is one they are aware of and have corrected in later updates.


            2. Contact them for a replacement file and follow their instructions on replacing the file in your system that is causing the problem, if you do not want to upgarde..


            Since we are unable to upgrade at this point we choose option 2. It was an easy copy past, and then a restart of services. So far no errors.


             If you want to use our Case number as a reference please feel free to let them know you are having a similar issue. Case#60950.


            Hope this helps.

              • Re: SWJobSchedulerSvc crashes with .NET Errors
                Ralphr

                Good Luck....I've been having this issue for over 6 months.  I am constantly told the next thing down the road will address this issue (next service pack, next release,etc) the latest was to upgrade to NPM 9 SP2 and APM 2 SP1.  Did that a few weeks ago and the JobScheduler began crashing within 5 days of the upgrade. Now I'm being told.....AGAIN....that this is a known issue that will be fixed in the next service pack.....

                  • Re: SWJobSchedulerSvc crashes with .NET Errors
                    denny.lecompte

                    Ralph,

                    Unfortunately, you have had an unusual number of problems with APM, relative to the rest of the user population.  However, to be clear, you had one problem with APM 1.0, and you now have a completely different problem with APM 2.0, even though the symptoms seems similar.

                    Your first issue was fixed with APM 2.0, and your second issue will be fixed in the next service pack.  

                      • Re: SWJobSchedulerSvc crashes with .NET Errors
                        Ralphr

                        Denny,


                        With all due respect...I have had one problem with APM and that is that the JobScheduler  frequently crashes on .NET 2.0 errors rendering APM useless in providing useful metrics as to application/service  availability. That is what was happening 6 months ago and that is what is still happening today.   The fact that SW support kept closing tickets each time we tried a "fix" as opposed to waiting a week or two forced me to open  new tickets when the problem returned.  Not alot of "problems" just alot of tickets about the same problem.  The fact that the specific lines of code that your development team has to fix today is in any way different then what they worked on previously is of little comfort to this Agency.


                        All that being said there is an awful lot to like about the SolarWinds suite and I remain cautiously optimistic that some day, hopefully very very soon, this issue will in fact be resolved.


                         


                          • Re: SWJobSchedulerSvc crashes with .NET Errors
                            denny.lecompte
                            With all due respect...I have had one problem with APM and that is that the JobScheduler  frequently crashes on .NET 2.0 errors rendering APM useless in providing useful metrics as to application/service  availability. That is what was happening 6 months ago and that is what is still happening today.   The fact that SW support kept closing tickets each time we tried a "fix" as opposed to waiting a week or two forced me to open  new tickets when the problem returned.  Not alot of "problems" just alot of tickets about the same problem.  The fact that the specific lines of code that your development team has to fix today is in any way different then what they worked on previously is of little comfort to this Agency.
                             

                             Ralph,

                            You're correct.  You have seen the same symptom, so in that sense, it's the same problem.  But the causes of these problem is v1 is different than the cause in v2.

                              • Re: SWJobSchedulerSvc crashes with .NET Errors
                                iainskwatt

                                I'm getting this message too, which looks a lot like your problem too.

                                 Event Type:    Error
                                Event Source:    .NET Runtime 2.0 Error Reporting
                                Event Category:    None
                                Event ID:    5000
                                Date:        14/10/2008
                                Time:        16:24:17
                                User:        N/A
                                Computer:    SENTINEL
                                Description:
                                EventType clr20r3, P1 swjobschedulersvc.exe, P2 1.1.0.238, P3 486c94d4, P4 solarwinds.jobengine, P5 1.1.0.238, P6 486c94d1, P7 22b, P8 0, P9 system.io.filenotfoundexception, P10 NIL.

                                For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
                                Data:
                                0000: 63 00 6c 00 72 00 32 00   c.l.r.2.
                                0008: 30 00 72 00 33 00 2c 00   0.r.3.,.
                                0010: 20 00 73 00 77 00 6a 00    .s.w.j.
                                0018: 6f 00 62 00 73 00 63 00   o.b.s.c.
                                0020: 68 00 65 00 64 00 75 00   h.e.d.u.
                                0028: 6c 00 65 00 72 00 73 00   l.e.r.s.
                                0030: 76 00 63 00 2e 00 65 00   v.c...e.
                                0038: 78 00 65 00 2c 00 20 00   x.e.,. .
                                0040: 31 00 2e 00 31 00 2e 00   1...1...
                                0048: 30 00 2e 00 32 00 33 00   0...2.3.
                                0050: 38 00 2c 00 20 00 34 00   8.,. .4.
                                0058: 38 00 36 00 63 00 39 00   8.6.c.9.
                                0060: 34 00 64 00 34 00 2c 00   4.d.4.,.
                                0068: 20 00 73 00 6f 00 6c 00    .s.o.l.
                                0070: 61 00 72 00 77 00 69 00   a.r.w.i.
                                0078: 6e 00 64 00 73 00 2e 00   n.d.s...
                                0080: 6a 00 6f 00 62 00 65 00   j.o.b.e.
                                0088: 6e 00 67 00 69 00 6e 00   n.g.i.n.
                                0090: 65 00 2c 00 20 00 31 00   e.,. .1.
                                0098: 2e 00 31 00 2e 00 30 00   ..1...0.
                                00a0: 2e 00 32 00 33 00 38 00   ..2.3.8.
                                00a8: 2c 00 20 00 34 00 38 00   ,. .4.8.
                                00b0: 36 00 63 00 39 00 34 00   6.c.9.4.
                                00b8: 64 00 31 00 2c 00 20 00   d.1.,. .
                                00c0: 32 00 32 00 62 00 2c 00   2.2.b.,.
                                00c8: 20 00 30 00 2c 00 20 00    .0.,. .
                                00d0: 73 00 79 00 73 00 74 00   s.y.s.t.
                                00d8: 65 00 6d 00 2e 00 69 00   e.m...i.
                                00e0: 6f 00 2e 00 66 00 69 00   o...f.i.
                                00e8: 6c 00 65 00 6e 00 6f 00   l.e.n.o.
                                00f0: 74 00 66 00 6f 00 75 00   t.f.o.u.
                                00f8: 6e 00 64 00 65 00 78 00   n.d.e.x.
                                0100: 63 00 65 00 70 00 74 00   c.e.p.t.
                                0108: 69 00 6f 00 6e 00 20 00   i.o.n. .
                                0110: 4e 00 49 00 4c 00 0d 00   N.I.L...
                                0118: 0a 00                     ..     

                                  • Re: SWJobSchedulerSvc crashes with .NET Errors

                                    Yup.  I am getting the same error as well.  I just responded in another thread about it.  Sounds like a common problem.

                                      • Re: SWJobSchedulerSvc crashes with .NET Errors
                                        iainskwatt

                                        Strange thing happened last night, got this new event about 23 times in the space of 20 minutes and since then I haven't been getting the above error. Very strange and I have know idea if it's anything I've done or not. I'm thinking not but I did reboot the machine a few times.

                                        Here's a copy of the new error (which has now stopped) :

                                         Event Type:    Error
                                        Event Source:    .NET Runtime Optimization Service
                                        Event Category:    None
                                        Event ID:    1101
                                        Date:        14/10/2008
                                        Time:        17:12:06
                                        User:        N/A
                                        Computer:    SENTINEL
                                        Description:
                                        .NET Runtime Optimization Service (clr_optimization_v2.0.50727_32) - Failed to compile: Microsoft.ReportingServices.QueryDesigners, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 . Error code = 0x80070002


                                        For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
                                         

                                        PS  Also noticed that now my solarwinds server is running like a dream again and I've just got my boss to buy a nice new big server to move the SQL install over ooooops.

                                          • Re: SWJobSchedulerSvc crashes with .NET Errors
                                            iainskwatt

                                             The plot thickens. Noticed that my Sophos AutoUpdater was not in the systray. Turned it back on and with in about 2 seconds of it loading I got my .NET error again! Any one else running Sophos that has this problem?

                                              • Re: SWJobSchedulerSvc crashes with .NET Errors
                                                iainskwatt

                                                 Ok more information. Completely removed Sophos but on restart got the Just-in-time debuging window backup, you have to be logged onto the machine not RDP'd to it to get this. If you leave that window on the screen error in event viewer stops. If you so no to debuging the error happens again and pops up the just-in-time debuging window again. Checked my services and the job schedular service seems to be running just fine still. I'm going to leave it like that over night and if my jobs run that's a work around!

                                                 Still would love to know what's going on though.

                                                  • Re: SWJobSchedulerSvc crashes with .NET Errors

                                                    I was finally able to get this resolved for the most part.

                                                    We did the following:

                                                    1. Upgraded to NPM 9.1 and rebooted
                                                    2. Installed NPM 9.1 SP1 and rebooted
                                                    3. Ran repair install on APM 2.0 and rebooted
                                                    4. Installed APM 2.0 SP2 and rebooted

                                                    This was done around 7 PM CST last night and the scheduler service has only died with a .NET error once since then. Previously it was crashing every 5 seconds.

                                                    Obviously only time will tell if this truly stops the issue or not. I will update in a few days and let everyone know.

                                                      • Re: SWJobSchedulerSvc crashes with .NET Errors
                                                        iainskwatt

                                                         Just a quick update from me on this one too.

                                                         I too have resolved this issue on my system. I installed SQL 2005 Enterprise on my NPM box and that's really when I started getting the problems. This was due to the fact that I had removed Microsoft SQL Server 2005 Compact Edition from add and remove programs. I stupidly thought that running the full version of SQL would allow me to tidy up the server and remove the above. However, after looking through Thwack it was a good idea to put the Compact Edition back on my machine, and bang the errors no longer happen!

                                                        I have subsequently removed SQL from the NPM server and moved it over to its own server to speed things up a bit but I still wont remove the Compact Edition just in case!

                                                        Hope that helps someone else

                                                          • Re: SWJobSchedulerSvc crashes with .NET Errors

                                                            That's strange, because I am pointing to a SQL 2005 server, but I still have SQL compact edition running on my NPM server.


                                                            Also, the errors have not completely stopped. I am now getting the following error, however, the service will only throw the error several times but not completely crash:


                                                            Event Type: Information
                                                            Event Source: .NET Runtime 2.0 Error Reporting
                                                            Event Category: None
                                                            Event ID: 5001
                                                            Date:  12/1/2008
                                                            Time:  9:27:42 AM
                                                            User:  N/A
                                                            Computer: XXXXXXXXX


                                                            Description:
                                                            Bucket 391881814, bucket table 5, EventType clr20r3, P1 swjobschedulersvc.exe, P2 1.1.1.3026, P3 48ed297a, P4 mscorlib, P5 2.0.0.0, P6 471ebc5b, P7 3404, P8 26d, P9 system.io.ioexception, P10 NIL.



                                                            Data:
                                                            0000: 42 00 75 00 63 00 6b 00   B.u.c.k.
                                                            0008: 65 00 74 00 3a 00 20 00   e.t.:. .
                                                            0010: 33 00 39 00 31 00 38 00   3.9.1.8.
                                                            0018: 38 00 31 00 38 00 31 00   8.1.8.1.
                                                            0020: 34 00 0d 00 0a 00 42 00   4.....B.
                                                            0028: 75 00 63 00 6b 00 65 00   u.c.k.e.
                                                            0030: 74 00 54 00 61 00 62 00   t.T.a.b.
                                                            0038: 6c 00 65 00 20 00 35 00   l.e. .5.
                                                            0040: 0d 00 0a 00               ....