SWJobEngineSvc2 Dying on Additional Poller

rjnicholson

This has now happened 2 times since I have upgraded to SAM 5.2.  This is only happening on my secondary poller at this time.

I see a couple of errors in my Applications Log in Windows Logs.  Has anyone else seen this since upgrading or having a similar issue?

I will open a support case on this shortly and place the Ticket Number here.

I show 2 errors logged, but they weren't until the service finally died.  I stopped polling at 12:29pm and the service finally died on the server at 12:59pm and generated 2 errors in the log.

.Net Runtime      Event ID 1026

 

Application: SWJobEngineSvc2.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: System.TimeoutException

Stack:

Server stack trace:

at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)

at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)

at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(System.Runtime.Remoting.Messaging.IMessage, System.Runtime.Remoting.Messaging.IMessage)

at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(System.Runtime.Remoting.Proxies.MessageData ByRef, Int32)

at SolarWinds.JobEngine.IJobExecutionEngine.IsTerminationRequested()

at SolarWinds.JobEngine.Engine.WorkerProcessWCFProxy.IsTerminationRequested()

at SolarWinds.JobEngine.Engine.WorkerProcess.IsTerminationRequested()

at SolarWinds.JobEngine.Engine.WorkerMonitor.ProcessWorkerProcess(System.Guid)

at SolarWinds.JobEngine.Engine.WorkerMonitor.MonitorMain(System.Object)

at System.Threading.ThreadHelper.ThreadStart_Context(System.Object)

at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

at System.Threading.ThreadHelper.ThreadStart(System.Object)

Application Error     Event ID 1000

 

Faulting application name: SWJobEngineSvc2.exe, version: 2.3.1.79, time stamp: 0x4ffc5891

Faulting module name: KERNELBASE.dll, version: 6.1.7601.17651, time stamp: 0x4e211319

Exception code: 0xe0434352

Fault offset: 0x0000b9bc

Faulting process id: 0x4a08

Faulting application start time: 0x01cda6618cba405a

Faulting application path: C:\Program Files (x86)\Common Files\SolarWinds\JobEngine.v2\SWJobEngineSvc2.exe

Faulting module path: C:\Windows\syswow64\KERNELBASE.dll

Report Id: 4685929f-17bb-11e2-9107-e4115bb9e4ec

I was able to restart the service from the Orion Service Manager.  My poller is only at 48% of it's total weight and monitoring 5800 total Elements.

aLTeReGo

I recommend you open a case with support. If you'd rather, you can first try and repair the job engine yourself by following the steps outlined below. This good first troubleshooting step that support may ask you to perform anyway.

Please run the repairs below:

How to Replace Job Engine

1. Stop all Orion services from Orion Service Manager.

2. Go to C:\Documents and Settings\All Users\Application Data\SolarWinds\JobEngineV2\Data\JobEngine35.sdf file and rename it to JobEngine35_OLD.sdf

3. Then create a copy of C:\Documents and Settings\All Users\Application Data\SolarWinds\JobEngineV2\Data\JobEngine35 - Blank.sdf one and rename it to JobEngine35.sdf.

4. This way you have the old one to revert back to and you always have a blank copy in case issue reoccurs.

5. Right click on properties of the JobEngine35.sdf you have created and untick the read only box and click ok.

How to Replace Job Tracker

1. Stop all Orion services from Orion Service Manager.

2. Go to C:\Documents and Settings\All Users\Application Data\SolarWinds\Collector\Data\JobTracker.sdf file and rename it to JobTracker_OLD.sdf

3. Then create a copy of C:\Documents and Settings\All Users\Application Data\SolarWinds\Collector\Data\JobTracker - Blank.sdf one and rename it to JobTracker.sdf.

4. This way you have the old one to revert back to and you always have a blank copy in case issue reoccurs.

5. Right click on properties of the JobTracker.sdf you have created and untick the read only box and click ok.

How to Replace Polling Collector

1. Stop all Orion services from Orion Service Manager.

2. Go to C:\Documents and Settings\All Users\Application Data\SolarWinds\Collector\Data\PollingController.sdf file and rename it to PollingController_OLD.sdf

3. Then create a copy of C:\Documents and Settings\All Users\Application Data\SolarWinds\Collector\Data\PollingController - Blank.sdf one and rename it to PollingController.sdf.

4. This way you have the old one to revert back to and you always have a blank copy in case issue reoccurs.

5. Right click on properties of the PollingController.sdf you have created and untick the read only box and click ok.

6. Restart all Orion Services

How to Repair Job Engine

- Open C:\Documents and Settings\All Users\Application Data\SolarWinds\Installers.

- On a 2008 system, this will be under C:\ProgramData\Solarwinds\Installers.

- Run the Jobengine.msi and select Repair.

How to Repair Job Engine V2

- Open C:\Documents and Settings\All Users\Application Data\SolarWinds\Installers.

- On a 2008 system, this will be under C:\ProgramData\Solarwinds\Installers.

- Run the Jobengine.v2.msi and select Repair.

How to Repair Information Service

- Open C:\Documents and Settings\All Users\Application Data\SolarWinds\Installers.

- On a 2008 system, this will be under C:\ProgramData\Solarwinds\Installers.

- Run the InformationService.msi and select Repair.

How to Repair Collector Service

- Open C:\Documents and Settings\All Users\Application Data\SolarWinds\Installers.

- On a 2008 system, this will be under C:\ProgramData\Solarwinds\Installers.

- Run the CollectorInstaller.msi and select Repair.