5 Replies Latest reply on Oct 16, 2018 9:46 AM by chadsikorra

    July .NET Patches and SolarWinds/SolarWinds Agents

    colew

      After installing, and then uninstalling the July Microsoft patches around .NET Framework, we have been dealing with some serious instability in our environment.  If you aren't familiar with the patches, they're documented here:

      Advisory on July 2018 .NET Framework Updates · Issue #74 · dotnet/announcements · GitHub

       

      Microsoft released these, we installed, they pulled them and then released another one to fix the issues that were found, but then said they did not think that it fixed everything on the 2008 R2 servers (we have two in our environment - one being the core Orion server, along with 9 2012 R2 servers).  We have since uninstalled all of the patches from our environment, but still experience the issues.

       

      The issues we are seeing is that the businesslayerhost process is crashing very often on our pollers, and we have a ton of apps (mostly the ones that monitor on our agent-based machines) going into an unknown state continuously throughout the day - about 1,000 out of the 8,000 total.  The event log errors we are seeing are at the bottom of this email.  My question is are you guys aware of these patches causing instability with SolarWinds?  What about on the agent side?  I know the agent relies on .NET framework, as it installs it during the installation process if it isn't already there.  With the way that we are seeing the issues on our pollers, it almost makes me think that we are having issues communicating with the agents, thus causing the unknown app numbers to bounce around all day as the pollers are having trouble getting the data in time.  I believe all of our agent-managed machines still have these patches, even though they are all 2012 R2 and up.

       

      For reference, here is the version(s) we are at:

       

      Errors:

      Application: SolarWinds.BusinessLayerHost.exe

      Framework Version: v4.0.30319

      Description: The process was terminated due to an unhandled exception.

      Exception Info: System.InvalidOperationException

         at SolarWinds.BusinessLayerHost.BusinessLayerHostService+<>c__DisplayClass25_0.<CheckPlugins>b__0(System.Object)

         at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)

         at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

         at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)

         at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()

         at System.Threading.ThreadPoolWorkQueue.Dispatch()

         at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

       

      Faulting application name: SolarWinds.BusinessLayerHost.exe, version: 2017.1.5300.1698, time stamp: 0x58ac4615

      Faulting module name: KERNELBASE.dll, version: 6.3.9600.18938, time stamp: 0x5a7dd8a7

      Exception code: 0xe0434352

      Fault offset: 0x00015ef8

      Faulting process id: 0x1d0c

      Faulting application start time: 0x01d42e91b9959d28

      Faulting application path: C:\Program Files (x86)\SolarWinds\Orion\SolarWinds.BusinessLayerHost.exe

      Faulting module path: C:\WINDOWS\SYSTEM32\KERNELBASE.dll

      Report Id: 00ec59ef-9a86-11e8-80fd-e4115bafdd78

      Faulting package full name:

      Faulting package-relative application ID: