8 Replies Latest reply on Jul 6, 2016 9:03 AM by aaswi

    SolarWinds.BusinessLayerHost.exe terminated due to an unhandled exception: System.IO.FileNotFoundException

    agarciadkp

      We initially attempted to resolve an issue where the poll date (and next poll date) for a particular node was in the past by following the instructions provided in the following article: Last Database Update is in the past, Next Poll... More past'er'er . Upon restarting the Orion services, we found that that, while the affected nodes were being polled, we were unable to access the web console with the following error:

       

      There was no endpoint listening at net.tcp://xxxxxx:17777/orion/core/businesslayer that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.

       

      Upon reviewing the Event Viewer, we're seeing the following error repeatedly (the service restarts every minute or so):

       

      Application: SolarWinds.BusinessLayerHost.exe

      Framework Version: v4.0.30319

      Description: The process was terminated due to an unhandled exception.

      Exception Info: System.IO.FileNotFoundException

      Stack:

        at SolarWinds.AgentManagement.Messaging.Core.MessagingService.Dispose(Boolean)

        at SolarWinds.AgentManagement.Messaging.Core.MessagingService.Finalize()

       

       

      Reviewing logs in ProgramData\Solarwinds\Logs\Orion indicates that a few assemblies aren't being found:

       

      /Orion/BusinessLayerHost-SOLARWINDS.AGENTMANAGEMENT.SERVICECORE.DLL.log:

      [2] ERROR SolarWinds.BusinessLayerHost.BusinessLayerHostService - Unhandled exception caught by business layer host. System.IO.FileNotFoundException: Could not load file or assembly 'SolarWinds.AMS.Proxy.dll' or one of its dependencies. The specified module could not be found. File name: 'SolarWinds.AMS.Proxy.dll'

        at SolarWinds.AgentManagement.Messaging.Core.MessagingService.Dispose(Boolean)

        at SolarWinds.AgentManagement.Messaging.Core.MessagingService.Finalize()

       

      /AgentManagement/AgentManagement.Service.log:

      [6] ERROR SolarWinds.AgentManagement.ServiceCore.AgentManagementServiceApp - <log4net.Error>Exception during StringFormat: Could not load file or assembly 'SolarWinds.AMS.Proxy.dll' or one of its dependencies. The specified module could not be found. <format>Error executing start operation '{0}'. Retrying in 5 seconds. {1}</format><args>{InitMessagingService, <Exception: Could not load file or assembly 'SolarWinds.AMS.Proxy.dll' or one of its dependencies. The specified module could not be found.>}</args></log4net.Error>

       

      /Orion/BusinessLayerHost.log:

      [7] INFO  SolarWinds.BusinessLayerHost.PluginInstanceAppDomain - Starting plugin: "DPI Business Layer"

      [7] ERROR SolarWinds.BusinessLayerHost.PluginInstanceAppDomain - Plugin "DPI Business Layer" failed to start.

      System.Runtime.Serialization.SerializationException: Unable to find assembly 'SolarWinds.DPI.BusinessLayer, Version=2.0.5.583, Culture=neutral, PublicKeyToken=null'.

      Server stack trace:

         at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()

         at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)

         at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)

         at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)

         at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryHeaderEnum binaryHeaderEnum)

         at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()

         at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)

         at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)

         at System.Runtime.Remoting.Channels.CrossAppDomainSerializer.DeserializeObject(MemoryStream stm)

         at System.Runtime.Remoting.Messaging.SmuggledMethodReturnMessage.FixupForNewAppDomain()

       

      /Orion/Core.BusinessLayer.log:

      [7] ERROR SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.PluginsFactory`1 - System.AggregateException: inspecting [SolarWinds.Data.Providers.Orion.Plugins, Version=2015.1.25300.8197, Culture=neutral, PublicKeyToken=null] requested by [SolarWinds.Data.Providers.APM.Plugin, Version=6.2.2.2825, Culture=neutral, PublicKeyToken=null] at 'C:\Program Files (x86)\SolarWinds\Orion\Information Service\2.0\Plugins\SolarWinds.Data.Providers.APM.Plugin.dll'. ---> System.IO.FileNotFoundException: Could not load file or assembly 'SolarWinds.Data.Providers.Orion.Plugins, Version=2015.1.25300.8197, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

         at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)

         at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)

         at System.Reflection.Assembly.ReflectionOnlyLoad(String assemblyString)

         at SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.PluginsFactory`1.CurrentDomain_ReflectionOnlyAssemblyResolve(Object sender, ResolveEventArgs args)

         --- End of inner exception stack trace ---

      ---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'SolarWinds.Data.Providers.Orion.Plugins, Version=2015.1.25300.8197, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

      File name: 'SolarWinds.Data.Providers.Orion.Plugins, Version=2015.1.25300.8197, Culture=neutral, PublicKeyToken=null'

         at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

         at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)

         at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)

         at System.Reflection.Assembly.ReflectionOnlyLoad(String assemblyString)

         at SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.PluginsFactory`1.CurrentDomain_ReflectionOnlyAssemblyResolve(Object sender, ResolveEventArgs args)

       

      We've confirmed that the affected assemblies are present on the server:

       

      Program Files (x86)/SolarWinds/Orion/

        AgentManagement/SolarWinds.AMS.Proxy.dll

        Information Service/3.0/SolarWinds.Data.Providers.Orion.Plugins.dll

        SolarWinds.DPI.BusinessLayer.dll

       

      We opted to install SolarWinds Orion Platform 2015.1.2 Hotfix 5 and there is no change in the issue. We've also applied the fix described at BusinessLayerHost crashes - SolarWinds Worldwide, LLC. Help and Support .

       

      We currently have a ticket in with SolarWinds support. Out of curiosity, has anyone else here encountered this issue?

       

      Thanks,

      Alex