So we have a windows 2012 R2 server whereby we can't monitor SSL Certificate Expiration Date & part of the IIS monitors, however the server in question can monitor other components namely a CPU perfmon counter and the standard Certificate monitor which is a powershell monitor plus the usual node details all work okay. Other 2012R2 servers work just fine so I know it's not a 2012 related issue. We run an agentless model.
The 2012 servers in our environment are montored via SNMP due to WMI issues and W2012 high CPU issues whereby WMI grabs all available CPU and kills the machine.
On the IIS monitor everything appears to be monitoring okay bar the actual website! The website is bound to 443 with a certificate and can be browsed via the link SW has discovered.
On the Certificate Expiration Date Monitor front this is the error we've got.
Debug log from the application
2023-02-24 09:09:20,146 [152] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Common.Retry.RetryPolicy - Retry action will be executed, max count of executions: 5, delay: 00:00:01
2023-02-24 09:09:25,177 [STP SmartThreadPool Thread #68] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Logging.ApmJobContextInfo - Resetting thread log data
2023-02-24 09:09:20,130 [STP SmartThreadPool Thread #68] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test starting - ComponentId: 117110; NodeId: 2264; NodeName: *HOSTname*; ApplicationId: 1040; ComponentName: SSL Certificate Expiration Date Monitor; ComponentType: SolarWinds.APM.Probes.SslExpirationDateProbe, SolarWinds.APM.Probes, Version=2020.2.6.50231, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: 4948fdba-a1e3-4cd8-b01d-e7af284303b7;
2023-02-24 09:09:20,130 [STP SmartThreadPool Thread #68] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.MonitorJob - Current process: SWJobEngineWorker2x64 (PID 24216)
2023-02-24 09:09:20,130 [STP SmartThreadPool Thread #68] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.MonitorJob - Using new credentials
2023-02-24 09:09:20,130 [152] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.ProbeBase`1 - Remaining Time: 291 sec.
2023-02-24 09:09:20,146 [152] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDateProbe - Started SslExpirationDateProbe. Target: 443, Port: *theIP*
2023-02-24 09:09:20,146 [152] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Trying to use authentication protocols configuration:
2023-02-24 09:09:20,146 [41] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Getting SSL stream for host *theIP*:443
2023-02-24 09:09:20,146 [138] [C117110] [WU:(null)] WARN SolarWinds.APM.Probes.SslExpirationDateProbe - Attempt 1 of execution of RemoteSslCertificateClient.Authenticate() failed with error.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Common.Utility.RemoteSslCertificateClient.<AuthenticateAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler.<HandleRequestAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
2023-02-24 09:09:21,162 [57] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Trying to use authentication protocols configuration:
2023-02-24 09:09:21,162 [37] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Getting SSL stream for host *theIP*:443
2023-02-24 09:09:21,162 [127] [C117110] [WU:(null)] WARN SolarWinds.APM.Probes.SslExpirationDateProbe - Attempt 2 of execution of RemoteSslCertificateClient.Authenticate() failed with error.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Common.Utility.RemoteSslCertificateClient.<AuthenticateAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler.<HandleRequestAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
2023-02-24 09:09:22,162 [128] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Trying to use authentication protocols configuration:
2023-02-24 09:09:22,162 [25] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Getting SSL stream for host *theIP*:443
2023-02-24 09:09:22,162 [127] [C117110] [WU:(null)] WARN SolarWinds.APM.Probes.SslExpirationDateProbe - Attempt 3 of execution of RemoteSslCertificateClient.Authenticate() failed with error.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Common.Utility.RemoteSslCertificateClient.<AuthenticateAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler.<HandleRequestAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
2023-02-24 09:09:23,162 [57] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Trying to use authentication protocols configuration:
2023-02-24 09:09:23,162 [25] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Getting SSL stream for host *theIP*:443
2023-02-24 09:09:23,162 [127] [C117110] [WU:(null)] WARN SolarWinds.APM.Probes.SslExpirationDateProbe - Attempt 4 of execution of RemoteSslCertificateClient.Authenticate() failed with error.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Common.Utility.RemoteSslCertificateClient.<AuthenticateAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler.<HandleRequestAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
2023-02-24 09:09:24,177 [40] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Trying to use authentication protocols configuration:
2023-02-24 09:09:24,177 [127] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler - Getting SSL stream for host *theIP*:443
2023-02-24 09:09:24,177 [37] [C117110] [WU:(null)] WARN SolarWinds.APM.Probes.SslExpirationDateProbe - Attempt 5 of execution of RemoteSslCertificateClient.Authenticate() failed with error.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Common.Utility.RemoteSslCertificateClient.<AuthenticateAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDate.ConnectionHandler.<HandleRequestAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
2023-02-24 09:09:25,177 [27] [C117110] [WU:(null)] ERROR SolarWinds.APM.Probes.SslExpirationDateProbe - Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
--- End of inner exception stack trace ---
at SolarWinds.APM.Common.Retry.RetryPolicy.RetryPolicyImpl.<ExecuteAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolarWinds.APM.Probes.SslExpirationDateProbe.<ProbeInternal>d__9.MoveNext()
2023-02-24 09:09:25,177 [27] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDateProbe - Outcome - Fail: SSL Stream or Certificate is not available.
2023-02-24 09:09:25,177 [STP SmartThreadPool Thread #68] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.MonitorJob - Monitor test finished - ComponentId: 117110; NodeId: 2264; NodeName: *HOSTname*; ApplicationId: 1040; ComponentName: SSL Certificate Expiration Date Monitor; ComponentType: SolarWinds.APM.Probes.SslExpirationDateProbe, SolarWinds.APM.Probes, Version=2020.2.6.50231, Culture=neutral, PublicKeyToken=null; CustomLogEnabled: True; TestSessionId: 4948fdba-a1e3-4cd8-b01d-e7af284303b7;
So 2023-02-24 09:09:25,177 [27] [C117110] [WU:(null)] DEBUG SolarWinds.APM.Probes.SslExpirationDateProbe - Outcome - Fail: SSL Stream or Certificate is not available. appears to be the issue - however a google reveals nothing around this.
So the server has been rebooted, I've also rebuilt the performance counters, TLS1.1 and 1.2 are enabled on this server.
The server does not have a firewall enabled nor does it sit behind a hardware f/w
Anyone got any thoughts around this please?
