This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.

You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

HTTPS Monitor

rob.simms over 7 years ago

We have several HTTPS monitors that are set to ignore Certs. They are working 99% of the time but seem to fail with the following error from time to time:

2016-08-26 09:30:33,960 [STP Pool:1324 Thread #0] [C12201] DEBUG SolarWinds.APM.Probes.HttpClientHelper - #1 GetWebResponse failed. The request was aborted: Could not create SSL/TLS secure channel.

2016-08-26 09:30:33,960 [STP Pool:1324 Thread #0] [C12201] ERROR SolarWinds.APM.Probes.HTTP.HttpProbeBase`1 - WebException caught.

System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.

at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)

at SolarWinds.APM.Probes.HttpClientHelper.GetWebResponse(WebRequest request)

at SolarWinds.APM.Probes.HttpClientHelper.GetWebResponseFromUri(Uri address, Stopwatch stopwatch)

at SolarWinds.APM.Probes.HttpClientHelper.Download(Uri address, Stopwatch stopwatch, Encoding encoding)

at SolarWinds.APM.Probes.HTTP.HttpProbeBase`1.ProbeInternal(ProbeInformation probeInfo, HttpMonitorResult result)

We also have some custom HTTP monitors writtin in powershell as we are sending some XML in the request.

This is the request:

$results = Invoke-WebRequest -Method Post -Uri http://"$hostname":"$port"/xxx/xxx/ -ContentType applicaiton/soap+xml -UseBasicParsing -Body $body

This is the error message:

Invoke-WebRequest : The operation has timed out

At line:19 char:12

+ $results = Invoke-WebRequest -Method Post -Uri http://"$hostname":"$port"/cgi-bi ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException

+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

I know the sites are not going down as we have a second system monitoring our HTTP(s) sites at present so it looks like a solarwinds error. I was thinking maybe running out of Ports on the Orion Poller, but we increased these and it didn't help.

Any ideas? Our clients are not liking the constant false positives - over 150 Web checks and they each fail 1/2 times a day.

Thanks

Top Replies

0 m-milligan over 7 years ago

We've had a similar problem since upgrading to SAM 6.2.4. Solarwinds support says this is a known issue and that it's fixed in SAM 6.3.
Cancel
Vote Up 0 Vote Down

Cancel
0 muckman over 7 years ago in reply to m-milligan

I've had this problem for the past year well before SAM 6.2.4. Periodically I'm receiving the same SSL error as you but its only from our load balanced URLs, not the same websites as monitored on the hosting servers. It sounds like your HHTPS monitors target specific servers, is that true? Support's only advice to me was to enable Retry on the HttpClientHelper which did not resolve my issue as I will get several polls in a row with this error. And essentially that is the same as a retry within a single poll anyways. Below are the settings for the HttpClientHelper if anyone cares to test it.
Open "c:\Program Files (x86)\SolarWinds\Orion\APM\SolarWinds.APM.Probes.dll.config" file for edit
Find HttpClientHelperSettings element
Add RetryCount="2" attribute so element will look following:
<HttpClientHelperSettings Timeout="10000" RetryCount="2">
Save file
Restart "SolarWinds Job Engine v2" service
Eventually I filtered this specific error condition out of the alert because I got tired of trying to explain and defend the tool to all my engineers. Like you I also setup paralell monitoring of the websites with alternate tools including WPM and did not find the same SSL error.
I found this article and I don't think it applies as its specific to SAM 6.2.4 and my problem did not start with 6.2.4 but Im going to give it a try anyways.
Unexpected error occurred. The request was aborted: Could not create SSL/TLS secure channel - SolarWinds Worldwide, LLC.…
Cancel
Vote Up +2 Vote Down

Cancel
0 m-milligan over 7 years ago in reply to muckman

Our situation is a bit different, but not very different. Since the upgrade, the HTTPS monitors will be testing just fine, fail once with the TLS/SSL error, and then go back to testing OK. I ended up (1) modifying the app monitor so it tests more frequently and (2) modifying the alert definition so it waits longer before notifying anyone that a site is down. Between these two changes, it's much less likely that a single bad test will trigger a "down" alert.
Thanks for posting your changes to HttpClientHelper. If the upgrade doesn't fix our problem, I will try that.
Cancel
Vote Up 0 Vote Down

Cancel
0 sans over 7 years ago in reply to muckman

Did this work of you?
Cancel
Vote Up 0 Vote Down

Cancel
0 stevenstadel over 7 years ago

Did this get fixed in SAM 6.3?
Cancel
Vote Up 0 Vote Down

Cancel
0 muckman over 7 years ago in reply to stevenstadel

For me it did. But SAM 6.3 has a new set of HTTPS/SOAP bugs. Make sure you install HF2 right away as the base 6.3 is buggy and HF1 is a land mine.
Cancel
Vote Up +2 Vote Down

Cancel
0 beastroy over 6 years ago

FYI... This error still exists in SAM 6.4. We are periodically receiving the error message "Unexpected error occurred. The request was aborted: Could not create SSL/TLS secure channel" on a few HTTPS Monitors. The issue appears to be fixed in SAM 6.4 HF1, which was released on April 18th, 2017.
Below are the hotfix details for SAM 6.4 HF1:
SolarWinds Server & Application Monitor 6.4 Hotfix #1 (SolarWinds-SAM-v6.4-Hotfix1.exe)
Requirements
------------
- This hotfix requires SolarWinds Orion Core 2017.1 and SAM 6.4.
- This hotfix must be installed on the main poller, and on any additional pollers.
- This hotfix addresses the described issues for SAM 6.4.
Fixed Issues
------------
HTTPS monitor error
This hotfix addresses an issue where the SOAP monitor fails because it cannot negotiate TLS 1.0 connection.
This can be identified by the following error message visible in the SAM logs:
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: The handshake failed due to an unexpected packet format.
Or
System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
You may receive this error after installing SAM 6.3 HF1, SAM 6.3 HF2, or SAM 6.4.
SOAP monitor error
This hotfix addresses an issue where the SOAP monitor fails because it cannot negotiate TLS 1.0 connection.
This can be identified by the following error message visible in the SAM logs:
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: The handshake failed due to an unexpected packet format.
Or
System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
You may receive this error after installing SAM 6.3 HF1, SAM 6.3 HF2, or SAM 6.4.
Cancel
Vote Up +1 Vote Down

Cancel
0 t0ta11ed74 over 6 years ago in reply to beastroy

Which log can this be seen in?
Cancel
Vote Up 0 Vote Down

Cancel
0 beastroy over 6 years ago in reply to t0ta11ed74

Enable debug logging on the application monitor with the HTTPS Monitor. Once you've enabled logging, let the test run a few times (you can also test the component if you do not want to wait for the polling cycles). After the test has run, note the Application Monitor ID and log into the polling engine for the node that the application monitor is assigned to. For example if Application Monitor "X" is applied to Node "Y" using polling engine "Z", then log into polling engine "Z".
Once on the polling engine, browse to "C:\ProgramData\Solarwinds\Logs\APM\ApplicationLogs\", then locate the folder with the application monitor ID. You can find the log files in this folder that will contain more detailed logging, including this error message if it is occuring.
Hope this helps!
Cancel
Vote Up 0 Vote Down

Cancel
0 esojka over 6 years ago in reply to beastroy

Sorry to jump in on an older thread, but is anyone else not getting anything useful out of these debug logs? I had the same situation, enabled debug logging for that app monitor, and the log doesn't shed any additional light...
2018-01-10 08:49:32,475 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HTTP.HttpProbeBase`1 - Getting https://url.example.com
2018-01-10 08:49:32,475 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HTTP.HttpProbeBase`1 - Starting to measure response time.
2018-01-10 08:49:32,475 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HttpClientHelper - #1 Creating WebRequest. Stopping ResponseTime measurement.
2018-01-10 08:49:32,490 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HttpClientHelper - WebRequest created. Getting the response. Starting ResponseTime measurement.
2018-01-10 08:49:32,490 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HttpClientHelper - Using asynchronous call.
2018-01-10 08:49:32,490 [STP Pool:402 Thread #0] [C45985] DEBUG SolarWinds.APM.Probes.HttpClientHelper - #1 GetWebResponse failed. The request was aborted: Could not create SSL/TLS secure channel.
2018-01-10 08:49:32,490 [STP Pool:402 Thread #0] [C45985] ERROR SolarWinds.APM.Probes.HTTP.HttpProbeBase`1 - WebException caught.
Cancel
Vote Up 0 Vote Down

Cancel