22 Replies Latest reply: May 25, 2011 1:50 PM by GoldTipu RSS

Socket failures on HTTP SLA operation??

mmunford

I just upgraded my VoIP Monitor to the new IP SLA module..

I'm setting up a few HTTP SLA operations as an initial test of the new features.. I've added 4 HTTP operations to one of my SLA endpoints, and one in particular shows down and also produces the following "status message" in the "Operation details" view:

Socket failures or other errors not relevant to the actual probe.

All 4 of these HTTP operations are against LOB web applications hosted in our European Home office. I can of course browse to all of these web server applications just fine..

Anybody run across this yet?? Any ideas??

 
  • Re: Socket failures on HTTP SLA operation??
    chris.smouse

    Hi mmunford -

    Can you verify that the IPSLA source node (the node running the IPSLA operation) can access the web servers hosting your applications?

    Just because your browser can access the web sites doesn't necessarily mean the node actually running the IPSLA operation is able to reach the same web sites.

    Hope this helps.

    • Re: Socket failures on HTTP SLA operation??
      mmunford

      I have verified that I can telnet (from the SLA router) to port 80 on the target server with similar response as the other HTTP operations targets produce..

      • Re: Socket failures on HTTP SLA operation??
        chris.smouse

        The error you're seeing is tied to an error response from the router itself.

        Here is a link to Cisco's instructions for debugging the HTTP operation.  You'll want to use the "debug" router commands to determine the cause of the router not being able to execute the HTTP operation.  Is the device low on resources? (memory/CPU)?

        http://www.cisco.com/en/US/docs/ios/12_4/ip_sla/configuration/guide/hshttp.html

        Hope this helps.

        • Re: Socket failures on HTTP SLA operation??
          mmunford

          The Router is definitely not low on CPU/memory resources.. avg of 10%CPU and 25% Mem Util..

           

          cranking up some HTTP debugs now..

           

          I have added a few more operations and a couple more produced the same error. One more HTTP operation and a DHCP operation...

          • Re: Socket failures on HTTP SLA operation??
            derhally

            Do you get the same error if you create the http operation manually?

            What do you see when you use the "sh ip sla statistics <operation number>" command?

            What IOS version are you using?

            • Re: Socket failures on HTTP SLA operation??

              Hi Guys,

              I have the same issue if i use dns names in the url

              http://www.google.com/

              But if i use the IP address for the HTTP IP SLA it works fine

              Is this a know issue and also when we create a new IP SLA operation it is transferred to the switch the opertaion is created but the config is not saved in the switch (wr mem)

              Can you please clarify?

              regards

              Chandru

              • Re: Socket failures on HTTP SLA operation??
                chris.smouse

                Chandru -

                Perhaps your IPSLA source node cannot resolve the DNS name?  Can you successfully ping or traceroute the DNS name from the switch itself?

                • Re: Socket failures on HTTP SLA operation??
                  mmunford

                  Hi,

                  I hadn't had time to dig into the Router Debugs as I had promised earlier.. I just circled back to this this morning after the other thread discussion and have this to report:

                  In my case, the DNS resolution works fine to the HTTP sites that produce the failure. Changing the operation to use the IP address instead of the FQDN did not make a difference. Here's my new info to add here. It seems if I browse to the URL using a browser and using the FQDN, I can see the site. Browsing using the IP address alone produces an HTTP error message (connection reset in my case). I am assuming from this finding that the IPSLA operation is not passing the URL FQDN as part of the HTTP request. Rather, it seems that perhaps it is resolving the IP via DNS and then constructing a HTTP request using the resolved IP?? I have not done any captures to confirm this yet..

                   

                  I also did a spot check comparison of the Running and Startup configs on the Router hosting the IPSLA operations and found them to be inconsistent as well..

                • Re: Socket failures on HTTP SLA operation??

                  Chris,

                  Yes i am able to ping and traceroute without any problems

                  regards

                  Chandru

                • Re: Socket failures on HTTP SLA operation??
                  GoldTipu

                  Perfect ! Thanks

                  That's isolated the issue and i think correct answer ... 

            • Re: Socket failures on HTTP SLA operation??
              mmunford

              I missed your  post Derhally.. sorry bout that..

               

              This router has IOS 12.3 (11) T11 code.. not the newest thing around, definitely. It's using the "RTR" syntax instead of "IP SLA" as well.. 

               

              Perhaps I'll try setting up a few of these operations on a router with newer IOS and post the results..

              • Re: Socket failures on HTTP SLA operation??
                mmunford

                The "debug RTR Trace <operation>" produces this output:

                 

                .Nov  2 15:48:09.429 est: SAA(40019) Scheduler: Starting an operation
                .Nov  2 15:48:09.429 est: SAA(40019) http operation: Starting http operation
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: Starting dns operation
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: Query name - laradirect.cma-cgm.com
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: Query name server - 10.150.8.4
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: Query return code - no error
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: received IP Address 10.0.101.1
                .Nov  2 15:48:09.429 est: SAA(40019) dns operation: RTT=1
                NOR-VGW-1#
                .Nov  2 15:48:09.533 est: SAA(40019) http operation: Sent 18 of 18 bytes
                .Nov  2 15:48:09.533 est: SAA(40019) http operation: Wait connection - connected
                NOR-VGW-1#
                .Nov  2 15:48:11.473 est: SAA(40019) Scheduler: Updating result

                • Re: Socket failures on HTTP SLA operation??
                  mmunford

                  The Target server in question is requiring Host Header requests and is dropping non-conforming connections.

                  I think perhaps I would need to setup a manual IP SLA HTTP operation and use the HTTP RAW option to specify a HTTP1.1 query that contains the HOST field option.

                  I haven't verified this yet...

                  • Re: Socket failures on HTTP SLA operation??
                    mmunford

                    I tried to manually add a custom IPSLA operation that specified a HTTP RAW request to the target server, but I'm not sure I had the correct syntax.. I'm going to attempt it again soon. 

                    This logically leads to my next question: If I can manually contruct the appropriate HTTP RAW query, What needs to happen to get ORION to pickup the manual operation for display?? perhaps I missed this in the Docs??

                    • Re: Socket failures on HTTP SLA operation??
                      chris.smouse

                      mmunford -

                      Once you have created the IPSLA operation manually on the router, make a note of the operation number.

                      Then, go through the Add Operations wizard in IPSLA Manager.  Instead of creating a new operation, you need to choose the "monitor existing" option.

                      Toward the end of the wizard, you will have the opportunity to enter the operation number for the custom operation.

                      Hope this helps,

                      Chris

                      • Re: Socket failures on HTTP SLA operation??
                        mmunford

                        DOH!! I did blow right by that in the docs!! I thought "existing operations" was referring to operations already exisitng in IPSLA MAnager, not the Router itself..

                      • Re: Socket failures on HTTP SLA operation??
                        mmunford

                        I have made some progress, but am still not quite there yet...

                        I was able to manually create an IPSLA HTTP RAW request operation that does not produce the socket error and appears to generate a valid HTTP page download from the target.

                        This Syntax seems to correctly satisfy the server's Host Header requirement (notice the placement of multiple \r\n codes):

                        rtr 40007

                         type http operation raw url http://target.webserver.com

                         http-raw-request

                         GET / HTTP/1.1\r\nHOST: target.webserver.com\r\n\r\n

                         exit

                         exit

                        rtr schedule 40007 life forever start-time now ageout 3600

                         

                        And a Packet capture confirms the operation seems to be working:

                         

                        GET / HTTP/1.1
                        HOST: target.webserver.com


                        HTTP/1.1 200 OK
                        Set-Cookie: 1113915402.25886.0000; path=/
                        Date: Tue, 17 Nov 2009 22:08:16 GMT
                        Server: Oracle-Application-Server-10g/10.1.2.2.0 Oracle-HTTP-Server
                        Last-Modified: Tue, 29 Sep 2009 13:52:40 GMT
                        ETag: "31db-2791-4ac21128"
                        Accept-Ranges: bytes
                        Content-Length: 10129
                        Content-Type: text/html

                        <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
                        "http://www.w3.org/TR/html4/loose.dtd">
                        <html>
                        <head>
                        <title>Target Home Page</title>
                        <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
                        <script type="text/javascript" src="LHP_overlib.js"><!-- overLIB (c) Erik Bosrup --></script>
                        <link href="LHP_homepage.css" rel="stylesheet" type="text/css">
                        </head>
                        ---->Additional Output Truncated<----

                         

                        I have added the manual operation to IPSLA using the operation number as outlined earlier in this thread (and of course the Docs, if you pay attention!).

                        Now the IPSLA operation details show a status of "Operation timed out." with RTT, DNS RTT and Tranaction RTT all at 0ms, and TCP connect RTT at 100ms.

                         

                         

                        I've cranked up the RTT thresholds just for kicks, with no difference in status..