8 Replies Latest reply on Apr 6, 2010 4:28 PM by JustBen

    Display targeted URL in NetFlow Reports?

      Is there a way via a custom report, or some other method, to display the URL/host header the host is trying to access vs. the IP address that is resolved?

      Like www.youtube.com vs. 208.117.254.167

      It would be very handy to be able to group traffic to/from a specific URL rather than have it be broken out by IP.

      Thanks!

        • Re: Display targeted URL in NetFlow Reports?
          aLTeReGo

          Someone correct me if I'm wrong, but it's my understanding that information URL/IRI/Host Header information is not contained in the NetFlow data stream. Therefore it would be impossible for NTA to report upon it.

            • Re: Display targeted URL in NetFlow Reports?

              Bummer.

              Thanks

                • Re: Display targeted URL in NetFlow Reports?
                  Andy McBride

                  Check you DNS resolution settings. If you set it to On Demand you can resolve the domain.

                    • Re: Display targeted URL in NetFlow Reports?

                      Thanks for the input..

                      While that is helpful for the example I used, (and I have adjusted my settings accordingly, thanks) I was hoping to pinpoint the traffic that goes to/from hosting companies like Akamai, 1e100.net, Voxel.net, etc as well..

                      Has anyone come up with a solution for this?

                      • Re: Display targeted URL in NetFlow Reports?
                        jswan

                        Just to be absolutely clear: there's a big difference between reverse DNS resolution and host header extraction. Reverse DNS resolution gives you the PTR record for the associated IP address (if there is one), but that may or may not have anything to do with the website hosted at that IP address. Commercial hosting providers might have dozens or even hundreds of websites hosted at a single IP address. The same goes for content distribution networks like Akamai, Time Warner, LLNW, etc--you have no way of knowing based on IP address what content is hosted there.

                        As Alterego states above, NetFlow contains no information about HTTP headers. This is not a limitation of the product; it's just not something NetFlow was ever designed to do.

                        If you want to see HTTP headers you either need something to analyze the raw packet stream (there are tons of ways to do this), or you need a proxy server that logs all the outbound requests.

                          • Re: Display targeted URL in NetFlow Reports?

                            Thanks for the clarification. I do understand the difference between the host header and the reverse lookup, but the reverse lookup, in some cases, gives me what I'm after..

                            I now understand this is not a limitation of the Orion product, I was just hoping to some how collect the information and display it with the rest of the NetFlow data.

                            Knowing host A downloaded xxxx mb from a specific URL would be very handy since so much traffic goes over HTTP these days.

                            I know I can get it with other solutions, but having that data tied to the rest of my network performance data would be even more awesomer..

                              • Re: Display targeted URL in NetFlow Reports?
                                jswan

                                I think that eventually we might have products that can do this: NetFlow v9 has the ability to export raw packet contents based on an offset value in the IP header. I don't see any reason that a developer couldn't use this to facilitate URL monitoring, but I imagine that there will be quite a lot of overhead associated with doing so.

                                  • Re: Display targeted URL in NetFlow Reports?

                                    That would be an awesome feature.


                                    Even if there was a small trade off for the overhead like only capturing the data if a certain interface utilization threshold is met, during scheduled times or something like that, it would be very worthwhile.

                                    You guys seem to be pretty sharp, I bet you can come up with a way to make that happen. :)

                                    Thanks for all the feedback!