6 Replies Latest reply on Oct 23, 2007 11:54 AM by bishop7

    Internet Traffic Monitoring


       Currently we have internet issues at two of our sites (poor performance, connection resets on both ends of the link.)  When we first encountered the issue at one of our sites, we used the Solarwinds Engineers Toolkit to monitor the utilization of the link and it determined that our internet link was congested.  We use Orion Netflow Traffic Monitor to monitor Netflow traffic.  It looks like a really powerful tool, but unfortunately i'm either having difficulty interpreting the data or we don't have something set up correctly to monitor it as the results don't seem to account for the congestion we're seeing.

       First, the setup:

      We have a core router (Cisco 3745) with the following commands:

       Interface (private network and WAN interfaces):

       ip flow ingress


      ip flow-export destination <IP of Orion NTA Server>
      ip flow-export version 5

      The core router is connected to a switch onthe private network and the WAN.  The core router sends all traffic NOT destined for the WAN or the private network to our Cisco ASA 5510 , which is our security gateway (also connected to a switch on the private network.)  I know the ASA doesn't support sending Netflow data on its own.

      To monitor the internet traffic i've been looking on the private interface on the router (in NTA) and looking at the top domains and the traffic going across the link.  Nothing there seems to account for the traffic we're seeing.

      So my first question, I guess, is, is the way we have it set up now adequate for monitoring the internet traffic (or are some changes needed either on the physical side or software end) and, if so, are there any other effective ways of using this tool to monitor internet traffic that I can use?

        • Re: Internet Traffic Monitoring

          To monitor the internet traffic i've been looking on the private interface on the router (in NTA) and looking at the top domains and the traffic going across the link.  Nothing there seems to account for the traffic we're seeing.



           Can you be more specific?  Screenshots would be great, but if that's not possible, maybe some paraphrasing of the data that you're seeing and the data you might expect to see.

            • Re: Internet Traffic Monitoring

               I'd be happy to...

               Here's the top 99 (well, the top as many as I can fit on a 1280x1024 screen at any rate) domains for the inside interface between the hours of 10 pm and 4:00 PM (our peak hours for internet usage):

              Here are the top 10 applications for the same time frame.  HTTPS is 100% traffic going to our ERP system on the network local to the NTA server and router interface we're monitoring here.  HTTP and SMTP combined are about 200 MB over a period of 6 hours -- not really significant enough to explain congestion.  We have a 3 Mb internet link.  The rest are all internal service ports with traffic destined to local nodes on the private network or the WAN.


              Here is the breakdown by destination / country:

              This information is what I have typically been looking at when trying to determine who our bandwidth hogs may be.  If you have any other ideas of what I should look at or if you want to see something else, please let me know.


                • Re: Internet Traffic Monitoring

                  Can you look at the Interface Details View (in the Orion NPM section of the website) and determine specific times where your percent utilization has been high?  From that, you should be able to use the NetFlow Interface Details view to select that specific time period and determine who/what is chewing up your bandwidth.


                    • Re: Internet Traffic Monitoring

                      Therein lies the problem, as the data on the ASA's outside interface doesn't line up with what the Netflow monitor is reporting as the internet traffic going through the inside interface of the router and destined to the internet.

                      For example, this image shows about 100 MB being received between 3:00 PM and 3:10 PM today:

                      Looking at the inside interface of our router, between 3:00 PM and 3:10 PM, sorted by destination country:


                      Outside of the reserved addresses, this doesn't add up to a figure that's even close to the utilization of the link we're seeing on the ASA's outside interface.

                        • Re: Internet Traffic Monitoring

                          Still not quite following.  Let me try to back up a bit.  Which interface is getting congested?  Can you post the percent utilization chart for that interface?

                          Given the NetFlow view of the inside interface, why are you focusing on traffic by country?  From my experience with using NTA to troubleshoot congestion, I've usually found that the Top Applications, Top Receivers, and Top Transmitters lists to be the most helpful.

                           Also, given the screenshots above, it looks like HTTP is consuming a majority of your bandwidth, so you should be able to click on HTTP in the web interface to see the top receivers, transmitters, countries, domains, etc. broken down specifically to HTTP.  Is this what you're looking for, maybe?

                            • Re: Internet Traffic Monitoring

                              The outside interface on the ASA is congested.  Here is the interface utilization chart (for a more broad time period rather than the isolated one to generate a more meaningful graph) for the 1.5 Mb location I used in the most recent example (the other is a 3 Mb and seemed to have a 'slow' day yesterday):


                              First, keep in mind that the ASA is not physically connected to the core router, it is connected to a switch which is connected to the private network (which the core router has access to)   Logically, the path to the internet is from the core router (default gateway for all workstations) to the ASA to the internet router.

                              Since the ASA itself does not support sending Netflow information, I have to gather the Netflow data from the inside interface on our core router (which is our default gateway) ... the reason I posted the Netflow traffic on this interface by country is because (i'm guessing) it provides an accurate summary of where the traffic is going.  Since the inside interface gets traffic going to the WAN as well as the internet, i'm using it as a way to differentiate between internet traffic and WAN traffic -- and in this case I used it as an example of the differences between what i'm seeing analyzed by Netflow and what i'm seeing on the outside interface on the ASA.

                              To give you a different perspective of what i'm seeing, using your example above, this is what I have for that time period on the routers inside interface (top applications):


                              So if you take the figures for HTTP , HTTP (alternate) for the proxy server, HTTPS (which is largely communications between users on this site and the HTTPS interface to our ERP system located at a central site) and Unmonitored traffic in its entirely (again, some of this traffic may be WAN traffic) it still doesn't amount to the actual traffic figures we are seeing on the outside interface of the ASA.  The rest are all internal service ports.