Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Tracking Web Sites Visited with NTA

In an effort to "increase productivity" I have been asked to track employee web site usage during business hours. I have been using NTA to monitor and track network usage as a whole in terms of protocols and have a pretty good indication of top users, top protocols, etc. However, when looking into web usage I find I get a little lost. I am tracking top conversations, top transmitters and receivers.

When I look at the lists of external hosts I see DNS info that I am having a hard time sorting out. I see stuff like www.facebook.com that are pretty easy to understand, then there are entries like 80.mtl-mg07.streamtheworld.net (208.80.52.80) and I can assume it is streaming audio via web browser. But then I see entries like a63-80-4-73.deploy.akamaitechnologies.com (63.80.4.74) or cdn-68-142-93-133.sea2.llnw.net (68.142.93.133) and I'm not sure what to do with those. These are all http conversations, using port 80.

I'm not really sure how to take that info and turn it into a web site. I've tried browsing to those IP's with port 80, but most of the time I get nothing. Any hints?

Thanks in advance.

Find more posts tagged with

Accepted answers

olivern

Take a look at running squid proxy in transparent mode. I am running pfSense firewall which actually includes a prebuild package.

All http traffic gets transparently ran through a squid cache which tracks all the requests (URLs).

Let me know if you need more details.

All comments

jswan

You can't get accurate statistics at the HTTP level with NetFlow. This isn't a limitation of NTA: it's due to the fact that most websites are hosted on servers that have multiple sites associated with the same IP address, and are differentiated only by the host headers in the HTTP request. Flow-level data doesn't contain HTTP headers. Furthermore, many sites are hosted on content distribution networks like Akamai and LLNW, which makes flow-based tracking extremely difficult.

You need an HTTP-level inspection tool to get accurate information about websites visited.

qle

Akamai (akamaitechnologies.com) and LimeLight Networks (llnw.net) are two Content Delivery Networks (CDN) that host a lot of the streaming media (both video and audio) going over the Internet. There are many other CDNs but I believe these may be the most popular, based on the information I'm seeing in NTA. These CDNs have agreements/deals/partnerships with different kinds of web sites ranging from YouTube and Facebook to CNN and MSNBC.

The problem here is that not many of these sites disclose who their CDN is and it could change from year to year, it seems. Some might even use more than one CDN so you can see how it could be very difficult to map a CDN to a web site or vice versa.

It sounds like jswan's suggestion would be the best option for you.

SeaDave

Having some of the same issues. I've been chasing 208.50.77.0/24 today. Owned by Global Crossing but no DNS assigned to any of the hosts in that range even though all of them show 80, 25, 443, 21 on them. I'm guessing they are Gmail or Hotmail, but I'm seeing a lot of traffic moving between my LAN and that range which makes me nervous. We use Fortinet firewalls for content filtering. You might look into those. Depending on the size of your company, you can purchase one for a modest price. It includes AV filtering, IDS, IPS, Web Content Filtering, VPN, etc, etc, etc. Makes it very easy to say no adult content, streaming media, etc can be viewed at work. Ticks off the users but they get over it. Can't recommend them highly enough.

SeaDave

Just figured out what it is. Windows Live.

acherman

Is Windows Live work related? haha We use an Allot NetEnforcer - not as easy to set up in terms of selecting no adult content, etc, but I can block URLs that contains cetain tags, hosts, etc. But I need to know the URL first - usually given to me by their managers. haha And you're right, it does tick off the users, but then they get back to work. What gets me is when they call and ask us to help troubleshoot getting to a website like Facebook, or Hotmail. We always ask how those are work related. ;-)

SeaDave

Social networking, blah! Just another vector to be used by web hackers. You are smart to block Facebook and Myspace. Filled with malware. Good luck.

olivern

Take a look at running squid proxy in transparent mode. I am running pfSense firewall which actually includes a prebuild package.

All http traffic gets transparently ran through a squid cache which tracks all the requests (URLs).

Let me know if you need more details.

acherman

Hey Oliver, that sounds great. The best part is I am also running pfSense here, but I've never played around with the squid package. Any info you can offer is appreciated.

Thanks!!!!

olivern

If you go to your pfSense webconsole, System, Packages, and then look for squid. Here is how my squid package is labeled squid 2.7.8_1. Once that is installed you can go to Services in your pfSense web console and click on Proxy Server. There you will be walked through the config. Keep in mind that this will generate IO and you will need diskspace for the cache and logs.

Let me know if you run into any problems.

acherman

Got it running now on one box - the output is pretty raw. Have you worked in any way to filter the text output to specific hosts? Or just slowly sift through. Obviously not ideal but it works quite well. Thanks again. Much appreciated.