Showing results for 
Search instead for 
Did you mean: 
Create Post

Related Attributes: NetFlow and Interface Traffic Statistics

Level 10

For many engineers, operators, and information security professionals, traffic flow information is a key element to performing both daily and long-term strategic tasks. This data usually takes the form of NetFlow version 5, 9, 10, and IPFIX as well as sFlow data. This tool kit is widely utilized, and enables an insight into network traffic, performance, and long-term trends. When done correctly, it also lends itself well to security forensics and triage tasks.

Having been widely utilized in carrier and large service provider networks for a very long time, this powerful data set has only begun to really come into its own for enterprises and smaller networks in the last few years as tools for collecting and, more importantly, processing and visualizing it have become more approachable and user-friendly. As the floodgates open to tool kits and devices that can export either sampled flow information or one to one flow records, more and more people are discovering and embracing the data. What many do not necessarily see, however, is the correlation of this flow data with other information sources, particularly SNMP-based traffic statistics, can make for a very powerful symbiotic relationship.

By correlating traffic spikes and valleys over time, it is simple to cross-reference flow telemetry and identify statistically diverse users, applications, segments, and time periods. Now, this is a trivial task for a well-designed flow visualization tool. It can be accomplished without even looking at SNMP flow statistics. However, where it provides a different and valuable perspective is in the valley time periods when traffic is low. Human nature is to ignore that which is not out of spec, or obviously divergent from the baseline. So, the key is in looking at lulls in interface traffic statistics. View these anomalies as one would a spike, and mine flow data for pre-event traffic changes. Check TCP flags to find out more intricate details of the flows (note: this is a bit of a task as it entails adding TCP flags as they are exported as a numerical value in NetFlow v5 and v9, but they can provide an additional view into other potential issues). Conversely, the flags may also be an indicator into soft failures of interfaces along a path, which could manifest as SNMP interface errors that are exported and can be tracked. Think about the instances where this may be useful: soft failures. Soft failures are notoriously hard to detect, and this is a step in the right direction to doing so. Once this kind of mentality and correlation is adopted, adding in even more data sources to the repertoire of relatable data is just a matter of consuming and alerting on it. This falls well within the notion and mentality of looking at the network and systems as a relatable ecosystem, as mentioned in this post. Everything is interconnected, and the more expansive the understanding of one part, the more easily it can be related to other, seemingly “unrelated” occurrences.

This handily accomplishes two important tasks: building a relation experience table in an engineer or operators mind, and, if done correctly, a well-oiled, very accurate, efficient, and documented workflow of problem analysis and resolution. When this needs to be sold to management, which will need to occur in many environments, proving out that most of these tracked analytics can be used in concert with each other for a more complete, more robust, more efficient network monitoring and operational experience may need some hard deliverables, which can prove challenging. However, the prospect of “Better efficiency, less downtime” is typically enough to get enough interest in at least a few conversations.

Level 13

Good Article. Lots to think about.

It seems at least some of the processes recommended are those which should be handled readily by an application, and then displayed in easily-understood graphics.  I'm thinking of the references:

  • correlating traffic spikes and valleys over time
  • checking TCP flags to find out more intricate details of the flows
  • soft failure detection & alerting & interpretation

Are any of these in the plans for future Solarwinds releases?

Level 10

I can't speak to that, but I do agree that much of this can be automated in most netflow collector platforms. I have done fairly extensive soft failure detection and alerting work over the years - it is very doable to automate and alert on known soft failure scenarios based on predefined criteria and existing baselines.


Nice article

Level 13

I concur... Nice job

Level 20

Netflow and the later enhancements made by Cisco like Medianet has been big for us lately.  Especially with all the QOS issues that arise when doing QOS across the entire network.

Level 16

At my previous employer we had Netflow deployed everywhere and then very large packet sniffers on the network core and on all of the WAN links. A team that included Management, Security, Network Architects, Network Engineers met every other week to review the traffic trends, changes, and QOS usage for the previous two weeks. We used Netflow to get the big picture, then drilled into specifics with the packet sniffers when needed. The network ran very smooth and everyone know what was normal and not.

I was very happy using Orion's Netflow and we were monitoring the network core for 2 data centers and 250 large retail locations, warehouses, etc. 

Level 15

Great article.  Things we sometimes overlook as you say are the lulls.  But, those are opportunities to dig.  Also, despite the fact that it is a rarity, is to utilize free time to learn more of the specifics of how our visualization tools work so we can use them to solve issues and provide reports.

Thanks for making me think!

About the Author
15+ years IT experience ranging from networking, UNIX, security policy, incident response and anything else interesting. Mostly just a networking guy with hobbies including, film, beer brewing, boxing, MMA, jiu jitsu/catch wresting/grappling, skateboarding, cycling and being a Husband and Dad. I don't sleep much.