NetFlow Source...

Hi All,

Complete newb here, first post.

We are half way through the NPM onboarding academy (we've brought the BW Analyser pack).

Looking ahead to NetFlow/IPFix, our WAN provider are currently migrating us from a CIsco MPLS WAN to a VERSA SD WAN, they have informed us that this will not support NetFlow/IPFix.

It does obviously, but will only send data to their own analytics platform and they could not forward only our companies data from there.

We could look to forward data from the switches, switch to router trunk port. Thats not ideal in any case, but we have rolled out Meraki switches that do not support NetFlow/IPFix.

Is there a SW or HW agent anyone is aware of that could send the data if we had it in a mirror port, mirroring the switch to router trunk port?

Some sort of USB pen drive sized dongle perhaps?



  • This article should help

    I don't have an off hand suggestion for a minipc or similar tool that I can say would definitely have enough horsepower, but essentially anything the runs linux and has fast enough ports and cpu will work.  "Fast enough" depends on your volume of traffic though, mirroring 10 gbps through is a much higher bar to meet than 10 mbps. 

  • Thanks mesverrum, that'll be worth a go. I'll give that a go.

  • So, this VERSA company owns the routers and won't do it for you?  Rather rude.   To be honest, I don't think I'd be overly happy with another company owning my WAN routers and not giving me any access there.   I'd probably want to put routers, or maybe firewalls, between their routers and my LANs, that way you have the ability to do some additional security if desired.   It would also allow you to do your Netflow and such at that point.

    But, that being said, what kind of switches do you have that the routers connect to?   Some Cisco switches, such as 3850's or 9000 series, will do Netflow also.  You could do it on them?

  • HI cnorborg. Versa are the universal CPE equipment manufacturer, they look to be a leader in the SDWAN space. A partner of theirs, our WAN supplier will be providing the managed WAN and the nature of a managed service is, you don't get access to it. Its the managed service via a multitenant orchestration platform that is the issue.

    I was 20 years on the other side of the fence with Cable and Wireless and then  Vodafone, I don't think I've ever heard of a end client putting security between the LAN and their suppliers managed WAN.

    As for the cisco switches, that would work, but we've gone to meraki switches that don't support NetFlow.

    I think we should be alright mirroring the switch/router trunk port and SW agents (porbes?) on PC's.


  • One example of customers putting routers on their side of a link would be the CE/PE concept in MPLS networks, its possible for the provider to do the CE/PE functionality in their router and just give an ethernet link, but at that time most customers wanted their own router on the border.    I've worked at two different commercial entities that did it also on non MPLS networks where the WAN provider put a router on site but the company still wanted their own router inside of it.   In the government space, where I am now, its definitely not unusual either.

    Just to be clear, the other suggestion was not to use SW agents, their agents don't do anything like that.  It was to use some public domain software, like f-probe or n-probe to pcap the traffic, generate its own netflow packets and send it on to nta.   You can definitely get traffic to NTA that way, but it will be a bit disjointed when I've tried similar things in the past.   The problem is that the f/nprobe box is out of band of the actual traffic, so it will use a different IP to forward this traffic to NTA.  Because of that, it will show up as a different device in NTA than the router itself, so everything will be a little disjointed, you probably won't see the actual interface names, just their ifindex numbers for instance.   You can definitely see the netflow traffic, it just won't be as nicely integrated as actually monitoring a router or something similar.

    Another thing to be cautious about is the bandwidth of the span interface.   ie: if you're WAN has 1 Gb of bandwidth for instance, that's 2Gb of bidirectional flow.   If you use a 1Gb interface for the SPAN, you'll could be trying to force up to 2Gb of traffic down that link to get it to your netflow box.   Just make sure you have double the bandwidth of the WAN link at least going to the SPAN box and you'll be ok on that.   

    There is also a potential issue with the PC doing the Netflow processing.   Most routers and such that process Netflow do it in hardware, which allows it to not impact the CPU of the router at all.   If you're using f/nprobe, that means the CPU on that box is going to have to interpret every frame of traffic in order to generate the flow data.   I've heard that CPU's and NIC's on boxes can have difficulty keeping up, just because they aren't designed for it.   You'll probably experience issues from packet drops, time stamp issues and maybe other things.

    Before investing a lot of money in the hardware requried to do this, I'd recommend some pretty serious testing.   Maybe pick your busiest WAN link and set this up on that.   If you can get it working well on it, then you should be able to get it working well everywhere.

    You know, I'm trying to remember where I heard a lot of this stuff, I think there is a research team working on a project to address problems like this at some university, for some reason I think they were based out of Czechoslovakia or some place similar.    Might be worth trying to look up for you...

  • When I was in a similar situation, I reached out to the president of the WAN service provider and pointed out the millions of $$ our company was annually spending on WAN service & network hardware purchases from the service provider.  I told him what I wanted, explained I was denied by his staff who weren't being very flexible, and noted his wasn't the only provider in town.  I asked what solution he'd provide to keep our business and get us the Netflow and utilization information we wanted.  

    Later that same day I was given secure access and credentials into the WAN provider's Solarwinds environment that had everything I wanted.  They'd set up access accounts and pages in their environment that allowed us RO access to all the equipment & interfaces that carried our traffic.

    I think there were probably simpler solutions, but this one was appreciated for the time & effort they put into doing it right.  It also further cemented our relationship to each other.  When you analyze it, this might be a best solution for ISP's to offer as a value-added proposition for their customers, since the data's all there, already being gathered.  It costs little in time & account maintenance and page building to give a customer this kind of monitoring access.

    The only shortfall was the initial reaction of their staff to the first request.  Just saying "no" doesn't cut it when the customer can turn to other vendors.

  • Thanks again for the detailed reply cnorborg.

    The WAN is a WAN provider managed PE/C(P)E model, SDWAN will be the same with universal CPE.

    Thanks for the advice on the mirroring though, I'll keep that in mind, our biggest links are 100Mbps, I'm goin to test on one of our regional sites, see how it goes.