cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

NetFlow on Cisco Switches: NetFlow v9 configuration for Cisco Catalyst 3850

Level 12

The Cisco Catalyst 3850 is a fixed, stackable GE (Gigabit Ethernet) access layer switch that converges wired and wireless within a single platform. This switch is based on Cisco’s programmable ASIC named Unified Access Data Plane (UADP) which supports the convergence as well as allows for deployment of SDN and Cisco ONE (Cisco’s version of SDN).


The Catalyst 3850 switch can stack and route, supports PoE, has a higher throughput, larger TCAMs, be your Wireless LAN Controller supporting up to 50 AP and 2000 clients and importantly supports Flexible NetFlow export. And why is NetFlow important? NetFlow has over the years become the de-facto standard for bandwidth monitoring and traffic analytics due its ability to report on the ‘Who, What, When and Where’ of your network traffic.


Flexible NetFlow configuration for Cisco Catalyst 3850 Switch:

The Cisco 3850 needs either an IP Base or IP Services Base license to support Flexible NetFlow (FNF) export.


Flexible NetFlow configuration involves creating a Flow Monitor, Flow Exporter and a Flow Record. Flow Monitor is the NetFlow cache whose components include the Flow Exporter and Flow Record. The Flow Exporter carries information for the export – such as the destination IP Address for the flows, the UDP port for export, interface through which NetFlow packets are exported, cache timeout for active and inactive flows, etc. The Flow Record carries the actual information about the network traffic which is then used by your NetFlow analyzer tool to generate bandwidth and traffic reports. Some of the fields in a Flow Record are source and destination IP Address, source and destination port, transport protocol, source and destination L3 interface, ToS, DSCP, bytes, packets, etc.


So, here is a sample configuration for enabling Flexible NetFlow on a Cisco Catalyst 3850 and exporting it to your flow analyzer such as SolarWinds NTA.


Flow Record:

We start with creating the flow record. From the 'global configuration' mode, the followings commands are to be applied.

flow record NetFlow-to-Orion           \\ You can use a custom name for your flow-record

match ipv4 source address                               

match ipv4 destination address

match ipv4 protocol

match transport source-port

match transport destination-port

match ipv4 tos

match interface input

collect interface output

collect counter bytes long        \\ Though "long" is an optional command, readers have stated that NetFlow reporting works only when "long" is used

collect counter packets long


Flow Exporter:

And next for the flow exporter, again from the 'global config' mode.

flow exporter NetFlow-to-Orion       \\ You can use a custom name for your flow-exporter

destination 10.10.10.10                     \\ Use the IP Address of your flow analyzer server

source GigabitEthernet1/0/1            \\ Opt for an interface that has a route to the flow analyzer server

transport udp 2055                             \\ The UDP port to reach the server. SolarWinds NTA listens on 2055

Flow Monitor:

Now to associate the flow record and exporter to the flow monitor.

flow monitor NetFlow-to-Orion          \\ Again, you can use a custom name

record NetFlow-to-Orion                  \\ Use the same name as your flow record

exporter NetFlow-to-Orion               \\ Use the same name as your flow monitor

cache timeout active 60                  \\ Interval at which active conversations are exported - in seconds

cache timeout inactive 15                \\ Interval at which inactive conversations are exported - in seconds

Enabling on an Interface:

And finally associate the flow monitor to all the interfaces you would monitor with your flow analyzer. Go to the ‘interface config’ mode for each interface and apply the command:

ip flow monitor NetFlow-to-Orion input          \\ Or use the name of your custom flow monitor

The above command attaches the flow monitor to the interface you selected after which the ingress traffic that passes across the interface is captured and send to your flow analyzer for reporting.


For a trouble free setup, ensure that your firewalls or ACLs are not blocking the NetFlow packets exported on UDP 2055, and that you have a route from the interface you had selected under flow exporter to the flow analyzer server. And then you are all set. Happy Monitoring!

30 Day Full Feature Trial | Live Product Demo | Product Overview Video | Geeks on Twitter


60 Comments
Level 10

Been looking for such a template for a while, after several hit and miss attempts at receiving flows in Solarwinds.. this worked great!

Did however have to modify a couple of lines:

collect counter packets

collect counter bytes

Both of these were incomplete commands and I added "long"

Also had to remove "name" from the interface command.

Could you, or would you even want to monitor both ingress/egress traffic?

Level 12

Whoops! It was supposed to be ip flow monitor "monitor name" and instead I somehow ended up typing ip flow monitor name NetFlow-to-Orion input.

And in a flow record, you can collect either the 64 bit or the 32 bit counters for packets and bytes. The optional "long" command sets the flow record to collect 64 bit counters for packets and bytes. I am not sure why the collect command without the optional "long" did not work on your switch.

Thank you so much for pointing that out! [Blog edited to capture the correct command for applying flow monitor to an interface]

Level 12

Found it:

In some cases the size of a field type is fixed by definition, for example PROTOCOL, or IPV4_SRC_ADDR. However in other cases they are defined as a variant type. This improves the memory efficiency in the collector and reduces the network bandwidth requirement between the Exporter and the Collector. As an example, in the case IN_BYTES, on an access router it might be sufficient to use a 32 bit counter (N = 4), on a core router a 64 bit counter (N = 😎 would be required.

Source: http://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html

Level 8

It is possible to look at the traffic for a specific vlan?  Say I want to see all traffic that is on vlan 10.

Level 12

Yes, that should be possible. Apply the command "ip flow monitor NetFlow-to-Orion input" on the interfaces you need to monitor including VLANs

Level 9

...followed the steps exactly. This error appears soon afterwards:

NetFlow Receiver Service [servername] received an invalid V9 template with ID 256 from device w.x.y.z. See knowledge base for more information.

http://knowledgebase.solarwinds.com/kb/questions/802/Required+flow+template+fields

Level 12

That usually happens when the template for processing the flows has not yet reached NTA or the template that reached NTA is not a valid one. Do you still get the error?

Level 9

@sk3l3t0r's reply above was the answer. Those 2 commands must have "long" at the end of them. I thought I had fixed it when the parser kicked back the commands I pasted in, but I only fixed one of the commands and not both. It's working fine after fixing the other command.

Level 12

I have edited the commands in the blog. Thank you for confirming this for me.

Level 8

List of Cisco 3850 Switches detailed information:

Cisco Switch Catalyst 3850 - Router-Switch.com

Level 15

This article just made me smile.  I am in the process of a cisco refresh migrating to 3850 and now I am looking forward to getting the details that netflow offer to assist in the analysis of bandwidth and connectivity issues.  Presently, I have to run MRTG against a particular switch to gain insight into the port interface from a historical review.  Now, netflow will help me get some realtime conversation analysis.  THanks!!!

Level 12

Love it when NetFlow gets fans.

Level 9

when I go into an interface config range or single I don't have the ability to enable the interfaces for NetFlow. The line "ip flow monitor NetFlow-to-Orion input" returns "Invalid input detected at marker" the marker is under flow. This is on a Catalyst 3850 running IOS XE 03.02.03.SE Any ideas?

Level 15


Did you define the flow monitor earlier in the config?  Something like this

flow exporter NetFlow-to-Orion

destination xx.xx.xx.xx

source GigabitEthernetX/X/X

transport udp 2055

!

!

flow monitor NetFlow-to-Orion

exporter NetFlow-to-Orion

cache timeout active 60

record NetFlow-to-Orion

Level 8

Guys, this is good and thank you for the configs. What about Egress traffic though? When I tried applying the flow monitor to output (ip flow monitor NetFlow-to-Orion output) I got this error:

% Flow Monitor: Flow Monitor NetFlow-to-Orion

Unsupported match field "interface input" for ipv4 traffic in output direction

Unsupported collect field "interface output" for ipv4 traffic in output direction

So I created another flow record (NetFlow-to-Orion-out) without those 2 fields and I applied it to output successfully (ip flow monitor NetFlow-to-Orion-out output) but solarwinds doesn't see that traffic.

Any ideas?

Level 9

Yes, here is the NetFlow portion of my config

flow record NetFlow-to-Orion

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

collect interface output

collect counter bytes long

collect counter packets long

!

!

flow exporter NetFlow-to-Orion

destination xxx.xxx.xxx.xxx

source TenGigabitEthernet1/1/4

transport udp 2055

!

!

flow monitor NetFlow-to-Orion

exporter NetFlow-to-Orion

cache timeout active 60

record NetFlow-to-Orion

I did try it with both a gigabit interface just in case and the VLAN interface. They all give the same error.

Thanks for the help!!

Level 8

You can apply it on L2 like this:

conf t

vlan configuration 2-3 (or whatever vlans you want)

ip flow monitor NetFlow-to-Orion input

Level 12

Did you check what license you have?


The Cisco 3850 needs either an IP Base or IP Services Base license to support Flexible NetFlow (FNF) export.
Level 9

I was soo excited too:

Switch(config)#vlan configuration 100
Switch(config-vlan-config)#ip flow monitor NetFlow-to-Orion
                                   ^
% Invalid input detected at '^' marker.

Switch(config-vlan-config)#ip flow?
flow

Switch(config-vlan-config)#ip flow ?
% Unrecognized command
Switch(config-vlan-config)#ip flow
% Incomplete command.

Switch(config-vlan-config)#

Level 9

That makes sense, sucks, but makes sense. Thanks!

Level 12

For the benefits that NetFlow provide, the cost of a license upgrade should be fine, right?

Level 9

$6k for the upgrade license... YIKES! I'll be getting it at some point, but just bought a bunch of new server and networking hardware. I'm not brave enough to go ask for more money yet,

Level 8

Sorry to hear that. Well I think you need to look at your cisco firmware and license then. I have IOS 03.07.00E RELEASE SOFTWARE (fc4) with IOS image CAT3K_CAA-U​NIVERSALK9-M and it works fine. The only problem I have is with Egress traffic.

Level 12

cevangelou @emoore's issue was a license issue - the Cisco 3850 needs either an IP Base or IP Services Base license from Cisco to support Flexible NetFlow (FNF) export.

Level 12

cevangelou Can you paste the configuration of the flow record you created? And on which interface are you applying the flow monitor to? VLAN, regular, L2, WLAN?

Remember that with wireless FNF, NetFlow supports only one flow monitor per interface, per direction. So if you already have a monitor on a WLAN interface, you will not be able to add a second one.

And SolarWinds NTA (and most NetFlow tools in the market) requires a few key fields including interface input and output to generate proper reports.

Level 16

Hi Chris,

could you send us sample of PCAPs coming from your devices so we can see netflow data?

thanks,

michal

Level 8

Thanks for responding donthomas. Here is the full config. I cant apply it on interface vlan so its applied on L2

flow record FLOW-RECORD-1

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

match flow direction

collect interface output

collect counter bytes long

collect counter packets long

!

!

flow exporter SOLARWINDS

destination 10.7.6.253

source Vlan200

transport udp 2055

!

!

flow monitor FLOW-MONITOR-1

exporter SOLARWINDS

cache timeout active 60

record FLOW-RECORD-1

flow record FLOW-RECORD-OUT

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match flow direction

collect counter bytes long

collect counter packets long

!

!

flow exporter SOLARWINDS-OUT

destination 10.7.6.253

source Vlan200

transport udp 2055

!

!

flow monitor FLOW-MONITOR-OUT

exporter SOLARWINDS-OUT

cache timeout active 60

record FLOW-RECORD-OUT

vlan configuration 2-4

ip flow monitor FLOW-MONITOR-1 input

ip flow monitor FLOW-MONITOR-OUT output

The only difference between the 2 records is:

match interface input

collect interface output

With them in the record I get the error above. Without them it let me apply it on the output but solarwinds doesn't see the traffic.

Thanks

Level 8

I guess I can install wireshark and get some captures going. Would a couple of minutes be good?

Level 12

cevangelou Can you try without the "match flow direction" command in the flow record?

Level 8

I just did and still no egress traffic.

From this: SolarWinds Knowledge Base :: Required flow template fields Im guessing solarwinds needs all the fields in the record you created originally. What do you think?

Level 8

How do I upload a file here? I have a capture that shows records with 11 and 8 fields. I'm guessing the ones that have 8 don't show in solarwinds. ( http://knowledgebase.solarwinds.com/kb/questions/802/Required+flow+template+fields)

Is this right to assume that tis is probably my record FLOW-RECORD-1 (with 11 fields)

11f.jpg

and this (with 8 fields) FLOW-RECORD-OUT (I took the match flow direction out from above)

8f.jpg

Level 12

cevangelou

I meant, can you try this config:

flow record FLOW-RECORD-1

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

collect interface output

collect counter bytes long

collect counter packets long

!

!

flow exporter SOLARWINDS

destination 10.7.6.253

source Vlan200

transport udp 2055

!

!

flow monitor FLOW-MONITOR-1

exporter SOLARWINDS

cache timeout active 60

record FLOW-RECORD-1

And apply this on the interface as "ip flow monitor flow-monitor-1 input" after removing all other monitors from the interface.

Level 8

Ok I'm confused now because the input "ip flow monitor flow-monitor-1 input" works fine. The output is the one that has the problem. I tried that on the output "ip flow monitor flow-monitor-1 output" and it doesn't work.

Thanks

Level 12

cevangelou‌‌

Solarwind NTA can process NetFlow data only if all required fields are present in the flows. The fields from "match interface input" and "collect interface output" are needed for NTA to generate reports but because your flows from your output flow monitor does not have the two fields, NTA would not process them.

We need to find out why the Cisco device is not accepting "ip flow monitor flow-monitor-1 output" when the interface input and interface output fields are added. Can you try the config I provided earlier? Make sure you remove "match flow direction" and add "match interface input" and "collect interface output to the flow record and use "ip flow monitor flow-monitor-1 output" only after removing all other monitors from the interface.

Level 12

The 2nd flow cap is missing the flow direction as well as the SNMP interface index through which a flow is entering and exiting the switch. This means NTA will not know to which interface a flow has to be associated with.

Level 8

Thanks for staying with me on this donthomas. I think we came to the same conclusion here. I tried what you are saying above and the switch doesn't accept it on the output.

What I found is that as long as the "match interface input" and "collect interface output" are in the record (which correspond to the solarwinds required fields input_snmp and output_snmp) the switch wont let you apply it on the output. (I tried different combinations and even tried with these two only in the record but it wont apply on the switch.)

So since I don't see a way to make the switch accept these two on the output and keep solarwinds requirements happy at the same time, is it possible to change the solarwinds template somehow so it doesn't require them?

I think its time to start looking into the solarwinds site at this point. Instead of trying to find out why the switch doesn't accept them on the output (I don't think its possible unless you have a way that I couldn't find) I think we should be looking at why they are required by solarwinds in the first place and if they can be removed from its required fields..

Thanks

Chris

Level 12

SolarWinds NTA needs those two fields to know which interface the NetFlow stats has to be associates to. Without interface info, it would be like having information about an IP conversation's source and destination and what it was, but with no information about the switching or routing device it passed through. Such flow records would be discarded as the flows are not associated with any input or output interfaces and NTA cannot show them in a report as stand alone conversation not associated with a device or interface.

Level 12

cevangelou‌ How many monitors do you have on your interface? Are you trying to apply the monitor in output direction with a monitor in input direction already applied on the interface?

Level 12

I think I have found an answer here. cevangelou can you check this:

If you apply a flow monitor in the input direction:

  • Use the match keyword and use the input interface as a key field.
  • Use the collect keyword and use the output interface as a collect field. This field will be present in the exported records but with a value of 0.

If you apply a flow monitor in the output direction:

  • Use the match keyword and use the output interface as a key field.
  • Use the collect keyword and use the input interface as a collect field. This field will be present in the exported records but with a value of 0.
Level 8

donthomas YO DA MAN!!! Thank you!! So all that needs to be done is reverse those 2. Here is the full config that works for both ingress and egress with the differences in italic: (The flow direction doesn't matter)

flow record FLOW-RECORD-1
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
match flow direction
collect interface output
collect counter bytes long
collect counter packets long
!
!
flow record FLOW-RECORD-OUT
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface output
collect interface input
collect counter bytes long
collect counter packets long
!
!
flow exporter SOLARWINDS
destination 10.7.6.253
source Vlan200
transport udp 2055
!
!
flow exporter SOLARWINDS-OUT
destination 10.7.6.253
source Vlan200
transport udp 2055
!
!
flow monitor FLOW-MONITOR-1
exporter SOLARWINDS
cache timeout active 60
record FLOW-RECORD-1
!
!
flow monitor FLOW-MONITOR-OUT
exporter SOLARWINDS-OUT
cache timeout active 60
record FLOW-RECORD-OUT
!

!
vlan configuration 2-4
ip flow monitor FLOW-MONITOR-1 input
ip flow monitor FLOW-MONITOR-OUT output
!

Level 12

Ha ha.. looove it when we have answers...

Level 8

Huge help for me after looking for answers on this for quite some time. Thanks VERY much for posting this...

Level 8

Team,

Just read the above blogs my V9 configs are below ,I want to capture both the upload and download utilization along with traffic analysis of the WAN interface.will the below config be enough.

low record NETFLOW_RECORD

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

match flow direction

collect interface output

collect counter bytes long

collect counter packets long

flow exporter SOLARWINDSNTA-EM

destination 10.10.10.1

source lo0

transport udp 2055

template data timeout 30

option interface-table timeout 30

!        

!

flow monitor SOLARWINDSNTA-FMM

exporter SOLARWINDSNTA-EM

cache timeout active 30

cache timeout inactive 15

record NETFLOW_RECORD

!

interface GigabitEthernet0/0/0

ip flow monitor SOLARWINDSNTA-FMM input

ip flow monitor SOLARWINDSNTA-FMM output

!

Level 12

cisco routing

Level 8

can you please elaborate..

Level 12

Apply "ip flow monitor SOLARWINDSNTA-FMM input" on all interfaces of your router. That should work.

Level 8

donthomas

HI

"ip flow monitor SOLARWINDSNTA-FMM output" what wrong with this command...i want to save the license count and only want to monitor the wan interface since it also contains to and fro details of the traffic .

Level 8

also if i dont use this "ip flow monitor SOLARWINDSNTA-FMM output"command ,my egress charts are empty...

Level 12

You can apply both input and output command on the WAN interface and see ingress and egress traffic.

Or you can apply input on all interfaces and still see the ingress and egress traffic on the WAN interface. To save your licenses, unmanage the interfaces you dont need to monitor in NTA.

Level 8

donthomas‌ sure don thanks but i was referring ur comments to @cevangelou above and prepared the template below...or can i use the same template above  for analyzing both inbound and outbound traffic.

flow record NETFLOW_RECORD
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
match flow direction
collect interface output
collect counter bytes long
collect counter packets long
!
flow exporter SOLARWINDSNTA-EM
destination 10.10.10.1
source lo0
transport udp 2055
template data timeout 30
option interface-table timeout 30
!   
flow monitor SOLARWINDSNTA-FMM
exporter SOLARWINDSNTA-EM
cache timeout active 30
cache timeout inactive 15
record NETFLOW_RECORD
!

interface GigabitEthernet0/0/0
ip flow monitor SOLARWINDSNTA-FMM input
ip flow monitor SOLARWINDSNTA-FMM output

OR THE BELOW SHOULD BE FINE..

flow record FLOW-RECORD-IN
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface input
match flow direction
collect interface output
collect counter bytes long
collect counter packets long


flow record FLOW-RECORD-OUT
match ipv4 tos
match ipv4 protocol
match ipv4 source address
match ipv4 destination address
match transport source-port
match transport destination-port
match interface output
match flow direction
collect interface input
collect counter bytes long
collect counter packets long

flow exporter SOLARWINDSNTA-EM-IN
destination 10.10.10.1
source lo0
transport udp 2055
template data timeout 30
option interface-table timeout 30
!
flow exporter SOLARWINDSNTA-EM-OUT
destination 10.10.10.1
source lo0
transport udp 2055
template data timeout 30
option interface-table timeout 30

flow monitor SOLARWINDSNTA-FMM-IN
exporter SOLARWINDSNTA-EM-IN
cache timeout active 30
cache timeout inactive 15
record FLOW-RECORD-IN

flow monitor SOLARWINDSNTA-FMM-OUT
exporter SOLARWINDSNTA-EM-OUT
cache timeout active 30
cache timeout inactive 15
record FLOW-RECORD-OUT


interface GigabitEthernet0/0/0
ip flow monitor SOLARWINDSNTA-FMM-IN input
ip flow monitor SOLARWINDSNTA-FMM-OUT output