Unable to capture NetFlow on Cisco 3750x switch

A few years back I had SolarWinds Real-Time NetFlow Analyzer working with my Cisco 3750x switch. Recently we had some suspicious traffic so I installed a new version of the software on a Win 10 machine. I can connect through the software to my switch, I can see all of the interfaces but none of them show NetFlow enabled. When I click on the interface I want to monitor then click "Start Flow Capture" I get a 'NetFlow is not detected on the selected interface'.

How do I get this port configured correctly to capture NetFlow data?

Additional Facts:

IOS version 15.0(2)SE6

Config:

int gig <port to be monitored>
ip flow ingress
ip flow egress

ip flow-export source <port to be monitored>
ip flow-export version 5
ip flow-export destination <IP of my Win 10 machine> 2055

Per this thread- https://thwack.solarwinds.com/thread/20498

I tried to run the ip nbar protocol-discovery and the ip route-cache flow on the port to be monitored. Neither of those commands were accepted on that port.

Any help is appreciated.

  • I've been able to use Solarwinds' documentation for getting Netflow configured on a good variety of Cisco devices.  3750's, Nexus 7K's and 5K's, 4510's, and a bunch of routers.  I use this basic guideline, and tweak and tune the commands based on individual platform limitations or requirements, which can be found if you Google Netflow and that particular Cisco box.

    Set up NetFlow NBAR2 on Cisco devices

    Network Based Application Recognition (NBAR) is the mechanism used by certain Cisco routers and switches to recognize a dataflow by inspecting some of the packets sent. SolarWinds NTA 4.2.1 supports unknown traffic detection and advanced application recognition through NBAR2.

    First, configure your Cisco devices to send NBAR2 data to SolarWinds NTA. Second, add those devices as nodes in SolarWinds NPM and SolarWinds NTA.

    The following values are examples used in the commands below:

    • NTArec

    • NTAexp
    • NTAmon
    • GigabitEthernet0/1
    • 10.10.10.10

    Create a new Flexible NetFlow configuration

    Add the flow record

    This process is similar to creating a standard NetFlow configuration. In this case, you add the collect application name command to enable the sending of AppID in each flow.

    flow record NTArec

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    match interface input

    collect interface output

    collect counter bytes

    collect counter packets

    collect application name

    exit

    Add the flow exporter

    The option application-table command enables the sending of a list of applications that can be classified using NBAR2, including applications that were manually created. The option application-attributes command enables the sending of categories for all applications.

    flow exporter NTAexp

    destination 10.10.10.10

    source GigabitEthernet0/1

    transport udp 2055

    export-protocol netflow-v9

    template data timeout 60

    option application-table timeout 60

    option application-attributes timeout 300

    exit

    Add the flow monitor

    The flow monitor connects the flow recorder and the flow exporter. You can configure multiple recorders, exporters, and monitors at once.

    flow monitor NTAmon

    description NetFlow nbar

    record NTArec

    exporter NTAexp

    cache timeout inactive 30

    cache timeout active 60

    exit

    When receiving long flows, these values may need to be adjusted, see Troubleshoot Long Flow Errors for more details. For more information about the timeout values, refer to the Cisco NetFlow Command Reference.

    Apply the monitor on an interface

    Assign the Flexible NetFlow configuration to the interface from which to monitor NetFlow.

    interface GigabitEthernet0/1

    ip flow monitor NTAmon input

    ip flow monitor NTAmon output

    exit

    Diagnostic commands

    show flow record "recordName"

    show flow export "exporterName"

    show flow monitor "monitorName"

    show flow exporter statistics

    show flow interface

    Determine the applications your device can recognize

    The Protocol Pack is a list of applications, definitions, and categories that your device can recognize.

    Check the Protocol Pack version

    show ip nbar version

    View a list of the available applications

    show ip nbar protocol-id

    Edit an existing record

    If you edit an existing record that is in use, you receive the following error:

    % Flow Record: Flow Record is in use. Remove from all clients before editing.

    To resolve this error, remove the connection between the monitor, record, and interface.

    Disable the connection

    interface GigabitEthernet0/1

    no ip flow monitor NTAmon input

    no ip flow monitor NTAmon output

    exit

    Add the application recognition field into the record

    flow record NTArec

    collect application name

    exit

    Add the application recognition field into the exporter

    flow exporter NTAexp

    option application-table timeout 60

    option application-attributes timeout 300

    Restore the connection

    interface GigabitEthernet0/1

    ip flow monitor NTAmon input

    ip flow monitor NTAmon output

    exit

    So let's say you have a Cisco 4510.  Here's my copy-and-paste instructions, minus the unique IP addresses or interfaces you need to add:

    How To Set Up Netflow on Cisco 4510 Version 8 Chasses:

    1. The switch hardware must be Version 8 or newer.  V7 and older requires NetFlow Modules to be purchased and installed in each Supervisor.
    1. The chassis must be licensed to run IP Base or Enterprise.  NetFlow is not supported on LAN Base license.

    conf t

    flow record NTArecord

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    match interface input

    collect interface output

    collect counter bytes

    collect counter packets

    collect timestamp sys-uptime first

    collect timestamp sys-uptime last

    !

    flow exporter NTAexport

    destination x.x.x.x (You add in your SW Poller's address here)

    source Loopback0 (Or use a different interface--whatever you use to manage the switch is the interface to report with)

    transport udp 2055

    export-protocol netflow-v5

    !

    flow monitor NTAmonitor

    description NetflowToOrion

    exporter NTAexport

    cache timeout inactive 10

    cache timeout active 5

    record NTArecord

    Add “ip flow monitor NTAmonitor input“ to every VLAN you want included.  You can also group them via this example:

    vlan configuration (Insert ALL the VLAN's on the 4510 in this area)

    ip flow monitor NTAmonitor input

    On the WAN interface's physical port(s):

    ip flow monitor NTAmonitor input

    Add this line for EVERY physical port to want to monitor on the switch: 

    ip flow monitor NTAmonitor input

    ! Modify the interface script that follows based on the modules you own:

    conf t

    int range gi1/1-48,gi2/1-48,gi3/1-48,gi4/1-48

    ip flow monitor NTAmonitor input

    int range gi7/1-48,gi8/1-48,gi9/1-48,gi10/1-48

    ip flow monitor NTAmonitor input

    Then tell the switch which interface to use as its Netflow source.   A 4510 serving as a WAN router and Distribution switch should use a loopback port, but you could choose the physical WAN interface.  Use the same port as is used by the switch for all its sourcing of logging, TACACS, snmp, etc.

    Build the exporter, then assign it to the correct Interface so Orion doesn’t throw a bunch of errors about an unmanaged device sending it Netflow info.

    Example: 

    conf t

    flow exporter NTAexport

    description LSEG internal

    destination (x.x.x.x is the IP address of your Solarwinds Poller)

    source Loopback0

    transport udp 2055

    export-protocol netflow-v5

    int loopback0

    flow monitor NTAmonitor

    exporter NTAexport

    record NTArecord

    Finally, ensure NPM is set to monitor all interfaces that have the “ip flow monitor NTAmonitor input“ command.  If it’s not, then it’ll send NTA interface errors.

    Removal is the reverse of the steps above, in this order:

    int loopback1

    no flow monitor FLOW-MONITOR-1

    no exporter EXPORTER-1

    no record NTArecord

    no flow exporter EXPORTER-1

    int range gi1/1-48,gi2/1-48,gi3/1-48,gi4/1-48

    no ip flow monitor NTAmonitor input

    int range gi7/1-48,gi8/1-48,gi9/1-48,gi10/1-48

    no ip flow monitor NTAmonitor input

    int range te5/1-8,te6/1-8

    no ip flow monitor NTAmonitor input

    vlan configuration x-x

    no ip flow monitor NTAmonitor input

    no flow monitor NTAmonitor

    no flow exporter NTAexport

    no flow record NTArecord

    Now let's suppose you had to do this on a 6509 Core or Distribution L3 switch.  Here's how:

    Enabling Netflow on 6509 Distribution Switches

    ip flow-cache entries 131072 (if you change this, the switch must be 

    rebooted or all flow must be removed before it takes effect)

    ip flow-cache timeout active 1

    ip flow ingress layer2-switched vlan x (must be done for every vlan)

    mls flow ip interface-full

    no mls flow ipv6

    mls nde sender version 5

    **VLAN/physical interface's**

    !  int vlan 2 (etc.  must be done for every SVI)

    ip flow ingress

    ip route-cache flow

    ip flow-export source lo0

    ip flow-export version 5

    ip flow-export destination x.x.x.x (this is the address of your Solarwinds server)  2055

    Let's say you want your ASA to report Netflow.  It's super easy:

    flow-export destination  ABCD  (the name of the ASA Interface that you want to send the Netflow traffic through--it might be really intuitive like "inside")   x.x.x.x (the IP address of your Solarwinds poller) 2055

    So you have 3750X's.  Are they compatible with NetFlow?

    pastedImage_13.png

    If they ARE compatible, I recommend you use Solarwinds' Netflow configuration guidance.  But you can also refer to Cisco's info here:

    Catalyst 3750-X and 3560-X Software Configuration Guide, Release 15.0(1)SE - Configuring Flexible NetFlow [Cisco Catalys…

    Good Luck!  Let us know how it works out for you!  Send pictures--or it didn't happen!

    pastedImage_16.png

  • We have a catalyst 6500  core and distribution with a Sup 2T.

    The Sup 2T has the flexibele netflow commands.

    An other 6500 switch with sup 32 has the ' old' version 5 commands.

    With the CAT6500 it's imported to notice witch supervisor you have.

  • rschroeder, I'm trying to analyze/capture netflow from the gigabit ports on my 3750x. That don't support flexible netflow. I've had non-flexible netflow working on my gigabit ports at some point in the past.

    Does the netflow analyzer only work with flexible netflow (one has to apply "ip flow monitor <name of flow monitor> input" on the specific interface they want monitored) now? The only netflow commands I can apply directly to the interfaces I want analyze are "ip flow ingress" and "ip flow egress".

  • Below is the code I had on my switch when netflow analyzer was working:

    flow record <record name>

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    collect counter bytes

    collect counter packets

    flow exporter <exporter name>

    destination 10.1.1.25

    transport udp 2055

    flow monitor <monitor name>

    description Original Netflow captures

    record ipv4

    exporter <exporter name>

    interface <interface to be monitored>

    ip flow ingress

    ip flow egress

    ip flow-export source <interface still not clear exactly this is for>

    ip flow-export version 5

    ip flow-export destination <netflow analyzer address> 2055

    ip flow-top-talkers

    top 10

    sort-by bytes

  • change the version from 5 to 9!!

    the default is 9.

    ip flow-export version 9

    If you are using vrf's?

    flow exporter <name>

    destination x.x.x.x vrf <vrf_name>

    source <management ip/vlan/loopback/interface>

    transport udp 2055

    Interface <NameOfInterface>

      ip flow monitor <NetflowMonitorName> input

      ip flow monitor <NetflowMonitorName> output

  • h.hendriks, I cannot apply ip flow monitor <monitorname> input nor output to the gigabit interfaces I want to monitor, my 3750x switch only supports those commands on fiber uplink (service module) ports. Is there no way to record netflow information from an interface when ip flow ingress and ip flow egress are applied?

  • NTA supports both version 5 and 9, but I recommend using version 9 with NBAR2 everywhere you can.  Some legacy devices aren't compatible with NBAR2, others can only do Netflow v5.  Find which ones have that limitation and compensate for them, and request budget to replace them with newer models that support Netflow v9 and NBAR2.

    I apply flow commands to every physical interface on my Cisco 4510 chasses now that the V8 model supports the commands, and it opens up another layer of granularity for traffic on a per-port basis.   In that particular environment, it's only possible to use the "ip flow monitor <name> input" command.  Initially I thought this was a limitation because there was no matching "output" command for the port.  It turns out that, while having both commands on the port seem intuitive and convenient, I'm really only interested in traffic coming "from" the device directly attached to the port.  Any traffic going "to" that device from another device is captured on the port(s) allowing the traffic into the switch from the other device.

    Regarding your 3750x, getting its Netflow going again most likely will require a review of the required commands and a fine-toothed comb going through the details.  Although you had it working previously, since it's not working now, you may benefit from thinking about what's changed that caused it to stop.

    • Was there an IOS update or downgrade that resulted in different capabilities, or that needs different commands applied to get Netflow going again?
    • Did a destination address change for the Netflow?  If you updated/changed a Solarwinds Poller, it could still be polling the 3750x, but the 3750x might not be sending Netflow to the correct destination address.

    If you have NCM, I'd recommend comparing a running-configuration from the 3750x at the time it was properly sending Netflow to today's running-config.  Maybe you'll see a change or a typo.  Or perhaps you'll find something that SHOULD have changed, but hasn't, to support a different destination address for a Solarwinds poller.

    I looked for a 3750x in my network that was running the right code and license level to use Netflow and I find I've retired them all.

    But here's a snip from one of my 4510's running Netflow on all interfaces that can be compared to your output:

    flow record NTArec

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    match interface input

    collect interface output

    collect counter bytes

    collect counter packets

    collect application name

    flow exporter NTAexp

    destination <x.x.x.x> (your Solarwinds APE running NTA)

    source <enter the interface on the switch that will be recognized as the source of the traffic.  Always use the same Interface that is being polled by Solarwinds--usually an SVI or a loopback>

    transport udp 2055

    template data timeout 60

    option application-table timeout 60

    flow monitor NTAmon

    description NetFlow nbar

    exporter NTAexp

    cache timeout inactive 30

    cache timeout active 10

    record NTArec

    ip flow monitor NTAmon input (this command goes on every physical port)

    vlan configuration (list all VLAN ID's here, comma-separated)

      ip flow monitor NTAmon input

    You may have to tweak this a bit for your 3750x's, but it should get you very close to running again.

    Swift packets!

    Rick Schroeder

  • This is the config I had/have on the switch, updated with the interface I want to monitor and the source ip of the new Netflow Analyzer. I have a feeling I'm getting tripped up on the ip flow-export source line, documentation did not make this clear. Right now I have it set as the interface I want monitored.

    flow record <record name>

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    collect counter bytes

    collect counter packets

    flow exporter <exporter name>

    destination <netflow analyzer IP>

    transport udp 2055

    flow monitor <monitor name>

    description Original Netflow captures

    record ipv4

    exporter <exporter name>

    interface <interface to be monitored>

    ip flow ingress

    ip flow egress

    ip flow-export source <interface still not clear exactly this is for>

    ip flow-export version 5

    ip flow-export destination <netflow analyzer address> 2055

    ip flow-top-talkers

    top 10

    sort-by bytes

  • pastedImage_0.png

    The "ip flow-export source" line tells the 3750x what IP address it should include as the "from" or "sender", when sending to your Solarwinds NTA poller.   For example, if you only have one IP address on the switch, and it's loopback0, then you'd say "ip flow-export source loopback0" on this line.

    If your 3750x has multiple IP addresses, always use the interface with the IP address that's being monitored by Network Traffic Analyzer for the "ip flow-export source".  It helps Solarwinds NPM and NTA keep everything aligned nicely when you use the same monitoring address that NPM knows about, for the source interface in NTA.

    If you don't do this, you'll be monitoring your switch in NPM with one IP address, and the switch will be sending Netflow information to NTA from an interface with a different IP address than the one NPM already is monitoring.  This will create an alert, and you'll be recommended to either add the new Netflow-associated IP address as an entirely new node (wasting license count and server resources), or you can simply change the "ip flow-export source" line to reference the Interface with an IP address that NPM monitors.

  • rschroeder, that makes more sense than what I read. When I ran the ip flow-export source command I couldn't put in an address specifically, I had to put in an interface, so I used the interface I ssh into the switch on. That didn't work unfortunately, when I open up netflow analyzer I still get no flow type next to any of the interfaces.

    I've also been working on getting the netflow configurator working. When I try to connect to my device using my read only SNMP community string, the software says I need a read/write community string to continue. I created a read/write SNMP community string, tried that in the software, it says cannot connect to device. Not sure what the issue is there either.