cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

Unable to capture NetFlow on Cisco 3750x switch

A few years back I had SolarWinds Real-Time NetFlow Analyzer working with my Cisco 3750x switch. Recently we had some suspicious traffic so I installed a new version of the software on a Win 10 machine. I can connect through the software to my switch, I can see all of the interfaces but none of them show NetFlow enabled. When I click on the interface I want to monitor then click "Start Flow Capture" I get a 'NetFlow is not detected on the selected interface'.

How do I get this port configured correctly to capture NetFlow data?

Additional Facts:

IOS version 15.0(2)SE6

Config:

int gig <port to be monitored>
ip flow ingress
ip flow egress

ip flow-export source <port to be monitored>
ip flow-export version 5
ip flow-export destination <IP of my Win 10 machine> 2055

Per this thread- https://thwack.solarwinds.com/thread/20498

I tried to run the ip nbar protocol-discovery and the ip route-cache flow on the port to be monitored. Neither of those commands were accepted on that port.

Any help is appreciated.

0 Kudos
15 Replies

It seems our hardware types and IOS versions aren't close enough to be completely compatible with each other, and my examples may not be helpful.

It may be time to open a TAC case AND a ticket with Solarwinds Support, to ensure you have

both the correct and compatible Cisco IOS/Hardware/Commands/Licenses/Capabilities on

the switches, and the right expectations on the Solarwinds side.

Here are the "show flow" outputs for one of my 4510's that is working well with NTA:

#sho flow exporter

Flow Exporter NTAexp:

  Description:              User defined

  Export protocol:          NetFlow Version 9

  Transport Configuration:

    Destination IP address: <Solarwinds APE IP address>

    Source IP address:      x.x.x.x (the SVI of the Management VLAN on the switch)

    Source Interface:       (the VLAN hosting the IP address)

    Transport Protocol:     UDP

    Destination Port:       2055

    Source Port:            60156

    DSCP:                   0x0

    TTL:                    255

    Output Features:        Not Used

  Options Configuration:

#show flow interface

Interface GigabitEthernet1/1

  FNF:  monitor:          NTAmon

        direction:        Input

        traffic(ip):      on

Interface GigabitEthernet1/2

  FNF:  monitor:          NTAmon

        direction:        Input

        traffic(ip):      on

(the above extends to all 384 physical ports on the chassis switch)

#show flow monitor

Flow Monitor NTAmon:

  Description:       NetFlow nbar

  Flow Record:       NTArec

  Flow Exporter:     NTAexp

  Cache:

    Type:                 normal

    Status:               allocated

    Size:                 4096 entries / 278544 bytes

    Inactive Timeout:     30 secs

    Active Timeout:       10 secs

    Update Timeout:       1800 secs

    Synchronized Timeout: 600 secs

#sho flow record

flow record NTArec:

  Description:        User defined

  No. of users:       1

  Total field space:  34 bytes

  Fields:

    match ipv4 tos

    match ipv4 protocol

    match ipv4 source address

    match ipv4 destination address

    match transport source-port

    match transport destination-port

    match interface input

    collect interface output

    collect counter bytes

    collect counter packets

    collect application name

0 Kudos

rschroeder , I'm begginning to think I didn't get netflow analyzer working (this was four years ago), but what other piece of solarwinds software would let me see IP endpoints and protocols flowing through an interface?

0 Kudos

You might have been using Netflow Realtime.  It's in Solarwinds Engineer's Toolset.

The Toolset also has Netflow Configurator; maybe it can help you out (if you own a copy of the Engineer's Toolset)?

0 Kudos

Below is the code I had on my switch when netflow analyzer was working:

flow record <record name>

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

collect counter bytes

collect counter packets

flow exporter <exporter name>

destination 10.1.1.25

transport udp 2055

flow monitor <monitor name>

description Original Netflow captures

record ipv4

exporter <exporter name>

interface <interface to be monitored>

ip flow ingress

ip flow egress

ip flow-export source <interface still not clear exactly this is for>

ip flow-export version 5

ip flow-export destination <netflow analyzer address> 2055

ip flow-top-talkers

top 10

sort-by bytes

0 Kudos

change the version from 5 to 9!!

the default is 9.

ip flow-export version 9

If you are using vrf's?

flow exporter <name>

destination x.x.x.x vrf <vrf_name>

source <management ip/vlan/loopback/interface>

transport udp 2055

Interface <NameOfInterface>

  ip flow monitor <NetflowMonitorName> input

  ip flow monitor <NetflowMonitorName> output

0 Kudos

h.hendriks, I cannot apply ip flow monitor <monitorname> input nor output to the gigabit interfaces I want to monitor, my 3750x switch only supports those commands on fiber uplink (service module) ports. Is there no way to record netflow information from an interface when ip flow ingress and ip flow egress are applied?

0 Kudos

rschroeder, I'm trying to analyze/capture netflow from the gigabit ports on my 3750x. That don't support flexible netflow. I've had non-flexible netflow working on my gigabit ports at some point in the past.

Does the netflow analyzer only work with flexible netflow (one has to apply "ip flow monitor <name of flow monitor> input" on the specific interface they want monitored) now? The only netflow commands I can apply directly to the interfaces I want analyze are "ip flow ingress" and "ip flow egress".

0 Kudos

NTA supports both version 5 and 9, but I recommend using version 9 with NBAR2 everywhere you can.  Some legacy devices aren't compatible with NBAR2, others can only do Netflow v5.  Find which ones have that limitation and compensate for them, and request budget to replace them with newer models that support Netflow v9 and NBAR2.

I apply flow commands to every physical interface on my Cisco 4510 chasses now that the V8 model supports the commands, and it opens up another layer of granularity for traffic on a per-port basis.   In that particular environment, it's only possible to use the "ip flow monitor <name> input" command.  Initially I thought this was a limitation because there was no matching "output" command for the port.  It turns out that, while having both commands on the port seem intuitive and convenient, I'm really only interested in traffic coming "from" the device directly attached to the port.  Any traffic going "to" that device from another device is captured on the port(s) allowing the traffic into the switch from the other device.

Regarding your 3750x, getting its Netflow going again most likely will require a review of the required commands and a fine-toothed comb going through the details.  Although you had it working previously, since it's not working now, you may benefit from thinking about what's changed that caused it to stop.

  • Was there an IOS update or downgrade that resulted in different capabilities, or that needs different commands applied to get Netflow going again?
  • Did a destination address change for the Netflow?  If you updated/changed a Solarwinds Poller, it could still be polling the 3750x, but the 3750x might not be sending Netflow to the correct destination address.

If you have NCM, I'd recommend comparing a running-configuration from the 3750x at the time it was properly sending Netflow to today's running-config.  Maybe you'll see a change or a typo.  Or perhaps you'll find something that SHOULD have changed, but hasn't, to support a different destination address for a Solarwinds poller.

I looked for a 3750x in my network that was running the right code and license level to use Netflow and I find I've retired them all.

But here's a snip from one of my 4510's running Netflow on all interfaces that can be compared to your output:

flow record NTArec

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

collect interface output

collect counter bytes

collect counter packets

collect application name

flow exporter NTAexp

destination <x.x.x.x> (your Solarwinds APE running NTA)

source <enter the interface on the switch that will be recognized as the source of the traffic.  Always use the same Interface that is being polled by Solarwinds--usually an SVI or a loopback>

transport udp 2055

template data timeout 60

option application-table timeout 60

flow monitor NTAmon

description NetFlow nbar

exporter NTAexp

cache timeout inactive 30

cache timeout active 10

record NTArec

ip flow monitor NTAmon input (this command goes on every physical port)

vlan configuration (list all VLAN ID's here, comma-separated)

  ip flow monitor NTAmon input

You may have to tweak this a bit for your 3750x's, but it should get you very close to running again.

Swift packets!

Rick Schroeder

0 Kudos

This is the config I had/have on the switch, updated with the interface I want to monitor and the source ip of the new Netflow Analyzer. I have a feeling I'm getting tripped up on the ip flow-export source line, documentation did not make this clear. Right now I have it set as the interface I want monitored.

flow record <record name>

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

collect counter bytes

collect counter packets

flow exporter <exporter name>

destination <netflow analyzer IP>

transport udp 2055

flow monitor <monitor name>

description Original Netflow captures

record ipv4

exporter <exporter name>

interface <interface to be monitored>

ip flow ingress

ip flow egress

ip flow-export source <interface still not clear exactly this is for>

ip flow-export version 5

ip flow-export destination <netflow analyzer address> 2055

ip flow-top-talkers

top 10

sort-by bytes

0 Kudos

pastedImage_0.png

The "ip flow-export source" line tells the 3750x what IP address it should include as the "from" or "sender", when sending to your Solarwinds NTA poller.   For example, if you only have one IP address on the switch, and it's loopback0, then you'd say "ip flow-export source loopback0" on this line.

If your 3750x has multiple IP addresses, always use the interface with the IP address that's being monitored by Network Traffic Analyzer for the "ip flow-export source".  It helps Solarwinds NPM and NTA keep everything aligned nicely when you use the same monitoring address that NPM knows about, for the source interface in NTA.

If you don't do this, you'll be monitoring your switch in NPM with one IP address, and the switch will be sending Netflow information to NTA from an interface with a different IP address than the one NPM already is monitoring.  This will create an alert, and you'll be recommended to either add the new Netflow-associated IP address as an entirely new node (wasting license count and server resources), or you can simply change the "ip flow-export source" line to reference the Interface with an IP address that NPM monitors.

0 Kudos

show flow interface command results:

Interface <interface used to SSH into switch>

  FNF:  monitor:          NTAmon

        direction:        Input

        traffic(ip):      on

  FNF:  monitor:          NTAmon

        direction:        Output

        traffic(ip):      on

0 Kudos

sh flow exporter NTAexp command results:

Flow Exporter NTAexp:

  Description:              User defined

  Export protocol:          NetFlow Version 9

  Transport Configuration:

    Destination IP address: <netflow collector>

    Source IP address:      <switch IP used to SSH in>

    Source Interface:       <above IPs interface>

    Transport Protocol:     UDP

    Destination Port:       2055

    Source Port:            56488

    DSCP:                   0x0

    TTL:                    255

    Output Features:        Not Used

0 Kudos

rschroeder, that makes more sense than what I read. When I ran the ip flow-export source command I couldn't put in an address specifically, I had to put in an interface, so I used the interface I ssh into the switch on. That didn't work unfortunately, when I open up netflow analyzer I still get no flow type next to any of the interfaces.

I've also been working on getting the netflow configurator working. When I try to connect to my device using my read only SNMP community string, the software says I need a read/write community string to continue. I created a read/write SNMP community string, tried that in the software, it says cannot connect to device. Not sure what the issue is there either.

0 Kudos

I've been able to use Solarwinds' documentation for getting Netflow configured on a good variety of Cisco devices.  3750's, Nexus 7K's and 5K's, 4510's, and a bunch of routers.  I use this basic guideline, and tweak and tune the commands based on individual platform limitations or requirements, which can be found if you Google Netflow and that particular Cisco box.

Set up NetFlow NBAR2 on Cisco devices

Network Based Application Recognition (NBAR) is the mechanism used by certain Cisco routers and switches to recognize a dataflow by inspecting some of the packets sent. SolarWinds NTA 4.2.1 supports unknown traffic detection and advanced application recognition through NBAR2.

First, configure your Cisco devices to send NBAR2 data to SolarWinds NTA. Second, add those devices as nodes in SolarWinds NPM and SolarWinds NTA.

The following values are examples used in the commands below:

  • NTArec

  • NTAexp
  • NTAmon
  • GigabitEthernet0/1
  • 10.10.10.10

Create a new Flexible NetFlow configuration

Add the flow record

This process is similar to creating a standard NetFlow configuration. In this case, you add the collect application name command to enable the sending of AppID in each flow.

flow record NTArec

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

collect interface output

collect counter bytes

collect counter packets

collect application name

exit

Add the flow exporter

The option application-table command enables the sending of a list of applications that can be classified using NBAR2, including applications that were manually created. The option application-attributes command enables the sending of categories for all applications.

flow exporter NTAexp

destination 10.10.10.10

source GigabitEthernet0/1

transport udp 2055

export-protocol netflow-v9

template data timeout 60

option application-table timeout 60

option application-attributes timeout 300

exit

Add the flow monitor

The flow monitor connects the flow recorder and the flow exporter. You can configure multiple recorders, exporters, and monitors at once.

flow monitor NTAmon

description NetFlow nbar

record NTArec

exporter NTAexp

cache timeout inactive 30

cache timeout active 60

exit

When receiving long flows, these values may need to be adjusted, see Troubleshoot Long Flow Errors for more details. For more information about the timeout values, refer to the Cisco NetFlow Command Reference.

Apply the monitor on an interface

Assign the Flexible NetFlow configuration to the interface from which to monitor NetFlow.

interface GigabitEthernet0/1

ip flow monitor NTAmon input

ip flow monitor NTAmon output

exit

Diagnostic commands

show flow record "recordName"

show flow export "exporterName"

show flow monitor "monitorName"

show flow exporter statistics

show flow interface

Determine the applications your device can recognize

The Protocol Pack is a list of applications, definitions, and categories that your device can recognize.

Check the Protocol Pack version

show ip nbar version

View a list of the available applications

show ip nbar protocol-id

Edit an existing record

If you edit an existing record that is in use, you receive the following error:

% Flow Record: Flow Record is in use. Remove from all clients before editing.

To resolve this error, remove the connection between the monitor, record, and interface.

Disable the connection

interface GigabitEthernet0/1

no ip flow monitor NTAmon input

no ip flow monitor NTAmon output

exit

Add the application recognition field into the record

flow record NTArec

collect application name

exit

Add the application recognition field into the exporter

flow exporter NTAexp

option application-table timeout 60

option application-attributes timeout 300

Restore the connection

interface GigabitEthernet0/1

ip flow monitor NTAmon input

ip flow monitor NTAmon output

exit

So let's say you have a Cisco 4510.  Here's my copy-and-paste instructions, minus the unique IP addresses or interfaces you need to add:

How To Set Up Netflow on Cisco 4510 Version 8 Chasses:

  1. The switch hardware must be Version 8 or newer.  V7 and older requires NetFlow Modules to be purchased and installed in each Supervisor.
  1. The chassis must be licensed to run IP Base or Enterprise.  NetFlow is not supported on LAN Base license.

conf t

flow record NTArecord

match ipv4 tos

match ipv4 protocol

match ipv4 source address

match ipv4 destination address

match transport source-port

match transport destination-port

match interface input

collect interface output

collect counter bytes

collect counter packets

collect timestamp sys-uptime first

collect timestamp sys-uptime last

!

flow exporter NTAexport

destination x.x.x.x (You add in your SW Poller's address here)

source Loopback0 (Or use a different interface--whatever you use to manage the switch is the interface to report with)

transport udp 2055

export-protocol netflow-v5

!

flow monitor NTAmonitor

description NetflowToOrion

exporter NTAexport

cache timeout inactive 10

cache timeout active 5

record NTArecord

Add “ip flow monitor NTAmonitor input“ to every VLAN you want included.  You can also group them via this example:

vlan configuration (Insert ALL the VLAN's on the 4510 in this area)

ip flow monitor NTAmonitor input

On the WAN interface's physical port(s):

ip flow monitor NTAmonitor input

Add this line for EVERY physical port to want to monitor on the switch: 

ip flow monitor NTAmonitor input

! Modify the interface script that follows based on the modules you own:

conf t

int range gi1/1-48,gi2/1-48,gi3/1-48,gi4/1-48

ip flow monitor NTAmonitor input

int range gi7/1-48,gi8/1-48,gi9/1-48,gi10/1-48

ip flow monitor NTAmonitor input

Then tell the switch which interface to use as its Netflow source.   A 4510 serving as a WAN router and Distribution switch should use a loopback port, but you could choose the physical WAN interface.  Use the same port as is used by the switch for all its sourcing of logging, TACACS, snmp, etc.

Build the exporter, then assign it to the correct Interface so Orion doesn’t throw a bunch of errors about an unmanaged device sending it Netflow info.

Example: 

conf t

flow exporter NTAexport

description LSEG internal

destination (x.x.x.x is the IP address of your Solarwinds Poller)

source Loopback0

transport udp 2055

export-protocol netflow-v5

int loopback0

flow monitor NTAmonitor

exporter NTAexport

record NTArecord

Finally, ensure NPM is set to monitor all interfaces that have the “ip flow monitor NTAmonitor input“ command.  If it’s not, then it’ll send NTA interface errors.

Removal is the reverse of the steps above, in this order:

int loopback1

no flow monitor FLOW-MONITOR-1

no exporter EXPORTER-1

no record NTArecord

no flow exporter EXPORTER-1

int range gi1/1-48,gi2/1-48,gi3/1-48,gi4/1-48

no ip flow monitor NTAmonitor input

int range gi7/1-48,gi8/1-48,gi9/1-48,gi10/1-48

no ip flow monitor NTAmonitor input

int range te5/1-8,te6/1-8

no ip flow monitor NTAmonitor input

vlan configuration x-x

no ip flow monitor NTAmonitor input

no flow monitor NTAmonitor

no flow exporter NTAexport

no flow record NTArecord

Now let's suppose you had to do this on a 6509 Core or Distribution L3 switch.  Here's how:

Enabling Netflow on 6509 Distribution Switches

ip flow-cache entries 131072 (if you change this, the switch must be 

rebooted or all flow must be removed before it takes effect)

ip flow-cache timeout active 1

ip flow ingress layer2-switched vlan x (must be done for every vlan)

mls flow ip interface-full

no mls flow ipv6

mls nde sender version 5

**VLAN/physical interface's**

!  int vlan 2 (etc.  must be done for every SVI)

ip flow ingress

ip route-cache flow

ip flow-export source lo0

ip flow-export version 5

ip flow-export destination x.x.x.x (this is the address of your Solarwinds server)  2055

Let's say you want your ASA to report Netflow.  It's super easy:

flow-export destination  ABCD  (the name of the ASA Interface that you want to send the Netflow traffic through--it might be really intuitive like "inside")   x.x.x.x (the IP address of your Solarwinds poller) 2055

So you have 3750X's.  Are they compatible with NetFlow?

pastedImage_13.png

If they ARE compatible, I recommend you use Solarwinds' Netflow configuration guidance.  But you can also refer to Cisco's info here:

Catalyst 3750-X and 3560-X Software Configuration Guide, Release 15.0(1)SE - Configuring Flexible Ne...

Good Luck!  Let us know how it works out for you!  Send pictures--or it didn't happen!

pastedImage_16.png

We have a catalyst 6500  core and distribution with a Sup 2T.

The Sup 2T has the flexibele netflow commands.

An other 6500 switch with sup 32 has the ' old' version 5 commands.

With the CAT6500 it's imported to notice witch supervisor you have.

0 Kudos