Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Giving up on NTA - Last Ditch Efforts

Hello there, here is an outline of my situation:

1. I am attempting to collect NetFlow on a Cisco 2811 router and a Cisco 2821 router. The routers and the SolarWinds server/collector (Server 2008) are on the same subnet.

a. There are ACL's specifically in place on the routers to allow for NetFlow to reach the collector. These have been examined by Support.

b. There are firewall exceptions in place on the SolarWinds server/collector to allow for all necessary UDP and TCP ports. These were added by Support.

2. I have spent well over 20 hours with support on the phone, through e-mail, and in GoToMeetings. We have exhausted the possibility of SolarWinds causing the issue.

a. SolarWinds recognizes NetFlow on UDP 2055 when generated from other sources (using a generic generator).

b. Traffic In, Traffic Out, and Last NetFlow Received all display data for GigabitEthernet 0/0, 0/1 (Cisco 2821) and FastEthernet 0/0, 0/1 (Cisco 2811).

3. WireShark shows incoming NetFlow on UDP 2055 when generated from other sources. WireShark shows no incoming NetFlow on UDP 2055 when generated from the 2811/2821.

4. I have used the NetFlow Configurator utility. I have used the newly-released Config Generator w/NetFlow template to generate the appropriate script (which was consequently uploaded via NCM).

5. Using show ip flow export lists no failures for NetFlow v5, but the flows never reach the collector.

If there is something here that I am missing, or if there is someone in the community that could help me step through the process of enabling NetFlow correctly on a 2811/2821, please help. I have reached the end of my troubleshooting skills for the third time now and would still like to have this running/configured as soon as I possibly can.

Thank you in advance for any help or suggestions,

Jeremy

Find more posts tagged with

Accepted answers

jalexander

The issue has been resolved. After some investigative work (remember, I'm just a programmer) I found that the traffic moving from the router to the collector does, indeed, pass through another piece of hardware (thank you for the many posts suspecting this). The traffic passes through a Panda Gatekeeper Performa.

I added several exceptions to allow for this traffic to pass through unscathed, but to no avail. I contacted Panda support and we dinked around with some advanced settings. The culprit ended up being some advanced peer to peer protection protocols that were put into place.

While this seems like a rather convincing argument for me NOT to be handling our organization's SolarWinds installation, I do appreciate all of the helpful feedback and willingness to aid me in my network stumbling. This has been a great learning experience, and I would still not have NetFlow sending if it weren't for the many thought processes I've explored after reading through your comments.

So thank you again, and apologies for not having a more mystifying solution to my issues.

All comments

kweise

Jeremy,

There isn't a lot to the NetFlow config on the 2800s.

ip flow-export source <interface> - the key here is you have to source NetFlow from the same interface that has the IP address of the node in Orion. For example, if the node was added to Orion using the IP address on the serial interface, you have to set the ip flow-export source to that serial interface.

ip flow-export version 5

ip flow-export destination <ip address of your Orion server> 2055

ip flow ingress or ip flow egress on your interface config(s)

Hope this helps. If not, let us know, maybe there is something else we can take a look at.

njoylif

to piggy back "kweise", we use Loopbacks for orion to manage nodes, so we have to source netflow from loopback.

secondly, wondered if you tried sniffing by the router to see if it was outputting netflow..could be bug in IOS maybe?
maybe do packet capture on each interface side of the firewall to see if coming in and going out through firewall...

good luck

MarieB

Hi Jeremy--

Sorry you are having trouble. I'm going to route this over to the NTA product manager and development as well to see if they can help.

jalexander

First, I have never posted to thwack before, and I am quite impressed by the speed and number of replies. Regardless of solving the problem, this is a very comforting thing.

Secondly, as an example I'll use the Cisco 2811.

The IP Address of the Cisco 2811 router is 172.16.100.247. The address on the 0/0 interface is 172.16.100.247. I am using this interface as the NetFlow source. Ingress and Egress have both been enabled on 0/0 and 0/1. Is this correct? Only the 0/0 interface is relevant, as both of these routers move traffic from outside the organization to inside (predominantly Internet-activities).

Fianlly, while I would gladly try out what njoylif has to say, I'm afraid I'm a lowly programmer and not that familiar with the tools available to me in evaluating this situation. I do understand a lot of the basics, but would appreciate any additional information on how to accomplish the two tasks you suggested.

I would also like to thank MarieB for the forward.

njoylif

on the (cisco) firewall, you should be able to do what is called a capture. You will have to define an interesting traffic ACL and apply that to the targeted interface. You can run multiple on the device, one per interface.

your interesting traffic can be very specific or very general...since you know the specifics - it might look something like this...
access-list dmzside permit udp host 172.16.100.247 host x.x.x.x (Netflow server IP) eq 2055

**** this assumes you are familiar with NAT and whether or not you guys are doing that...

you would apply that to the interface facing the router by doing something like:
capture <capname> access-list dmzside interface <interface nameif>

This will not affect production traffic <cya> assuming there is not a TON of data to flood the link and it is configured correctly</cya>

you would repeat steps for inside interface (side to get to netflow system). DO NOT use the same ACL. even if same, create another and use different name.

good luck.

viperar15

have you verified that the NetFlow Collector Service is "UP" on the NetFlow Page for NPM?

Should be something like this...

ServerName Status Port
Server1 Up 2055

If it is not.... you have a problem on NTA Collector itself.

The other thing might be a Windows Firewall Problem or IPSEC if you're using that.

mcbridea

Jeremy,

can you post the output of sho ip flo exp and sho ip cache verbose flo

Andy

jalexander

I have not yet contacted Cisco TAC and the NetFlow Collector Service is UP. I can ping 172.16.121.22 (the collector) from the router. The Windows Firewall on the collector has all of the necessary exceptions (as verified by support; also verified by successfully sending generic NetFlow data to the collector from an unmanaged device) Thanks to everyone for all of the feedback. Also, please let me know if more information is required from the verbose command.

Show IP Flow Export yields:

Flow export v5 is enabled for main chache
Exporting flows to 172.16.121.22 (2055)
Exporting using source interface FastEthernet0/0
Version 5 flow records
64401130 flows exported in 2151625 udp datagrams
0 flows failed due to lack of export packet
0 export packets were sent up to process level
0 export packets were dropped due to no fib
0 export packets were dropped due to adjacency issues
0 export packets were dropped due to fragmentation failures
0 export packets were dropped due to encapsulation fixup failures

sho ip cache verbose flo yields:

IP packet size distribution (1543M total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .431 .023 .005 .004 .003 .003 .005 .004 .003 .004 .003 .002 .003 .002

512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.003 .002 .053 .022 .415 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 278544 bytes
275 active, 3821 inactive, 81426648 added
933428740 ager polls, 0 flow alloc failures
Active flows timeout in 1 minutes
Inactive flows timeout in 15 seconds
IP Sub Flow Cache, 21640 bytes
275 active, 749 inactive, 64409336 added, 64409336 added to flow
0 alloc failures, 12615 force free
1 chunk, 4169 chunks added
last clearing of statistics never
Protocol         Total    Flows   Packets Bytes Packets Active(Sec) Idle(Sec)
--------         Flows     /Sec     /Flow /Pkt     /Sec     /Flow     /Flow
TCP-Telnet        3703      0.0        31   247      0.0      13.5      12.9
TCP-FTP         115336      0.0         3    54      0.0       0.6      15.0
TCP-FTPD           136      0.0       391   893      0.0       2.2       2.9
TCP-WWW       63715701     14.8        19   717    288.6       2.2       6.4
TCP-SMTP          7554      0.0         7    85      0.0       1.8       3.3
TCP-X              179      0.0         6   716      0.0       0.6      14.4
TCP-NNTP             1      0.0         1    40      0.0       0.0       1.1
TCP-Frag           279      0.0        30    27      0.0      11.2      15.5
TCP-other      7022222      1.6        37   690     60.7       3.6       6.5
UDP-DNS         257883      0.0         1    66      0.0       1.7      15.4
UDP-NTP         351078      0.0         1    76      0.0       0.0      15.4
UDP-TFTP         12011      0.0         5    49      0.0      20.0      15.4
UDP-Frag           931      0.0         2   660      0.0       0.1      15.6
UDP-other      9742115      2.2         4   238      9.5       1.9      15.4
ICMP            196462      0.0         1    68      0.0       0.4      15.5
Total:        81425591     18.9        18   699    359.3       2.3       7.6

SrcIf SrcIPaddress DstIf DstIPaddress Pr TOS Flgs Pkts
Port Msk AS Port Msk AS NextHop B/Pk Active

mcbridea

Router is caching and exporting properly and you previously demonstrated that the NTA collector sees NTA data. The only possibility is that there is something in the path from the exporting routers to the collector dropping the flow data. I edited out the unnecesary portion of the verbose cache output to help make this more readable.