17 Replies Latest reply on Nov 22, 2010 10:57 AM by mazydizzle2K10

    NETFLOW information not being received on 1 interface

    mazydizzle2K10

      Hi all,

      i have a cisco 6509 switch with several L3 interfaces all sending Netflow data to our Orion server.

      I've added a new L3 interface, added the ip flow ingress/egress commands to the interface and I only seem to ever receive netflow data for about 1 minute at around 23:27 at night. It's really wierd, all other interfaces send data every minute of the day.

      The only difference being is that the IP assigned to this interface is a public IP rather than private IPs that all exist in our LAN.

      I have added this IP range in to the Netflow settings in Orion and I am seeing the "Data Last received" column as Never.

      I did not set this all up myself, i have taken this on from a previous engineer.

      Any help appreciated! I've done some searches in the forums but all the posts I have come accross are from users where NetFlow is not being recieved at all.

      My problem is different where it is only a newly added L3 VLAN interface that is not sending netFlow data. All other interfaces are working fine.

      I have simply coied the netflow interface config from a working interface in an attempt to get this netflow stream working.

      thanks again!

       

      Mario De Rosa

        • Re: NETFLOW information not being received on 1 interface
          Andy McBride

          have you added that new interface to NPM?

          • Re: NETFLOW information not being received on 1 interface
            mazydizzle2K10

            Hi,

            These are the steps of what I did

            1. create L3 interface and IP addressing

            2. add ip flow ingress / egress commands

            3. added new L3 int in NPM to be managed

            4. In NetFlow settings, selected new L3 int to receive netflow traffic.

            All interfaces are sending netflow data every minute, where as this new interface, it is advising that the last time netflow data was received was 23:27 at night.

            Is there anything that I have missed do you think??

             

            Mario

              • Re: NETFLOW information not being received on 1 interface
                pyro13g

                show ip cache flow  at the switch.

                if you see any DstIf as null than netflow won't be created for that flow.  Probably process switched traffic.

                  • Re: NETFLOW information not being received on 1 interface
                    mazydizzle2K10

                    thanks for the reply.

                    i ran the following

                    sh ip cache flow | i vl198 and nothing appeared. I think the guys have stopped using that VLAN for the day now so I just got your message to late.

                    how long are these entries listed for?

                    also, do you know if there are any solar winds logs that i can inspect to get an idea if netflow data is being dropped by our NPM server?

                    thanks

                    Mario

                      • Re: NETFLOW information not being received on 1 interface
                        mazydizzle2K10

                        OK,

                        forget what I just said... the command was wrong. The good news is that the DstIf column is not null. I am running a continuous ping from a remote site to this L3 interface and the output is below...

                        TH_DC_core_02#sh ip cache flow | i Vl198
                         Vl198            67.226.192.234   Vl96             10.130.12.46    00 0000 0000   163
                         Vl91             10.130.12.46     Vl198            67.226.192.234  00 0000 0000   163

                        Hope this output helps!

                        Mario

                          • Re: NETFLOW information not being received on 1 interface
                            mazydizzle2K10

                            haha! i'm getting there!...

                            right, i can now see netflow data of the pings that I am running!!

                            So, you are correct pyro, it seems like the Video Conferencing traffic that this VLAN is used for is being process switched.

                            It's a pitty I cannot check at the moment as no one is using the system but I imagine that if i run the same command whilst there is a video conference going on, the DstIf would be null???

                            Time for me to read my CCNP books to refresh my memory about CEF etc...

                            Once i confirm what you advised about our video traffic being processed switched, i will consider the question answered!

                            thanks!!!

                            Mario

                              • Re: NETFLOW information not being received on 1 interface
                                mazydizzle2K10

                                Hi Pyro,

                                sorry to be the bearer of bad news but it appears that I can see traffic flows whilst a video conference is in progress. The output is below...

                                TH_DC_core_02#sh ip cache flow | i Vl198
                                 Vl12             67.226.192.238   Vl198            67.226.192.234  00 0000 0000   201K
                                 Vl198            67.226.192.234   Vl12             67.226.192.238  00 0000 0000   314K
                                TH_DC_core_02#sh ip cache flow | i Vl198
                                 Vl12             67.226.192.238   Vl198            67.226.192.234  00 0000 0000   210K
                                 Vl198            67.226.192.234   Vl12             67.226.192.238  00 0000 0000   331K
                                TH_DC_core_02#sh ip cache flow | i Vl198
                                 Vl12             67.226.192.238   Vl198            67.226.192.234  00 0000 0000   344K
                                 Vl198            67.226.192.234   Vl12             67.226.192.238  00 0000 0000   590K

                                Looking in NTA, it still says that the last data received is 23:27 last night.

                                Do you have any other ideas of what it could be?

                                Any more help appreciated!

                                Mario

                                  • Re: NETFLOW information not being received on 1 interface
                                    dusak

                                    Looks like this is what I experience as well, and still looking for solutions.

                                    I am using Cisco 7209, I can see other interface netflow but one particular interface no netflow at all (confirm already add the command in the interface)

                                    an on that interface, I can see it's being used since Bandwidth utilization above 10% only no netflow.

                                    Hope someone can help with an ans

                                     

                                    yamin

                                    • Re: NETFLOW information not being received on 1 interface
                                      Andy McBride

                                      The protocol and port are zeros, indication L2 traffic only.

                                        • Re: NETFLOW information not being received on 1 interface
                                          dusak

                                          Hi Andy,

                                           

                                          so what need to be configure to monitor netflow on the interface then ?

                                            • Re: NETFLOW information not being received on 1 interface
                                              Andy McBride

                                              I've never set up flow export for L2 and we don't have any L2 specific resources so I'm not sure if you would be able to see much about non-routed traffic. Check out the Tech Ref and see if your config matches the samples.

                                              http://www.solarwinds.com/support/Netflow/docs/OrionNetFlowSwitches.pdf

                                                • Re: NETFLOW information not being received on 1 interface
                                                  mazydizzle2K10

                                                  Hi Andy,

                                                   

                                                  I dont understand how my output shown above is L2 traffic. Its a L3 interface with an IP assigned to it.

                                                  If I run a continuous ping from another network in to this L3 interface, I get a different output.

                                                   

                                                  Mario

                                                    • Re: NETFLOW information not being received on 1 interface
                                                      mazydizzle2K10

                                                      Hi Andy,

                                                      i have been running a continuous ping from a host in another subnet to a host in this L3 VLAN in question and below is the output...

                                                      TH_DC_core_02#sh ip cache flow | i Vl198
                                                       Vl198            67.226.192.234   Vl96             10.130.12.46    00 0000 0000   625
                                                       Vl91             10.130.12.46     Vl198            67.226.192.234  00 0000 0000   326

                                                      The output is actually the same however this time, NetFlow is recieving this data! I can happily see details about the ICMP packets going in and out of this L3 VLAN now.

                                                      Is there something about the video traffic that cannot be processed by NetFlow? is there any way that I can find out??

                                                      Thanks for you help so far Andy.

                                                      Mario

                                                        • Re: NETFLOW information not being received on 1 interface
                                                          Andy McBride

                                                          There is no L3 protocol being seen by the collector. That is the 00 column.

                                                            • Re: NETFLOW information not being received on 1 interface
                                                              mazydizzle2K10

                                                              Hi Andy,

                                                              I'm confused at the moment. If i run a continuous ping from host to host, I get the same output from the sh ip cache flow (00) BUT, NTA DOES recieve the data about the icmp traffic.

                                                              However, then there is a video conference in place, no traffic is received what so ever in NTA yet the output of sh ip cache flow is always the same, 00.

                                                              Is there something about the video traffic that netflow cannot handle and how can I find that out?

                                                              thanks

                                                              Mario

                                                              • Re: NETFLOW information not being received on 1 interface
                                                                Donald_Francis

                                                                Just throwing my 2 cents in here.  if you have a busy chassis it is very easy to overrun the TCAM's especially if you also use WCCP as WCCP and netflow compete for the same table space.

                                                                When the TCAM is full netflow processing gets booted to CPU which if there is a ton of it is bad and I think in there somewhere stuff can get dropped.

                                                                You can run the following command to see where the utilization is at.

                                                                sh mls netflow table-contention summary 

                                                                There are default aging parameters for netflow to try to keep that TCAM clean but they are very lax and I know in my environment I had to get much more agressive especially since we use apps that have lots of tiny flows like IM for instance.

                                                                  • Re: NETFLOW information not being received on 1 interface
                                                                    mazydizzle2K10

                                                                    There is a conference call in place now. TCAM is not over utilised and again, the output below is showing no L3 protocol information.

                                                                    TH_DC_core_02#sh ip cache flow | i Vl198
                                                                     Vl198            67.226.192.234   Vl12             67.226.192.238  00 0000 0000   267K
                                                                     Vl12             67.226.192.238   Vl198            67.226.192.234  00 0000 0000   246K


                                                                    TH_DC_core_02#sh mls netflow table-contention summary
                                                                    Earl in Module 1
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   3%
                                                                    ICAM Utilization             :   0%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0
                                                                    Earl in Module 2
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   2%
                                                                    ICAM Utilization             :   0%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0
                                                                    Earl in Module 3
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   2%
                                                                    ICAM Utilization             :   0%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0
                                                                    Earl in Module 5
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   12%
                                                                    ICAM Utilization             :   9%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0
                                                                    Earl in Module 7
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   1%
                                                                    ICAM Utilization             :   3%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0
                                                                    Earl in Module 8
                                                                    Summary of Netflow CAM Utilization (as a percentage)
                                                                    ====================================================
                                                                    TCAM Utilization             :   0%
                                                                    ICAM Utilization             :   0%
                                                                    Netflow Creation Failures    :   0
                                                                    Netflow CAM aliases          :   0

                                                                    TH_DC_core_02#