9 Replies Latest reply on Dec 9, 2015 9:07 AM by knowram

    L3 Netflow from Nexus

    knowram

      I am trying to get Layer 3 netflow working from a nexus 7706 running 6.2(10). I have tried using version 9, specifying a source interface, and creating a custom flow recorder and nothing seems to work. My configurations are as follows Thank you.

       

      flow timeout active 60

      flow exporter Netflow-Exporter-Prod

        description Production-Netflow-Exporter

        destination {NTA IP Address}

        transport udp 2055

        version 5

      ip sla responder

      sampler NF-Sampler-Prod

        description Netflow-Prod-Sampler

        mode 1 out-of 1000

      flow monitor Netflow-Monitor-Prod

        description Use Predefined "Original-Netflow-Record"

        record netflow-original

        exporter Netflow-Exporter-Prod

       

       

      interface Vlan918

        ip flow monitor Netflow-Monitor-Prod input sampler NF-Sampler-Prod

        ip address IP Address/30

        ip directed-broadcast WOL

        ip ospf authentication message-digest

        ip ospf message-digest-key 1 md5 7 03405803565F0D

        ip router ospf 1 area 0.0.0.100

        ip pim sparse-mode

        description AS-P2P to CC106-MDF-sw01

        no shutdown

        • Re: L3 Netflow from Nexus
          Craig Norborg

          Try setting the flow-source in your "flow exporter" record.   This interfaces IP address should be the same as what you have in NPM for managing the device.

           

          There is a Nexus 7000 section in this document also...

          http://www.solarwinds.com/documentation/NetFlow/docs/NetFlowDeviceConfiguration.pdf

            • Re: L3 Netflow from Nexus
              knowram

              Craig,

               

              Thanks for the info using I was sure I tried that however NTM now seems to be receiving netflow info however I am getting the following error receiving flow data from unmanaged interface '#160'

              Do you have any ideas on that one? When I view the inventory of the nexus the vlan interfaces are listed and I also getting this error for more interfaces then I have enabled netflow on.

                • Re: L3 Netflow from Nexus
                  Craig Norborg

                  Yes, that means that its seeing Netflow, but its from an unmanaged interface, or one that NPM doesn't know about.  By default NTA doesn't monitor all sources of Netflow packets that it receives, so you either need to add it manually, or tell Netflow to add all sources.

                   

                  Go to your NTA settings in Orion, at the top of the page there should be a line that says "Enable automatic addition of NetFlow sources", its probably unchecked, your first option is to check this.  Why isn't it enabled already?   There might be devices that are sending netflow for interfaces you don't care about, so by default it doesn't automatically add them to NTA.

                   

                  Second choice is to manually edit it.  Further down on this same page you will find a section labelled "Netflow Sources" with the description "if you have disabled the automatic addition of Netflow Sources your will need to manually add and remove Netflow interfaces".    You can do that by clicking on the link "Manually manage Netflow sources and CBQoS Polling" here.

                   

                  Another possibility is that you don't have the interface managed in NPM.   #160 is the SNMP interface index, you can find out what interface this corresponds to on your device by logging into it and issuing the "show snmp mib ifmib ifindex" command.   Look for one with the ifIndex of 160 and make sure that interface is monitored in NPM.   Or, alternatively, you can go to that interface and disable netflow for it by removing "ip flow ingress/egress" on the interface.

                   

                  Lastly, if Netflow is exporting with a different IP than what the device is managed, like I mentioned first, you could get this error because it doesn't recognize the device too I think.   But you might get a "doesn't recognize the device" error instead...  Not sure on this one to be honest.   The solution would be to set your source interface correctly for netflow.

                    • Re: L3 Netflow from Nexus
                      knowram

                      Craig again thanks for the info however still no progress has been made.

                       

                      "Enable automatic addition of NetFlow sources" was already check


                      I have manually configured the 2 vlan interfaces I added the netflow config to on my nexus.


                      The command "show snmp mib ifmib ifindex" isn't available on the nexus. I did find sh interface snmp-ifindex that seems to give interface snap index numbers but they look like the following.

                      Vlan913         151061393  (0x9010391)

                      Vlan915         151061395  (0x9010393)

                      Vlan917         151061397  (0x9010395)

                      Vlan918         151061398  (0x9010396)


                      And I verified that the netflow source interface I am using is the same ip address as the one NPM is using to monitor the device.


                      The odd thing is that if i look at Netflow Sources the last received netflow for the nexus is never however if i look at Top 10 netflow sources by % one of the vlan interfaces on the nexus shows up in that list.

                • Re: L3 Netflow from Nexus
                  zboyal

                  Hi,

                   

                  I experienced the same issue, what causes this is that you enabled netflow and sent it out an unmanaged interface. The NTA database will have issue linking the ip to the main node.There's two ways to fix this disable netflow on the device and interfaces. Then list resources on the devices and manage all interfaces and re enable netflow.

                   

                  or

                   

                  You can use the managed sources and uncheck the device and then re check it.. Either way you will lose data so it makes sense to manage all interfaces you may use as your source on a device prior to enabling netlfow.

                   

                  To add to this I had a few issues myself trying to get nexus netflow enabled and working right.We are using Version 9.


                  This link helped. - http://www.irisns.com/how-we-got-reliable-sampled-netflow-analyzer-data-from-ciscos-nexus-7k/

                   

                  Flow exporter NetFlow-Orion:

                      Destination: (NTA collector)

                      VRF: management (1)

                      Destination UDP Port 2055

                      Source Interface mgmt0 (your mgmt ip or any interface being monitored)

                      Export Version 9

                          Sampler-table timeout 60 seconds

                          Interface-table timeout 60 seconds

                          Exporter-stats timeout 60 seconds

                          Data template timeout 60 seconds

                      Exporter Statistics

                          Number of Flow Records Exported 1537438

                          Number of Templates Exported 23120

                          Number of Export Packets Sent 92394

                          Number of Export Bytes Sent 83766724

                          Number of Destination Unreachable Events 0

                          Number of No Buffer Events 0

                          Number of Packets Dropped (No Route to Host) 0

                          Number of Packets Dropped (other) 0

                          Number of Packets Dropped (LC to RP Error) 0

                          Number of Packets Dropped (Output Drops) 0

                  sh sampler

                  Sampler: netflow-v9

                      ID: 196609

                      mode 1 out-of 100

                  sh flow record

                  Flow record netflow-original:

                      Description: Traditional IPv4 input NetFlow with origin ASs

                      No. of users: 1

                      Template ID: 256

                      Fields:

                          match ipv4 source address

                          match ipv4 destination address

                          match ip protocol

                          match ip tos

                          match transport source-port

                          match transport destination-port

                          match interface input

                          match interface output

                          match flow direction

                          collect routing source as

                          collect routing destination as

                          collect routing next-hop address ipv4

                          collect transport tcp flags

                          collect counter bytes

                          collect counter packets

                          collect timestamp sys-uptime first

                          collect timestamp sys-uptime last

                  Flow record Netflow-orion:

                      No. of users: 0

                      Template ID: 0

                      Fields:

                          match interface input

                          match interface output

                          match flow direction

                   

                  sh run | i flow

                  feature netflow

                  flow timeout 30

                  flow timeout active 60

                  flow timeout inactive 30

                  flow exporter (name of flowexporter)

                  flow record Netflow-orion

                  sampler netflow-v9

                  flow monitor flowmon-v9

                    record netflow-original

                    ip flow monitor flowmon-v9 input sampler netflow-v9

                   

                   

                   

                  I also needed to update my Sample rate under managed sources to get accurate usage to display. e.g our core device is 1-100 and data center is 1-10000

                  hope this helps.

                   

                  thanks

                    • Re: L3 Netflow from Nexus
                      knowram

                      Thanks zboyal for the info however my experiences are different.

                       

                      Ghosts in the machine. I had thought I had done everything you mentioned countless times however this time around when I go to managed sources interfaces are showing up for the nexus I have been trying to get set up. Now my issue is that it even though I have only enabled netflow on the inbound direction on 2 vlan interfaces several interfaces keep popping up in Solarwinds as sources. does solarwinds track outbound instead of inbound or something?

                    • Re: L3 Netflow from Nexus
                      zboyal

                      Speaking to my network guys basically an input in the nexus is an output from another..

                       

                      I m sure you already checked this but I found a few issues with our DNS that really helped improve my settings. We had multiple entries for the same device. This can cause your issue as Netflow does DNS lookup up and depending which entry comes back it can show up as not managed. If you keep having issues try disabling Netflow on the device do a full node rebuild and make sure all your interfaces are monitored before you re-enable Netlfow. I did this at least once.

                      • Re: L3 Netflow from Nexus
                        knowram

                        For future reference I had to run the following command as i also had DHCP forwarders configured on the L3 interface

                         

                        Hardware access-list resource feature bank-mapping