Load balancing of NTA collection - a report needed

patriot

I have a logistical challenge that involves at least a report, and possibly a script to help me make some changes to the way flow data is being collected.

We have 5 APEs and my intention was to configure each NetFlow source device so that the flow data would be sent to the same polling engine being used by NPM to poll that node. My auditing indicates that may not have happened in most cases. So, my initial need is to have a report that shows the hostname and IP address of all nodes that are designated as an NTA source, the current destination of the flow data and the assigned polling engine for the node. I further want to have the option of filtering the report to show only nodes where the destination and polling engine do not match (but I want to be able to run the report with and without this filter in place).

Now for the "really cool to have" option. Once the report identifies those NTA sources that do not have the flow destination and polling engine that are matched, I want to have an automated way to change the polling engine. I thought that changing the polling engine for a node would be a lot less trouble than editing the device configuration to match the assigned polling engine. Of course, I will have to keep an eye on the load balancing stats for the polling engines.

Can anyone help with these requests. I will be very appreciative. Thank you.

rschroeder

OK, it sounds fun.  I do NOT have your solution, but your request brought to mind some similar items for which I've built some reports and workarounds.

1. Synchronizing Nodes' polling with their Netflow destinations.  Not too hard.

I, too, have five APE's.  I have my nodes distributed to those APE's by region or by poller-resource/availability.  I want all the nodes sending their syslog and Netflow data to the poller that monitors them.   I had old / duplicated logging settings and Netflow settings in many nodes.  I decided it was best to clean everything up that is obsolete using NCM jobs/scrips to make it simple.

1. 1. Go to NPM's Settings > Manage Nodes.  On the left, Group by Polling Engine.  Select a polling engine to see which nodes it manages.  Remove any nodes you don't want it to manage.  Do this for all polling engines  If grouping by polling engine doesn't make life easy, then group by Location, or Custom Property, or whatever you like.  Get to where you can see and select the nodes you want managed by a specific polling engine, then select them and go to More Actions, then select Change Polling Engine.  Do this for all nodes, all polling engines.  Once complete, you know which nodes are monitored by each poller.
   2. Open up NCM (My Dashboards > Configs > Configuration Management).
   3. Choose Group By whatever gets you the nodes you want selected.  Select them.  It would be SO wonderful to be able to sort / group the nodes by the Polling Engine that polls them, but this doesn't seem to be an option.
   4. Click execute Script
   5. Build a script that tells them to:
      1. Remove any other syslog destinations (no logging host x.x.x.x).  Add a new line for every possible syslog incorrect destination (no logging host z.z.z.z, etc.)
      2. Remove any other Netflow destinations  (this is where your previous consistency in selecting interfaces and naming them helps you do bulk jobs)
      3. Add into the script the new syslog information they should use (logging host a.a.a.a)
      4. Add into the script all the Netflow settings appropriate for these nodes using the standard interface selection and naming convention you previously deployed
      5. Determine what else needs to be removed and remove it
      6. Determine what else needs to be added and add it in
      7. Execute the script in NCM against a test switch, a test router, a test ASA, etc.  Don't do this in Prod if it risks any access.
      8. If it works there, test with a larger subset of your nodes.
      9. If it passes all your tests and syslogs and Netflow are going to the correct pollers, deploy it to all the nodes for that poller.
   6. Repeat this process for every pollers' nodes.
   7. Verify by building a Compliance Report that searches for incorrect syslog or Netflow information and alerts you on it.  You can even use the Compliance Report's Remediation feature to remove those incorrect settings and install new ones that meet your needs.
   8. Update the Compliance Report nodes and check which ones fell through the cracks.  You might even do this as your first step, since you can combine all the items in my Step 1 into a remediation / configuration script tied with a Compliance Report. Just be careful to do what needs to be done, to the right nodes, and nothing more.  NCM is powerful, and if you tell it to do something inappropriate, it'll do it. Your competence and trustworthiness are on the line when you have NCM rights

2. Next you asked for a Report  that shows the hostname and IP address of all nodes that are designated as an NTA source, plus the current destination of the flow data, and the assigned polling engine for the node.  So build that report.  You could get the first two items manually with an NCM script with anything you want to discover about the node:

• show run | in logging
• sho run | in section flow   (but if you monitor all 384 interfaces on a Cisco 4510 chassis, you'll get a lot of lines that just say "ip flow monitor (flow monitor name) input".  You could filter that out pretty easily, though.
• show flow exporter
• show flow interface
• Getting the name of the poller that's assigned to each node can't be done with this type of scripted report output, but you should be able to pull this info from SWQL, and certainly you can get it by using Step 3 above.  You MIGHT just be able to get what you want from a custom SWQL query using the SDK.  There are some AMAZING SWQL experts in Thwack who've helped me do basic and advanced things. Or, if you're a DBA you might be able to figure it all out from SDK on your own.  Or if you have access to an internal DBA, introduce them to SWQL and the SDK and start making beautiful reports together.

3. Creating and filtering a report to show only nodes where the destination and polling engine do not match should be a good NCM config change report solution involving SWQL.  It's above my head, but I'm so impressed with the abilities of the DBA's and SQL/SWQL people in Thwack that I must believe they can not only show you the query/commands, but that they've already done this, or something very similar.  I apologize for not having that ability to share with you.

4.  Getting NPM to automatically configure your polling engines to match the Netflow destination of your nodes might be possible, but I'm not aware how.  Again, the fine SQL/SWQL experts here may have the answer.  I do it using NCM by changing the nodes to point at the correct poller, rather than using info in the nodes' configurations be used to change the poller.  My apologies a second time.

Once you've got some results, post them here for all to share & learn.  You'll probably get some useful suggestions and some requests for help or explanations.

You've got the power!