NTA\NPM as separate databases

lfransen

Hi everyone,

I know this question has been asked long before but since I have never seen any appropriate response I thought I'd ask again. Are there any plans to separate the Netflow data into a separate DB than the NPM data? I ask this because of issues of size and criticality.

I have purchased NTA in addition to my NPM and APM installation. We are not using the NTA module simply because of the issues regarding the amount of data from netflow. Our networking team is currently using another tool to capture Netflow data (Netflow Analyzer) and we decided to purchase NTA to eliminate one tool and have a consolidated view. Their current database is maxed out to 500GB and is only capturing less than one days worth of data. They run continuous flows for troubleshooting purposes for both the corporate and remote office connections (over 50 flows in total). It used to be about 5 days but the volume has gone up. This database is not backed up and if it crashes or gets accidentally deleted it's not a big deal.

Here is where it gets to be a problem. In order to get back to the 5 days worth of data with the current capture rates, I can expect the DB to need to be about 3TB in size. And that is only for now. Since the DB is unified with NTA, NPM and APM, I cannot afford to not have it backed up. The two paths I see are if the DBs were to be separated or if I build a big honking array for the SQL DB in such a fashion as to give me tons of space in a RAID 10 or something like it for protection. I'd still be fearful of not having a backup of all the NPM\APM configuration and node information. Right now my NPM and APM database is about 10GB in size.

It would be nice to see a setup where all the config and node information is in DB and all the detail data is in another. I know it is a much harder configuration as far as the application is concerned but somebody must have thought of this a long time ago. If this is not something in the foreseeable future then I will most likely have to remove my NTA install, kill maintenance, keep running and upgrade Netflow Analyzer.

Thoughts? Thanks!

Len

mavturner

Len,

There are a couple of ways to solve the problems you are describing. Splitting the database to a seperate server is certainly a work around, but it is still probably not the best solution (for the reasons you mention above). This is a well known issue with many NetFlow collectors and one we have a few options we are actively researching.

I'm happy to talk with you offline to give you a better picture of where we are going. 

Thanks,

Mav