6 Replies Latest reply on Jul 21, 2009 4:27 AM by GZhytar

    Report to track database growth due to NetFlow?

    borgan

      Has anyone built a report to track the rate of growth of the Orion database due to NetFlow data? I am testing a few source interfaces and want to see if I can predict how fast the database will fill up with NetFlow data.

      Can that be done?

        • Re: Report to track database growth due to NetFlow?
          borgan

          I was looking at Report Writer and also in Database Manager to try to get a handle on the DB table(s) that may have the data I seek.

          It appears the data is in the NetFlow Summary 1, 2, and/or 3 tables. When I construct a report that takes the total bytes fron the Traffic field in Report Writer, I see that the SQL is summing data from the NetFlwo Summary table. Does that mean that it is summing data from all three tables, or from just one table?

            • Re: Report to track database growth due to NetFlow?
              Andy McBride

              Hi borgan,

              For the first post, database growth rate depends on a few factors, some of which you can control, some you can not. The ones you can control are full exports vs sample NetFlow (in the device config), data retention settings (NetFlow Settings) and number of exporters (sometimes adding a source just shows the same flows as another source (topology)). Some factors you can't control are flow durations (lots of short conversations use more storage), circuit utilization and IP address range (a wide range of end points will make a larger database).

              The growth should be close to linear if your data environment is steady. Measuring the database over a couple of weeks should give you a line to extrapolate. 

              For the second question, we do sum across tables in some reports - it depends on what data you are looking for and the time range.

              Andy

                • Re: Report to track database growth due to NetFlow?
                  davidmaltby

                  As for the second question, the reports are using SQL view called NetflowSummary which is a union of all three tables.  As Andy mentioned, it is dependent on the date range that it is filtering on.  SQL Server is smart enough not to access the individual summary tables that are not involved in the date range filter.

                  Thanks,

                  David

                    • Re: Report to track database growth due to NetFlow?
                      borgan

                      So, do I understand correctly that each of the three NetFlow Summary tables stores a different statistical summary of NetFlow data (detailed, hourly and daily)?

                        • Re: Report to track database growth due to NetFlow?
                          davidmaltby

                          Yes, somewhat.  But here are the details, just to make sure what you're thinking is how it is.....

                          The Details tables store the conversations summarized by the minute .  Each of the detail stores stores 15 minutes worth of data.  By default we store at least an hours worth in the details tables.  Therefore for a node, there is at least 4 (15 minute) tables and the tables use a value in their name called the interval.

                          Once data is over an hour old, it goes to the summary1 table which stores conversations summarized by an interval (15 minutes).  By default it stores a day's worth of data.

                          Once data is over a day old, it goes to the summary2 table which stores conversations summarized by the hour.  By default it stores a 3 day's worth of data.

                          Once data is over 3 days old, it goes to the summary 3 table which stores conversations summarized by the day.  By default it stores 30 day's worth of data.

                          After 30 days the data is removed from the database.

                          Thanks,
                          David