cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post
Level 7

Split database load for NPM/Netflow

Jump to solution

We primarily use NPM for alerting and event monitoring of around ~1500 devices and have been approached about adding netflow analysis for an additional ~900 devices.

We are concerned about the additional load placed on the database server, primarily in terms of I/O and finding it difficult to scope what hardware upgrades will be required, and how well it will scale with such a high number of devices. It is extremely important that it doesn't negatively impact the primary purpose of our solarwinds which is the alert and event monitoring. 

Ideally we'd like to setup a 2nd database server that we could use solely for netflow to ensure it does not impact performance of our current setup. The only other option would be another fresh Solarwinds install but the cost of this and ongoing maintenance of two installs isn't ideal.

Are there any plans of the horizon for allowing segregation of database load across different servers for different products ala Netflow? It would definitely help when scoping/planning for growth in large networks.

0 Kudos
1 Solution
Level 18

We are concerned about the additional load placed on the database server, primarily in terms of I/O and finding it difficult to scope what hardware upgrades will be required, and how well it will scale with such a high number of devices. It is extremely important that it doesn't negatively impact the primary purpose of our solarwinds which is the alert and event monitoring. 

The number of devices isn't as important as knowing the number of flows per second.   The best way to determine this is to install Orion NTA, export NetFlow to the machine, and review the Windows performance counters.   It's not necessary to have NTA actually collecting/storing the flows.

Here's a reference NTA implementation from a customer who's been able to see 40,000 flows per second (60,000 flows per second in peak).

Orion Server
Intel Xeon Dual Quad Core @ 2.00GHz
4.00 GB of RAM
Disks – DELL PERC 5/I
 6 136GB 15K – RAID 5
8706 elements
279 NetFlow interfaces (spread across 6 devices)

Poller 1 – Same HW as Main
9309 elements
Poller 2 – Same HW as Main
10893 elements

SQL Server
Server 2008/SQL Standard 2005
Intel Xeon Dual Quad Core @ 2.50GHZ
8.00 GB of RAM
Disks  -DELL PERC 6/i
2 300GB 15K – RAID 1 (Operating System & DB Log)
4 300GB 15K – RAID 10 (DB files)

Are there any plans of the horizon for allowing segregation of database load across different servers for different products ala Netflow? It would definitely help when scoping/planning for growth in large networks.

This is something we're actively exploring.   In parallel to this longer-term architectural change, we continue to make significant performance improvements in each release to mitigate the I/O load on the primary Orion DB.  Just as an example, we added Top Talker Optimization in NTA 3.6.

View solution in original post

0 Kudos
4 Replies
Level 18

We are concerned about the additional load placed on the database server, primarily in terms of I/O and finding it difficult to scope what hardware upgrades will be required, and how well it will scale with such a high number of devices. It is extremely important that it doesn't negatively impact the primary purpose of our solarwinds which is the alert and event monitoring. 

The number of devices isn't as important as knowing the number of flows per second.   The best way to determine this is to install Orion NTA, export NetFlow to the machine, and review the Windows performance counters.   It's not necessary to have NTA actually collecting/storing the flows.

Here's a reference NTA implementation from a customer who's been able to see 40,000 flows per second (60,000 flows per second in peak).

Orion Server
Intel Xeon Dual Quad Core @ 2.00GHz
4.00 GB of RAM
Disks – DELL PERC 5/I
 6 136GB 15K – RAID 5
8706 elements
279 NetFlow interfaces (spread across 6 devices)

Poller 1 – Same HW as Main
9309 elements
Poller 2 – Same HW as Main
10893 elements

SQL Server
Server 2008/SQL Standard 2005
Intel Xeon Dual Quad Core @ 2.50GHZ
8.00 GB of RAM
Disks  -DELL PERC 6/i
2 300GB 15K – RAID 1 (Operating System & DB Log)
4 300GB 15K – RAID 10 (DB files)

Are there any plans of the horizon for allowing segregation of database load across different servers for different products ala Netflow? It would definitely help when scoping/planning for growth in large networks.

This is something we're actively exploring.   In parallel to this longer-term architectural change, we continue to make significant performance improvements in each release to mitigate the I/O load on the primary Orion DB.  Just as an example, we added Top Talker Optimization in NTA 3.6.

View solution in original post

0 Kudos

So does NTA need a separate Database than from NPM? or can they share the space on the SQL Server?

0 Kudos

NTA uses the same database as it is an extension module to Orion NPM.   The recommendation above is that you spec out the appropriate hardware for the Orion DB so that you can handle the additional I/O load from NetFlow data collection.

0 Kudos
Level 13

Hi Andy--

I've marked this for the product manager to see about adding to the feature request list.

Thanks,

M

0 Kudos