Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Scaling NPM/SAM/NCM

Hi there...

I have searched around the forums and maybe I'm missing something but wanted to ask for opinions and feedback.

We have two physical servers for Solarwinds products today which include NPM, SAM, and NCM. The system is under a lot of load and we are looking to upgrade.

I'm looking at replacing the two existing servers with newer hardware and beefy specs. Also looking to upgrade from 2008 to 2012 Server Standard Edition (plus licensing for cores etc). Hardware wise I believe I have a good grasp on things - I like to beef things up way beyond "specs" as I always find them "bare minimum".

Software wise is where I get a bit confused so hoping for some input.

The new system I'm proposing looks like this:

Main application server (web server, primary server) - Dell R720, Dual CPU (8 cores), 128G RAM, 16 X 146GB 15k SAS, RAID, Server 2012 Standard

SQL 2012 server (dedicated to only Solarwinds) - Dell R910, Quad CPU (32 cores), 256G RAM, 16 X 600GB 15k SAS, RAID, Server 2012 Standard, SQL 2012 Standard

3 X Additional Polling Engines - Dell R320, single CPU (4 cores), 64G RAM, 4 X 146GB 15k SAS, RAID, Server 2012 Standard

Obviously a fair chunk on hardware along with a significant chunk on additional licenses (Windows, SQL, and 3 polling engine licenses).

Today, we are monitoring:

Network Elements	5056
Nodes	1110
Interfaces	3610
Volumes	336

Using the above numbers, it's important to note that we poll interface every 5 minutes vs default 9 minutes. Along with some other details we have concluded that we are maxxing out our current system. We also have approximately 3500 nodes that I would like to migrate from another system over to Solarwinds, along with much more intense application level stuff we want to monitor. At some point I'd like to look at adding Netflow as well but for now we'll leave that out of the equation.

How do the polling engines function for distributing load? The docs indicate that you can run them in "auto" mode and let the system determine which poller to assign for a node. What happens when a polling engine is taken offline - does one of the other polling engines take over or are those nodes that the particular polling engine is responsible for just lose that data for that time period?

We have no failover plans at this point - need to get a solid system running first and then look at adding failover options to another location in the future.

I'm assuming that throwing lots of RAM at these servers will help with performance - anyone else find something to the contrary?

Many thanks for any input...

Paul

Find more posts tagged with

Accepted answers

All comments

RichardLetts

We have two physical servers for Solarwinds products today which include NPM, SAM, and NCM. The system is under a lot of load and we are looking to upgrade.

Could you quantify what you mean by under a lot of load: are you seeing the system running at 100% CPU, Swapping, or bottlenecks writing to disk?

Main application server (web server, primary server) - Dell R720, Dual CPU (8 cores), 128G RAM, 16 X 146GB 15k SAS, RAID, Server 2012 Standard
SQL 2012 server (dedicated to only Solarwinds) - Dell R910, Quad CPU (32 cores), 256G RAM, 16 X 600GB 15k SAS, RAID, Server 2012 Standard, SQL 2012 Standard
3 X Additional Polling Engines - Dell R320, single CPU (4 cores), 64G RAM, 4 X 146GB 15k SAS, RAID, Server 2012 Standard

I'm assuming that throwing lots of RAM at these servers will help with performance - anyone else find something to the contrary?

In my experience the application server and pollers are not memory or CPU bound -- mine are all physical dual/quad-core (X5687) with 16GB of RAM on them and do not swap or max out the CPU (except when someone sends them tens of thousands of syslog messages in a minute). Since all the data is stored in the database app/polling server disk i/o and CPU are not (for me) a limiting factor.

my main application server is identical to the additional pollers, but doesn't have any NPM/NCM nodes assigned to (other than as the initial poller).

Your SQL server looks under-powered in the disk area: it's not the amount of storage but the number of spindles. My server has 24 spindles for the database. if I were building my server today I'd be looking at server-class SSD storage for the database. I can easily see that three 300GB SSD in a raid-5 would out-perform 6 600GB SAS in a raid-10. How large is your solarwinds database? min'e around 260GB (of which ~80GB is wireless and UDT client data) with the following characteristics:

Network Elements	29324
Nodes	5624
Interfaces	23673
Volumes	27

I poll at the default rate.

How do the polling engines function for distributing load? The docs indicate that you can run them in "auto" mode and let the system determine which poller to assign for a node. What happens when a polling engine is taken offline - does one of the other polling engines take over or are those nodes that the particular polling engine is responsible for just lose that data for that time period?

I've not found an automatic mode -- I have to move nodes around manually to keep polling.

/RjL

pstewart726

Thanks very much for the helpful response.

I was just talking to someone else about their Solarwinds deployment and they told me the came thing - that CPU/memory isn't overly important. They also asked why I wouldn't be looking at a SSH solution on SQL server. We are not Windows/SQL people overall - we "dabble" in it so will be looking to bring a professional in for the initial setup.

Our actual database is 97GB .... which seems pretty small overall.

I'll take a look at sizing up a different SQL server that can support more spindles and/or SSD. Any comments on RAM for SQL Server - I'm assuming more is always good for it?

Cheers,

Paul

ageopfarth

SQL will always try to use up as much RAM as is available. I have always thought it a good idea to put enough RAM in the server to pull the entire DB into RAM if possible. We were finally able to pull our SQL database of a multi-tasked SQL box and use a dedicated Solarwinds SQL instance. Ours has 70GB ram but is using 6 146GB SSDs in a raid 10. I have a very small DB (about 10GB) so I don't need a ton of RAM. Response times are pretty good from the database, but I think my main solarwinds poller is a little underpowered. Best thing you can do on the database is make sure you are defragging the indexes and, if you are running Netflow, running the Netflow specific DB maintenance routine regularly. I forgot that last one for about 6 months and saw my database balloon to 70GB.

I noticed you are running 2012 R2 Standard. Check to make sure you are actually able to use ALL of your RAM. 2012 STD will only use 32GB by default. 2012 R2 Enterprise addresses all RAM.

Message was edited by: Alen Geopfarth

sja

I think that a total over kill but I do'nt know what is preformance you expect.

Main application server (web server, primary server) - Dell R720, Dual CPU (8 cores), 128G RAM, 16 X 146GB 15k SAS, RAID, Server 2012 Standard

SQL 2012 server (dedicated to only Solarwinds) - Dell R910, Quad CPU (32 cores), 256G RAM, 16 X 600GB 15k SAS, RAID, Server 2012 Standard, SQL 2012 Standard

3 X Additional Polling Engines - Dell R320, single CPU (4 cores), 64G RAM, 4 X 146GB 15k SAS, RAID, Server 2012 Standard

If it was my setup to 20,000 element I will keep the SQL maybe with SSD

The rest should and could run on VM blade that get the ram and cpu it need.

I don't see why you should need that harware if you don't have netflow on the system.

mtgilmore1

I run NPM/SAM on a virtual server with 2 cpu's and 8GB memory. I have 3,000 elements and no issues with slowness. I do run NCM on another virtual server with 1 cpu and 4GB memory. My SQL server is large 24 core 92GB memory Windows 2008 R2. We are just doing the default polling on our devices.