Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Environment setup - Best Practice?

All,

This is probably not the right location for this posting, but I don't see a better place. I'm in need of advice from all those experienced people out there. My current environment includes...3 servers, (1) SQL 2005 Standard, (2) 2nd poller, and (3) NPM, APM, NCM, NTA, IPAM; and they are currently monitor about 500 devices. I have various, intermittent performance issues (UI, polling, alarms/alerts, etc.) resulting in an unhappy experience. And I need a monitoring environment for nearly 2000 devices.

As the servers are all VM, it is easy to add servers and split out the other applications. In your experience, what is a good environment configuration for optimal monitoring and user experience? What are valuable Windows and IIS settings that make for an optimal environment?

Many thanks in advance.

Dwyane

Find more posts tagged with

Accepted answers

All comments

DirtySouth

Dwayne, just to clarify...you're using two polling engines, which together monitor 500 devices? Is that correct?

I'm not a DBA by any means, but I can tell you that our environment benefits greatly from having a dedicated physical SQL box. We have 4 spindles configured in a Raid 10 for the DB. The other two spindles are for OS & Logs, running in a Raid 1. The high-speed IO in the Raid 10 is ideal. We have NPM, APM & NCM apps & IIS running on a single physical box (2x Dual Core AMD Opteron w/ 4GB RAM) We are monitoring 3600 total elements.

You may want to take a look at the IO performance counters on your SQL server to see if its the bottleneck. I hope that helps a little.

Donald_Francis

I agree totally with DirtySouth. A physical seperate SQL server for just SolarWinds is a HUGE help and probably the best help to your enviroment.

The other stuff is dependent on the counts, you have 500 devices but how many elements? 500 devices could be ALOT of elements depending on what they are.

deverts

All,

Does a "physical" SQL server truly make that much difference? We are almost entirely VM, and run our Sharepoint and other apps. on VM SQL servers.

To answer the question about nodes and elements, I have 515 nodes with 2762 elements, and 2208 interfaces. But that number is going to grow exponentially in the coming months as we expand our business and infrastructure. That's why I'm trying to get optimal performance and best practice design in place now, while the environment is still relatively manageable.

Dwyane

Donald_Francis

I think a physical server can make that much difference when you are talking about a server that will use as many resources as a busy SQL box will.

I think you can run everything else on VM without an issue so long as you give it enough resources.

Just as an example here is my layout.

Server 1 - Primary Box and poller

Server 2 - NCM Server

Server 3 - Additional Poller

Server 4 - SQL server for SW

Server 5 - Web server

Server 6 - Additional Poller

GaryP

I've just taken delivery of additional hardware to expand our evaluation NPM setup from a single server to the following:

Server 1 - SQL server - HP DL380, 2.83GHz quad-core, 8GB RAM, Windows Server 2k8 Enterprise

Server 2 - Orion NPM+NCM+poller, web server - HP DL160, 2.53GHz quad-core, 6GB RAM, Windows Server 2k8 Enterprise

Server 3 - Additional poller - HP DL160, 2.53GHz quad-core, 6GB RAM, Windows Server 2k8 Enterprise

We're looking to eventually monitor ~1000-1500 nodes for about 30,000 elements and will obviously need an additional poller to achieve this. Is there much to be gained from shifting the web server off onto another box? It's interesting that you run NCM on a seperate server, also, did this speed things up much for you?

Disk transactions on the SQL server are the current bottleneck in our system, it'll be interesting to see how we wind up tweaking this.

deverts

Donald,

Now we're cooking with fire! I had the same thoughts about my environment. I was thinking about:

1. Primary (NPM, APM, IPAM)

2. Additional Poller

3. NCM

4. NTA

5. SQL (still not convinced on VM vs physical, but I will definitely take the advice and investigate deeper. As we are making everything VM, there may be some tweeks to make it work better in VM)

6. Web server (tell me more, how do you split up the web server from NPM?)

GaryP

Do you need to purchase an 'additional web server' licnse or can you just split off the one running on a normal NPM install?

Donald_Francis

GaryP,

For 30,000 elements you will need 3-4 pollers in my experience. SW says to cut off each poller at about 8k elements which I think is a good number. On Dual quad core boxes I noticed that they started to really suffer at about 10k elements.

We moved the web server to another box because it is cheap (something like a $500 license) to do and we have a lot of web users. For us it enhanced the speed of th web site a good bit as the web site is not competing with anything else just directly hitting the SQL box.

I fight with SQL as well especially when it comes to netflow so I know your pain.

NCM on a seperate box just kinda happenned it was not really a designed situation. But it has worked out for us well especially since we have about 1300 devices in there now it is free to take as long as it needs for operations unabaided by anything else.

deverts

You are killing me! Where were you when I built this thing initially! I would have done all this then. This is great advice and I will move on building a diverse environment immediately.

I'm still investigating the VM/SQL tweeks, my VM and SQL admins say we should be fine in VM...but I'm still having performance issues. I think after I break this stuff up, re-evaluate the performance.

I'm not sure how to assign all of you as the verified answer provider for this one. You each provided valuable input.

Dwyane

DirtySouth

I think the important thing to remember in regards to SQL is the Raid configuration. Thats what is really going to make the difference. Depending on your virtual environment, you may be able to get away with it, if you've got optimal raid configurations and its dedicated. Our SQL box is physical, but it also has a couple other small databases running on it.

If we continue to add elements to our environment, I will probably look into moving the IIS to a dedicated box, perhaps even virtual.

GaryP

I just had a quick chat with our DBA re. optimised RAID configuration and with our setup (8x 300GB disks on one HBA) he recommended to the following:

2 disks in RAID1 for system, logs, page, applications, etc

2 disks in RAID1 for SQL logs

3 disks in RAID5 with 64k stripe for SQL data

1 hot spare available to all other arrays

It's not ideal but without investing in an additional HBA it's the best I think we can get with the current hardware. I realise RAID5 isn't generally recommended for databases but looking at resource monitor on the machine it's currently doing 10x more reads than writes so seems appropriate for now. We'll see if that balance changes as we scale to polling more nodes in the future.

lag

This post might help: geekblog.solarwinds.com/.../the-fast-and-the-furious-orion-sql-and-sans.aspx