Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials

Orion NPM is horrifically slow at loading!

Afternoon all,

I have an issue with NPM, We're running APM and NCM along side.

We have got an unlimited licence for the products, and are currently running in evaulation before registering them until we can get this bug worked out.

The server is a HP DL380 G6 with 8 physical and 8 Hyper Threaded processors, Running 24GB of RAM.
Operating system is on 2x 300GB DP SAS in raid 1
There is a 300GB Page File on 2x 300GB DP SAS in raid 1
The SolarWinds data is stored on a 3x 300GB DP SAS in raid 5.
Both the SQL DB and the IIS data is stored on the raid 5.

We are using SQL 2008 R2 Express which gives us 10GB of usable space over the 4GB from 2003. We plan to migrate to a full SQL 2008 R2 install when we get close to the limit.

The DB is currently 2.5GB in size, and we are currently monitoring:

Network Elements : 2059
Nodes                    : 254
Interfaces              : 1767
Volumes                 : 38

We are running a single polling engine, Which is currently at 43%.

We are monitoring Cisco routers and switches via SNMP V2c, 3 HP SAN's via SNMP V2c, and i'm starting to add Windows Servers (2003 and 2008 R2) via WMI.

Polling Settings:

Default Node Poll Interval        : 30 seconds
Default Interface Poll Interval : 30 seconds
Default Volume Poll Interval     : 120 seconds
Default Rediscovery Interval    : 15 minutes

Default Node Statistics Poll Interval        : 10 minutes
Default Interface Statistics Poll Interval : 10 minutes
Default Volume Statistics Poll Interval     : 15 minutes

The server is averaging around 10% cpu utilisation across all 16 cores, and around 6GB of RAM usage.

We have 10 users of SolarWinds, and 4 large TV's showing various maps and data.

We are having issues with horrifically slow refresh rates for the users pulling in node details.

One screen is pulling the Message Centre page, and is taking approximately 15-20 seconds to refresh most of the time, the map screens are taking around 40-50 seconds to refresh dependant on network status.

I turned on caching for aspx files in the IIS manager. This caused issues with the maps, Due to them being on a single file i couldn't exclude from the cache. However this sped up the reload time for pages considerably, taking around 1-2 seconds to load, which is where i would expect it to be given the performance of the hardware SolarWinds has to run on.

I also increased the IIS worker threads, to 16 from 1. This again caused issues with the maps where they complained if it wasn't set to 1!

I'm starting to get questions now if we spent £50,000 on something that is unable to perform how we want.

We are getting a further 4 staff, and the usage of SolarWinds will continue to get heavier.

Find more posts tagged with

Accepted answers

All comments

tomas.mrkvicka

Hi,

there can be various reason for slowness of your system so it is difficult to invastigate it without further details.

But there two obvious problems in your system:

1) RAID 5 is not recommend for data storage because it is very slow, we recommend to use RAID 1+0 for data storage disk (or better say for all disk you have)

2) SQL Server Express is very limited in performance, it can use only limited amount of memory and also limited number of CPUs. Therefore if you want to have real performance it is absolutely necessary to use fully scalable database.

Maybe there are another hidden issues, but I would focus at these two at first.

uscallesen

I've been having issues with poor Orion performance aswell - also while running on highend hardware with the DB using 16 spindles on our SAN. Upgrading to NPM 10.2 made a world of difference to us although we still get poor performance during the day when many people are using the web frontend.

I'm currently looking into optimizing the db as I'm positive this is where the bottleneck is - you'll probably end up with the same conclusion. We're using Sql 2008 x64 btw.

MattIggo

We're running NPM 10.2 upgraded from 10.1, Which made quite a bit of difference.

We looking into getting a RAID 10 setup, However it'll mean moving the DB and other items around.

Having looked into SQL Express 2008 x64 R2, It appears it only supports 1GB of RAM and 1 processor.

I am not maxing out any of the processors, So i am guessing it's the RAM that's potentially killing performance.

We are looking to upgrade to SQL 2008 R2 x64 Enterprise to see if this fixes the issue before we copy the data, Destroy the RAID 5 and rebuild a RAID 10.

The DB continually complains about bad indexes, And from what i can see using APM on the SQL on our SolarWinds box, It's not very well indexed, and it's continually doing full DB searches.

I wonder if there is a way of patching / fixing this issue as I expect this would speed the read requests up quite substancially?

tomas.mrkvicka

Yes database is typical bottleneck for such system as Orion.

I would recommend to create special filegroups for heavily loaded tables and put those filegroups on separated spindles. You can also try to use SSD disks or even RAMdisk, I usually recomend to use RAMdisk for tempdb, tempdb is heavily loaded and therefore it is first candidate for some speed up.

MattIggo

Right,

Today I have:

Installed SQL Enterprise 2008 R2 in a trial mode. It's running on 16 processors with 18GB of RAM assigned.

The RAID5 is still present, and according to the performance monitor for SQL it's not using any database IO.

We are still suffering issues with speed, Please can you advise what else we can try?

Really really don't want to remove the RAID 5 and rebuild with RAID 10 considering we can't see that much data being transfered from the disks.

Turning on Caching really helped, But caused us issues, is there another way we can turn on caching?

This has taken me all day to get back running right, after various issues with SQL servers and options and corrupt DB's using the DB tool built into the SolarWinds platform of tools.

Deltona

Hi there,

Running all SolarWinds modules, massive amounts of data, Syslogs and Traps engros, spanning resources heavily accross modules and views, maps for days. Our SQL server publishes Orion site files faster than our client machines can load them in their browsers.

SQL Server 0.7x seconds to publish

Orion Console 3.4 seconds to load

You will not get anywhere close this type of performance if you do not - at least - take the SolarWinds system requirements and recommendations into account.

MattIggo

Specifically which parts of the requirements and recommendations are we not compliant to?

Speaking with SolarWinds, They appear to believe it's not the hardware at fault?

Where did you get your stats from?

Deltona

www.solarwinds.com/.../OrionQuickStartGuide.pdf

Page 15.

This is the only good starting point. Software and settings tweaking follows hereafter along with network/client tuning.

sja

I will try to set the polling to the default.

this is not the default.

*****************************************************

Polling Settings:

******************************************************