20 Replies Latest reply on Oct 23, 2008 5:10 PM by denny.lecompte

    Improving NPM Performance

      Hello:  Is there anyway to improve the responsiveness of Orion?  We run SQL on another server already.  Should we run IIS on another box too and leave the Orion box to poll only?


       Thanks.

        • Re: Improving NPM Performance
          denny.lecompte

           Can you give us more detail on what's not responding?  It is polling?  Is it the web console?  If the latter, which pages? 

            • Re: Improving NPM Performance

              Sure.  The polling seems fine, it is the webpages.  Pretty much all of them take awhile.


              For example, I click switch in my all nodes (ajax) tree and it took about 10 seconds to show them.  Then I click one of them and it took about anopther 10 seconds for the node details to show up.


               


                • Re: Improving NPM Performance
                  denny.lecompte

                   How many nodes?  How many interfaces?  What are your database specs?   Are you using APM or NetFlow?  Give us as detailed a picture as you can, and I and the broader community can probably provide some guidance.

                    • Re: Improving NPM Performance

                      We have about 240 nodes and about 963 interfaces.  We are also using APM and Netflow.  The srver is running Server 2003 R2 and has 2 GB of ram and dual Xeon cpus at 2.9 Ghz. The DB server is running SQL standard edition.  It has 8 GB of ram.


                      Thanks


                        • Re: Improving NPM Performance
                          aLTeReGo

                          More information on the DB Server specs would be helpful since in my experience that's where the bottleneck is. Processors, how many, RAID type, number of physical disks in the array, etc. Also is this DB server dedicated to Orion or does this SQL server also service other databases?


                          As a side experiment on the Orion server stop the NetFlow service and test your website performance when you're not collecting netflow data. Report back your findings.

                            • Re: Improving NPM Performance

                              Stopping Netflow had no impact.  The DB guy, rather than provide the information requested, will provide me with a dedicated SQL server for Orion.  I will let you know the results when it happens.

                                • Re: Improving NPM Performance
                                  r0berth1

                                  I have the same problem. But i have a dedicated SQL server, the poller and website are on the same box. All pages are slow. I am able to clear the cache and try to open each of the items on the home page and they all load is less than 2 sec. including the maps. Would like to know what the fix it. The specs are as follows:


                                  Poller and web server: Intel Xeon 2.33GHz with 1GB RAM. With Microsoft Windows Server 2003 R2 standard Edition Service Pack 2.


                                  SQL Server: Intel Xeon 2.33GHz with 2GB RAM. With Microsoft Windows Server 2003 R2 standard Edition Service Pack 2.


                                  connected to the same Cisco 6509 via 2GB port channels.

                                    • Re: Improving NPM Performance
                                      jeff.stewart

                                       Anyway to improve the amount of time it takes for the NetPerfMon service to get the memory it needs to run?  This is taking a very long time and pretty much makes my Orion server useless while its in this stage.

                                        • Re: Improving NPM Performance
                                          r0berth1

                                          do you have Orion on a dedicated server?

                                            • Re: Improving NPM Performance
                                              jeff.stewart

                                               Yes.

                                                • Re: Improving NPM Performance
                                                  aLTeReGo

                                                  How many elements are you monitoring?

                                                    • Re: Improving NPM Performance
                                                      jeff.stewart

                                                       My smallest server has 5900 and the largest has a little over 10,000.  The one with 5900 loads faster, which I would expect.  However, I shouldn't have to wait 15-20 minutes for the others.

                                                      • Re: Improving NPM Performance
                                                        scraig84

                                                        Are there any general guidelines for server specs?  Separation of polling engine from DB always recommended? 


                                                         Single box here dedicated to running NPM and NTA as well as the SQL instance.  Roughly 450 nodes and 700 interfaces currently.  Server is dual-quad-core 3Ghz processors and 4Gb RAM.  Not sure on the exact specs of the drives but know they were the high RPM drives with hardware-accelerated RAID.


                                                         Shutting down NTA has a very very minor effect on performance.  Even without it the web interface is still fairly painful.  I would love any and all help in improving this.  My server guys took a look at some of the perfmon counters for drive, cpu, and memory utilization and didn't see any red flags or anything they classified as alarming.


                                                         Thanks,  Scott

                                                          • Re: Improving NPM Performance
                                                            r0berth1

                                                            I would split them up. Currently have mine split to 2 VMs on an ESX server. one for Polling and Website, and the other for the SQL server. I would like to get 3 real servers to put them on. I currently have the following:


                                                            928 Elements


                                                            344 Nodes


                                                            584 interfaces


                                                             It was recommended to split them up to separate servers by SW. but they should have the specs on their website.

                                                            • Re: Improving NPM Performance
                                                              denny.lecompte
                                                              Are there any general guidelines for server specs?  Separation of polling engine from DB always recommended? 
                                                               

                                                              For 500+ elements, you should have a separate database. 

                                                                • Re: Improving NPM Performance
                                                                  jeff.stewart
                                                                  Denny, anything on the long load times of the System Manager?
                                                                    • Re: Improving NPM Performance
                                                                      denny.lecompte
                                                                      Denny, anything on the long load times of the System Manager?
                                                                       

                                                                      You have 5900 and 10,000 elements polled by your servers, respectively.   System Manager is on the same thread as those elements.  It's not going to load quickly.  Are you doing anything on System Manager that couldn't be done on the web console, which would avoid the problem?

                                                                    • Re: Improving NPM Performance
                                                                      scraig84

                                                                      Would you recommend moving the poller or the DB?  Also, the minimum specs listed on NPM's datasheet don't really provide much insight considering my current hardware is well over what is listed.  Can you give me a better idea of what type of hardware would be adequate for each?  Also note that we are looking to add APM as well which will likely add at least 4-500 elements.  That said, if we can't improve the performance considerably, I might have mutiny on my hands. 


                                                                       I may also be forced to putting one or the other in a VM environment.  Is either one not recommended to be on a VM?


                                                                       Thanks again,  Scott

                                                                        • Re: Improving NPM Performance
                                                                          r0berth1

                                                                          I would not run them on a VM. I have that setup now. With the poller and web on one vm and the sql database on the other, it is slower that having all 3 on the same size hardware server. VMs just dont work that well for this type of app. or at least that is my experance.

                                                                            • Re: Improving NPM Performance
                                                                              scraig84

                                                                              I was more curious than anything - especially if I were to just move the poller/web function off the existing server. 


                                                                               The most likely thing for me to do would be to move the db to one of our dozen or so SQL boxes.  That said, I need to have a better understanding of the requirements for the db server before I go to our DBA's with the request.  Does the db server need to be dedicated to Orion?  Are there certain hardware requirements or performance/load metrics it should meet to determine what we need to do?