7 Replies Latest reply on Nov 30, 2013 12:36 PM by nicole pauls

    Slowness and Timeouts in LEM

    ttl

        I have to say, since the upgrade to LEM 5.6 and the new database, getting results for more than a week or two in the past is next to impossible. I just tried searching for an event for the past month and a half, and the graph got about halfway through populating before Time Out occurred. And even though the graph (and listing on the left) said that there were over 800 results, the Results Details pane says No Matching Results.

       

        I'm really hoping that the next update fixes this; otherwise this product has become almost useless.

        • Re: Slowness and Timeouts in LEM
          evanr

          How beefy is your box?  Just last week I had to pull a log from 4 months ago and while it wasn't instantaneous the query finished in a little over 4 minutes.  I find that the more detailed and granular I get in my refinement fields the faster I see results.

            • Re: Slowness and Timeouts in LEM
              ttl

              It's a VM from the OVA that Solarwinds supplied, 8GB / 2 vCPUs.

                • Re: Slowness and Timeouts in LEM
                  evanr

                  Any other vms running on the host?  We have a dedicated host box for our SW environment.  I would imagine it all depends on how much data is flowing into it as well that could cause additional overhead.  Sorry to hear your having issues with it.  Ours has been pretty stable since the upgrade to 5.6 with the exception of having to restart the manager every once in a while.

                   

                  FYI I just saw RC 5.7.0 out. 

              • Re: Slowness and Timeouts in LEM
                curtisi

                Realizing that it might take a while, but if you run a Database Maintenance Report and/or an Event Summary - Top Level Statistics report, how many events are you gathering in a day?  In a week?

                 

                If you SSH to the LEM, under MANAGER type RCC and then pumpstatus.  Do you have anything waiting?

                  • Re: Slowness and Timeouts in LEM
                    ttl

                    You've touched on one of the other problems I never experienced until the 5.6 upgrade:  I can't run reports (at all) with the Reports tool. Reports run forever (open case 525068) or I get an "Unknown Database Connector Error" even though the Data Source is correct. I did finally get the DB Maintenance report to run; it looks like we're getting ~ 27M events per week.

                     

                    Pumpstatus shows nothing waiting.

                      • Re: Slowness and Timeouts in LEM
                        curtisi

                        Starting with the basic checks:

                        • LEM version and Reports version match, right?  LEM 5.6 needs reports 5.6, and so on.
                        • You've deployed Hotfix 1?
                        • Reports was installed as an Administrator? (On WIndows Vista and up, even if you're logged in as an all-powerful user, you have to Run As Administrator to execute those rights)
                        • You're running Reports as an Administrator?
                        • Have you tried reinstalling Reports?
                          • First, go to the Reports folder in Program Files.  If you've made any Custom Reports, there is a directory called CustomReports that you want to save by copying it to another location, else you'll be really upset when all your custom reports go away
                          • In Windows, run the Add Remove Programs (or Programs and Features) and remove both the Solarwinds Reports Console and the Crystal Reports Runtime
                            • You can download both these installers from the Customer Portal if you need them, and they are two separate items there
                          • Now some housekeeping:
                            • Make sure the Program Files\Solarwinds Log and Event Manager Reports folder is deleted
                            • If you've ever run the AIR console, go to C:\Users (or Documents and Settings)\[YOUR USER]\AppData\Roaming\sim.console.  Backup the Local Store folder to a safe location, then nuke the sim.console directory
                            • While you're in there, if you've been running LEM for a few version, look for a Contego Console directory.  This is deprecated, and if you're new to LEM you won't have it, but Reports still occasionally gets mixed up and pulls config data from that directory, which can mess things up.  Nuke it.
                          • Reinstall the Crystal Reports Runtime as an Administrator
                          • Reinstall the Reports Console as an Administrator
                          • Run the Console as an Administrator
                          • Is it any better?
                        • I've also seen some odd problems where installing Reports on another machine (as an Administrator) and running it there works better.  If nothing else, it tests if the issue is isolated to a specific machine.  You may need to UNRESTRICTREPORTS under SERVICES in an SSH session to let the other machine connect.
                        • Are you running any special configuration, like a separate database and manager?  Reports has to point at the Database.
                    • Re: Slowness and Timeouts in LEM
                      nicole pauls

                      Hey TTL, and others, not sure if you're in the other thread I just replied to, but we did make some serious improvements in LEM 5.7 RC3 (released and on the portal yesterday) that tries to address this extremely slow/endless reports stuff. If you are already on the RC, I'd upgrade to this one and see how that affects your reporting. If not (or you're not comfortable), we're hoping to confirm these improvements for our upcoming release.