21 Replies Latest reply on Aug 15, 2017 2:31 PM by jrouviere

    Error with Reports Manager

    raingeek

      Ok, I've been bashing my head on this problem for a few days now and I'm calling in for reinforcements!

       

      Just joined a new organization using LEM, and I'm happily learning all it can do. However I cannot get the Reports Manager application to run on my desktop system (Windows 7, 32-bit). Three other team members can use it just fine, I've checked and verified my settings against theirs over and over again and I can't spot anything wrong.

       

      The issue is that when I attempt to generate a report the application reports a Logon failure.

       

      To clarify - not all users are logged in at the same time. Typically I am the only one connecting. I've confirmed nobody else is logging in when I attempt. After my test fails I've seen one of them log in without any problem on their own desktop system.

       

      I've done a lot of searching on Reports Manager errors on this site so let me list everything that I've already done:

      • Confirmed that Reports version (6.1.0.2) and LEM version (6.1.0) match
      • Re-downloaded the software from this site, to ensure I have the proper versions
      • I've uninstalled Reports, rebooted (to ensure the system was clean) and re-installed using the "Run As Admin" option
      • I've uninstalled Crystal Reports Runtime (v11), rebooted and re-installed using the "Run As Admin" option
      • I've created and tested multiple users on the LEM appliance. Right now I'm testing against three accounts - the built-in Admin account, my Domain account, and a LEM User account. All three are configured as LEM Administrators. I can login to the web client with all three accounts so I know the accounts work and the passwords are accurate.
      • The proper LEM certificate is loaded in the application. I manually exported the cert and loaded it into my system.
      • I have run the application in my normal user context as well as in the "Run as admin" context. No difference.

       

      In the Report application I successfully test the connections for Primary Data Source, Syslog Server and Managers - Credential and Certificates. Yet every time I try to run a report it gives me the logon error.

       

      Also, after it fails the application acts like it is querying the database (progress bar in the lower right corner) but never comes up with anything. Unfortunately this locks up the application and I have to close the app, which causes a Windows error report that it didn't close correctly. This happened prior and after the application re-installation.

       

      I'm attaching the SWLEMReports.txt file as well as a few screenshots to back me up. Please note the weird database errors in the log file.

       

      Any ideas? Thanks!

        • Re: Error with Reports Manager
          curtisi

          Do you have the Microsoft Telnet Client feature enabled on your station?  Can you try opening a command prompt and running:

           

          TELNET SOLARWINDS-LEM 9001

           

          You should get something back like this:

           

          HSQLDB JDBC Network Listener.

                                      Use JDBC driver with Network Compatibility Version2

          .1.0.0 and a JDBC URL like jdbc:hsqldb:hsql://hostname...

           

          Connection to host lost.

           

          Does that work?

            • Re: Error with Reports Manager
              raingeek

              Confirmed, my telnet output looks exactly as predicted. Output copy is below:

               

              HSQLDB JDBC Network Listener.

                                          Use JDBC driver with Network Compatibility Version2

              .1.0.0 and a JDBC URL like jdbc:hsqldb:hsql://hostname...

               

              Connection to host lost.

                • Re: Error with Reports Manager
                  curtisi

                  I know you've already nuked Reports and paved, but...

                   

                  Can you uninstall Reports, nuke the install directory if it gets left behind, and then reinstall with the "Run as Admin" and let me know if that works?

                   

                  Has someone run the RESTRICTREPORTS command on the LEM, maybe?  You may need to have them run the UNRESTRICTREPORTS and try again.

                  1 of 1 people found this helpful
                    • Re: Error with Reports Manager
                      raingeek

                      I'll try a re-nuke and directory delete this afternoon. I was kicking myself that I didn't think to verify the directory removal last time prior to re-install.

                       

                      I considered the RESTRICTREPORTS command - I asked around to see if they had configured this and the answer was negative (LEM has only been installed for about 6 months so it's a recent addition). Also their documentation of the installation process/procedure doesn't show them configuring RESTRICTREPORTS. I wish there was a way to view if anything was set there (Hmm, maybe that should be a feature request) and I'm hesitant to make any changes without being able to see what may be potentially set there. Making that change is kind of an absolute last step if nothing else works out.

                       

                      I'll report back after the re-nuke.

                        • Re: Error with Reports Manager
                          curtisi

                          Support could root in and see if the changes made by "RESTRICTREPORTS" are present in the LEM config.

                           

                          You can also see the config yourself.  Under MANAGER run VIEWSYSINFO.  The last screen of data (keep hitting spacebar!) will show the firewall ACLs:

                           

                                ---------------------------------

                            Network access configuration:

                          *filter

                          :INPUT ACCEPT

                          :OUTPUT ACCEPT

                          -A INPUT -s 0/0 -d SWI-LEM/32 -p tcp -m multiport --destination-ports 4803 -j REJECT

                          -A INPUT -s 0/0 -d SWI-LEM/32 -p udp -m multiport --destination-ports 4803 -j REJECT

                          -A INPUT -s 0/0 -d SWI-LEM/32 -p udp -m multiport --destination-ports 4804 -j REJECT

                          -A INPUT -s 127.0.0.1/32 -p tcp -m multiport --destination-ports 10101:10102 -j ACCEPT

                          -A INPUT -s 10.254.10.12/30 -p tcp -m multiport --destination-ports 10101:10102 -j ACCEPT

                          -A INPUT -s 10.254.10.16/30 -p tcp -m multiport --destination-ports 10101:10102 -j ACCEPT

                          -A INPUT -p tcp -m multiport --destination-ports 10101:10102 -j REJECT

                          -A INPUT -s 0/0 -d SWI-LEM/32 -p tcp -m multiport --destination-ports smtp -j REJECT

                          -A INPUT ! -d 127.0.0.1 -p tcp --dport 5432 -j REJECT

                          COMMIT

                          *nat

                          :PREROUTING ACCEPT

                          :OUTPUT ACCEPT

                          :POSTROUTING ACCEPT

                          -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080

                          COMMIT

                           

                          You'd see some line mentioning dport 9001 and REJECT if RESTRICTREPORTS had been run.  Mine doesn't have that, so it's not enforced on my system atm.  You'd also see a bunch of dport 9001 and ACCEPTs, one for each allowed reports client.

                          1 of 1 people found this helpful
                            • Re: Error with Reports Manager
                              raingeek

                              Nice to know it is possible the RESTRICTREPORTS can be viewed, the other knowledgebase article (I will have to dig it up) made it sound like it was impossible to do that.

                               

                              It doesn't look like we are restricting dport 9001. Here is my output:

                               

                                ---------------------------------

                                Network access configuration:

                              *filter

                              :INPUT ACCEPT

                              :OUTPUT ACCEPT

                              -A INPUT -s 0/0 -d SolarWinds-LEM/32 -p tcp -m multiport --destination-ports smtp -j REJECT

                              -A INPUT -s 0/0 -d SolarWinds-LEM/32 -p tcp -m multiport --destination-ports 4803 -j REJECT

                              -A INPUT -s 0/0 -d SolarWinds-LEM/32 -p udp -m multiport --destination-ports 4803 -j REJECT

                              -A INPUT -s 0/0 -d SolarWinds-LEM/32 -p udp -m multiport --destination-ports 4804 -j REJECT

                              -A INPUT -s 127.0.0.1/32 -p tcp -m multiport --destination-ports 10101:10102 -j  ACCEPT

                              -A INPUT -s 10.254.10.12/30 -p tcp -m multiport --destination-ports 10101:10102  -j ACCEPT

                              -A INPUT -s 10.254.10.16/30 -p tcp -m multiport --destination-ports 10101:10102  -j ACCEPT

                              -A INPUT -p tcp -m multiport --destination-ports 10101:10102 -j REJECT

                              -A INPUT ! -d 127.0.0.1 -p tcp --dport 5432 -j REJECT

                              COMMIT

                              *nat

                              :PREROUTING ACCEPT

                              :OUTPUT ACCEPT

                              :POSTROUTING ACCEPT

                              -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080

                              -A PREROUTING -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8443

                              :

                              COMMIT

                               

                              Update - one of my other colleagues is now also unable to run Reports with the same error I am getting (successfully used it last week). So it's another piece of the puzzle. I have another colleague who doesn't do anything with LEM, I'm going to install Reports on his system and try access with my LEM logins to see what happens. Since his system is "clean" it would provide a good comparison.

                               

                              Great information and help here, thanks a lot - I really appreciate it!

                          • Re: Error with Reports Manager
                            raingeek

                            Okay, we have some progress and change here....

                             

                            As requested I uninstalled the Reports app. Then I rebooted my system and went into C:\Program Files\ and deleted the "SolarWinds Log and Event Manager Reports" directory. I then re-installed the Reports app using the "Run As Administrator" option.

                             

                            The first time I ran the app, just to be safe, I also did a "Run As Administrator" on the shortcut. It probably wasn't necessary but I figured it couldn't hurt.

                             

                            I entered the Data Source and used the Admin username and password. Then I attempted to run a report (limited query on 24 hours of data to keep it simple). I DID NOT GET AN ERROR. However, the application just sat there with the progress bar moving back and forth for 15 minutes. After that length of time I finally closed the application out. (See attached picture reports1.png)

                             

                            Then I realized that I never attached the Certificate in the Manager Configuration. So I went back, attached the cert and started the same report query. Now I get a new error: Login failed. Error Code: -2147189176

                            (See attached picture reports2.png)

                             

                            I tried all three of my LEM accounts and none were successful.

                             

                            Then I remembered I needed to add the /L in order to capture a log file, so I went back, configured that and tested again. This time I got my original error of "Logon failed." (See attached picture reports3.png)

                             

                            Then the application crashed and generated a Windows error  (see reports4.png).

                             

                            ...and now I think I'm going to call it a weekend and get back to it on Monday. :-)

                             

                            I've attached the screenshots and the newest log file.

                              • Re: Error with Reports Manager
                                earlonics

                                Folks I had the Same problem it was fixed by me changing Yes Changing AD password, its either too long or my password contained a " speech mark once I changed my password to something simple  I was able to login and run reports.

                                 

                                Not sure why this should happen since i can login using my AD via the webportal no problem is there a fix for this?

                                 

                                 

                                Paul

                                  • Re: Error with Reports Manager
                                    raingeek

                                    Interesting. Sadly I won't be able to test this and confirm/deny because my workplace enforces a very strict password complexity rule in the AD environment. However, one of the user accounts I have been testing is a LEM user account (non-AD) and it uses a simple password. It gets the exact same error and symptoms. But that still doesn't rule out your theory on the AD password....

                                     

                                    Thanks for the input. Still bashing my head on this.

                                      • Re: Error with Reports Manager
                                        earlonics

                                        Hi raingeek....

                                         

                                        I've Also been banging my head against the wall all day on this today 6 installs and reviewing group policies and Firewall settings, its been for me a result of the AD account password and the one I enter in the Reporting module....

                                         

                                        it might well be something to do with password length, I had a repeatable password ie my Password says is   passwordpassword   in AD...then if I enter the details within the report module as just "password" I can run the reports no problem which is even odder ( if there is such a word!)

                                         

                                        I have been playing around with different passwords my repeatable password is 18 Characters in total, you just might want to give that a try, if that's any help :-)

                                         

                                         

                                        Paul

                            • Re: Error with Reports Manager
                              raingeek

                              I located and walked through the following Knowledgebase artice:

                              SolarWinds Knowledge Base :: Installing LEM Reports

                               

                              ...and event after following all of the steps (again) I still don't have a resolution. So per the note at the bottom of the article I am going to escalate this to Support. If I get a definitive answer I'll post back here again with an update.

                               

                              Thanks again to everyone for your assistance.

                              • Re: Error with Reports Manager
                                raingeek

                                Here is an update in case anyone else is tracking this or having similar problems. I had a 1.5 hour long tech support call from SolarWinds yesterday on this issue. The tech had even read this discussion to see what had been tried previous to the call. That said he still walked me through the entire process all over again - uninstall and reinstall, etc. In the end the problem is still unresolved and it sounds like I'm going to be escalated to the Dev team for review.

                                 

                                A couple interesting things to pass along:

                                 

                                1. In my SWLEMreports.log file there were some strange SQL errors. The tech connected to my LEM installation as root and rebuilt the LEM hsql database. This had an immediate positive impact on the issue and the sql errors in the log went away.

                                 

                                2. We have Sophos Enterprise suite for anti-virus and device security. The tech was using snort to look at the network traffic between the LEM server and my workstation. It appears that Sophos was blocking or preventing network traffic even though we don't use Sophos firewall product or network management! When we disabled the Sophos client services (Control Panel \ Admin Tools\ Services) the network traffic went through to LEM. Note that Reports worked previously under version 5.7 and 6.0.1 but not under 6.1 - and our Sophos installation has not changed in that time frame. So this is something we are researching on the side.

                                 

                                3. That said, even when Sophos was disabled the Reports application was sporadic - sometimes it worked, sometimes it didn't. There were a lot of "this doesn't make sense" comments on the phone. I think this is the reason for the possible escalation to Dev.

                                 

                                4. The tech identified some memory configuration issues on our LEM appliance (Vmware) - we needed to configure the 'reserved memory' for the appliance. This probably isn't contributing to this particular issue but it's worth passing along.

                                 

                                All in all it was a very interesting and productive phone support session. I'll report back on what happens on the next call.

                                • Re: Error with Reports Manager
                                  rmorton

                                  I had been having an issue with the login failure error message too. I tried different account types but had the same message regardless. Disabling TLS in the connection profile sometimes let it in but it then threw up errors in the log file.

                                   

                                  However, I think I have sorted by running the enabletls command from cmc::ccm# prompt. This enables tls connections to the database.

                                   

                                  I logged in to console as cmc user, type manager to get to manager prompt cmc::ccm#. Typing enabletls checks that you have a valid certificate, enables tls to db then restarts the Manager Service. This has resolved for me and I can now login using tls and without error.

                                   

                                  Hopefully this will help someone else!

                                  1 of 1 people found this helpful
                                    • Re: Error with Reports Manager
                                      raingeek

                                      This was an excellent addition to the thread, thank you!

                                       

                                      I ran the steps as described and when the system checked the existing certificate it threw up this error:

                                       

                                                                 The current certificate does not match the network configuration and it might not work with TLS.

                                       

                                      So I selected the menu option to generate a new certificate, exported it and imported the new cert into my Reports application. Unfortunatly this did not resolve the problem for me, in fact I started getting new memory exception errors (I took screenshots and am forwarding them to my SolarWinds support ticket). Attempting to run a report with or without TLS enabled has the same errors as before, followed by a new memory exception error.

                                       

                                      So thanks for the information, I think it did identify something we hadn't considered before, but I'm still back to where I was.

                                    • Re: Error with Reports Manager
                                      raingeek

                                      It's been a while since the last update. Continuing to work with Customer Support on this issue.

                                       

                                      The three Windows 7 desktops that were previously configured with Reports (and had an operational link to LEM) no longer communicate with LEM. Troubleshooting isn't finding anything obvious. Even a test running Reports in Win7 'safe mode with networking' to eliminate any 3rd-party application conflict didn't get a connection between LEM and Reports. We did a fresh install on a brand new host and still can't get Reports to work.

                                       

                                      One system that was a dedicated Reports system running on a VM of Win2008 Server also went offline. Tech support restarted the manager in LEM and that fixed that problem. So this system is running again, but none of the rest are. We are scratching our head why this one host should be any different - aside from the OS and being a VM it's configured the same.

                                       

                                      The SWMLEMReports.log still shows login authentication errors no matter what has been tried. The list of things attempted is very exhaustive and I'm not going to type it all in.

                                       

                                      So still working on this, no fix as yet.

                                      • Re: Error with Reports Manager
                                        raingeek

                                        Sorry for the slow reply. No, we never really got a satisfactory resolution - but not for lack of trying. I spent hours and hours on the phone with support and they pulled in a lot of different resources trying to figure this out. In the end we got two workstations to work but three others would never work. It's a mystery as all five systems have a common configuration (image) and everyone has identical security privileges. After spending so much time my boss declared that two operational systems was sufficient for our purposes and had me move on to another project.

                                         

                                        I do want to be clear that we got excellent SW support on this, I feel like they tried everything they could and were just as mystified and frustrated as I was that it didn't make sense.

                                         

                                        I'm just starting to look at the 6.2 update and I'm curious to see if that has any impact on this problem.

                                        • Re: Error with Reports Manager
                                          jason.mauer

                                          Hi I know this is a very old post.

                                           

                                          However I am having the exact same issue described here.

                                           

                                          Has anyone been able to resolve this issue?

                                            • Re: Error with Reports Manager
                                              jrouviere

                                              Short answer is that you will want to make sure that you're running at least LEM 6.3.1 and reach out to support.

                                               

                                              I worked on validating earltronics assertion that the password may have something to do with it.  It looks like a double quote (") does generate an error.  It's a different error, however, than what raingeek originally reported (and may have something to do with the version as there have been several updates since then).

                                               

                                              If you're able to run the Database Maintenance Report and post a screenshot of the error you're seeing we may be able to provide some information here, otherwise you'll want to reach out to Support and have them look at the error.

                                                • Re: Error with Reports Manager
                                                  jason.mauer

                                                  Hi jrouviere,

                                                   

                                                  I am on the following versions:  Console - 6.3.1

                                                                                                     Report Manager - 6.3.0.1

                                                                                                    Windows Version - 6.3.9600

                                                   

                                                  I can confirm that the password i am using does not include (") and I am able to communicate with the console on all ports.

                                                   

                                                  When running the "Database Maintenance Report" i get the same error(attached).

                                                   

                                                  - This was working fine, and all of a sudden, that error on all accounts.

                                                   

                                                  Scratching my head on this one.

                                                   

                                                  Thanks for your help so far.