Slow Orion Login Timeout connect to domain

We've had an issue for a while that we've just identified what looks like the cause.

When logging in to the console with MYDOMAIN\UserName, it takes exactly 50 seconds (+/-2) from pressing enter to being logged in.  The 50 seconds is reproducible every time.

Looking at the OrionWeb.log (from C:\ProgramData\SolarWinds\Logs\Orion\), I see the following occurs every time someone tries to log in:

WARN  SolarWinds.Orion.Web.Authentication.Windows.DirectoryServices - (null)  Can't connect to domain 'OTHER-DOMAIN-WE-TRUST.com': System.Runtime.InteropServices.COMException (0x8007203A): The server is not operational.

As soon as this warning appears in the log file, the user is then logged in straight away.

The domain OTHER-DOMAIN-WE-TRUST.com is a domain of a partner organisation that my organisation's domain trusts, as a one-way trust.

Why is it trying to authenticate against that domain first, instead of using the MYDOMAIN in the login field?

  • If we change to LDAP authentication, it works instantly, but we can't rely on a single DC being specified. Changing back, causes the delay again.

  • Hey @AubsUK, 

    I think I got a fix for you. I was running into a similar problem with 2 domains that we recently added a one-way trust with and this solved it for me after 2 weeks of back and forth with Solarwinds Support. Thought I would loop around and provide the solution here for you if it has not already been resolved, but also for other members of the community that stumble upon this post in their googling efforts.

    The Solarwinds website has a hidden setting that you can edit to blacklist authentication to certain domains so that the Orion environment ignores these domains during the web authentication process. I will outline how this was completed in my environment (version 2023.3.0.1591), but please be aware I am not familiar with your environment and I can't be held responsible for how this change will affect your environment.

    Step 1: As always, BACKUP your SQL database before making any changes. I didn't have any troubles with this, but you don't want to be making any changes a stranger on the internet is telling you to do, or any changes in general without a backup. 

    Step 2: Logon to your Solarwinds Application Server (likely where you found the OrionWeb.log file) and open up the application:
    SolarWinds Platform > Database Manager

    Step 3: Click the Add Orion Server button, and expand the Orion server in the drop down followed by right clicking your Orion DB (not your logging or other module specific DB's) and selecting New Query...

    Step 4: I recommend as a baseline to read your DB to see if this value already exists so you can tell if this has been done before in your environment. To do so you will have to run the following SQL query and observe the result.
    SELECT TOP 10 * FROM [dbo].[WebSettings] WHERE SettingName = 'InoperableDomainsBlacklist'

    Step 4a: If there is no result, outputted, that means this value does not exist which is the default state of the Orion environment.
    Step 4b: If a SettingValue is returned, you should note those down and investigate if these domains need to remain.

    Step 5: Here comes the big scary change we have been building up too. We are going to do a SQL insert query to add the domains from your log file to your DB. You will need to have the domains you found in your log file OrionWeb.log (from C:\ProgramData\SolarWinds\Logs\Orion\).
    In your query window, you will need to enter the following, substituting the place holder <invalid domain> for the domain in your logs:
    INSERT INTO WebSettings VALUES ('InoperableDomainsBlacklist', '<invalid domain>')

    Example of this using a single domain of OTHER-DOMAIN-WE-TRUST.com:
    INSERT INTO WebSettings VALUES ('InoperableDomainsBlacklist', 'OTHER-DOMAIN-WE-TRUST.com')

    Example of this using multiple domains (as my case was) of DOMAIN1.com and DOMAIN2.com:
    INSERT INTO WebSettings VALUES ('InoperableDomainsBlacklist', 'DOMAIN1.com;DOMAIN2.com')

    Step 6: Once executing that INSERT SQL query, you should see that 1 row was affected. You can confirm this by repeating step 4 to confirm that the change has taken affect using the SELECT SQL query.

    Step 7: Test the authentication time, and you should notice a significant improvement (mine went from 50 seconds to less that 3 seconds). If this process did not improve your login times, I would either check the logs again for any missed domains, or possible typo'ed domains and repeat the steps if so.

    Really hoping this helped you or other people in the same boat.
    This is my first time responding to a question on THWACK as naturally I am an introvert that lurks, but I thought since I had just gone through the support case on this it was worth posting. Feel free to reply to this post if it helped out so that I know it has helped someone. Slight smile

  • We have the same issue, confirmed with the logs and for 7 of our domain trusts. The login process was unbearable for us and kept staff from utilizing the system. Applied the steps above and it loads in a few seconds now. THANK YOU for posting the fix!

  • That's an awesome solution, and thank you for sharing it.

    We did eventually get our issue fixed with SolarWinds support.  We're on an older version, but upgrading this week.

    It looks like in the version we had, Orion had a 'New Group Membership Manager, setting it to not use that fixed it.  Below is the solution for the specific issue we had:

    From the logs, we can see the following:
     

    2023-09-07 08:45:22,151 [229] (2) WARN  SolarWinds.Orion.Web.Authentication.Windows.DirectoryServices - (null)  Can't connect to domain 'OTHER-DOMAIN-WE-TRUST.com': System.Runtime.InteropServices.COMException (0x8007203A): The server is not operational.

    For isolation purposes, kindly try out the following:

    NOTE: Please ensure that you have a backup of your database and snapshot of your VM before making any changes just in case there is a need to revert.
     

    •         INSERT INTO WebSettings VALUES ('UseNewGroupMembershipManager', 'False')

    Try to refresh the page and hopefully, it helps. 

    It may be related to this issue: Web Console Windows Authentication Fails for Orion Platform after Upgrade to 2020.2.6 (solarwinds.com)

  • Thanks for sharing, I ran across this and it was slowing down my SWQL Studio and Powershell scripts as well. Made the changes and they are running well again.