Solarwinds locking out AD account!

FormerMember

Solarwinds is locking out my AD account!

I have tried looking at:

Account lockout due to expired or mistyped credentials can occur in the following areas:

NCM Jobs - Check Scheduled Tasks in Windows Server 2003 or Task Scheduler in Windows Server 2008 for any jobs running under your credentials. The jobs are set in Schedule > Display Edit / Jobs in NCM.
SAM Credentials - Log in to the Web Console > Settings > SAM Settings > Credentials and check the credentials used.
NPM Monitoring Servers with WMI - In NPM 10.2 and above, servers can be monitored using WMI in addition to SNMP and ICMP. Check the credentials that Orion is using here. You can check the Nodes table for servers being monitored with WMI and failing. Select Caption, IP_Address FROM Nodes WHERE ObjectSubtype = 'WMI'.
NPM Scheduled Tasks - Common tasks that can be scheduled in NPM are Scheduled Reports and Unmanaging Elements. These tasks appear in the Task Scheduler Library. Check credentials for used for these jobs.
IPAM Jobs - The Active Directory account is used for scanning the DHCP scopes. Check the credentials used for these jobs.
VCenter Credentials – Check the credentials used for virtualization.
UDT Module – See /Orion/UDT/Admin/Credentials/CredentialManager.aspx to check the credentials used.
Core Windows Credentials - See Settings->Windows Credentials to check the credentials used.
Orion Services in Service Manager - Create a separate OU and place this service account in that OU. Apply a specific set of policies to that OU excluding it from the “password expires after X days security policy" so customers don’t have to change the password after every X days.
Alerts with Actions – Check the credentials used for these alerts.

none of these have worked? it gets locked out once a day?

Any ideas?

Thanks

Find more posts tagged with

Accepted answers

All comments

solaradmin

I see this with my account as well, although its a domain admin ( I know shame on me) it still doesn't have access to some servers, so it will cause account lockout

ryoung

foxy_no5 did you get your issue resolved?

FormerMember

No not yet?

mr_wu66

Old post which has more than likely been resolved. Either way, this is my story!!!

So... Solarwinds support (FACE PALM) was no help. However, I had the same issue and my AD Admin account was locking out within seconds. I had to troubleshoot this with AD tools to first pinpoint the machine/server in questions. I then disabled all Solarwinds services and started them one by one trying a process of elimination. Finally, I created a service account to log in to the server and re-poll with instead of my AD Admin account.

If you've looked at all of the above you more than likely had nodes added that were being polled with your AD credentials. You can actually see how many nodes are assigned to specific credentials in the Orion GUI > Settings > Credentials > Manage Windows Credentials. Unfortunately it only gives you a count and not which devices as of version 2014.1.0 (Orion). I then went to my Windows nodes, Orion GUI > Settings > Node & Group Management > Manage Nodes > Windows, to mass updated the polling credentials. This resolved my issue that perplexed me for a few days...

Hope this helps someone else out there!!!

FormerMember

I did managed to resolve this! It was my account with an old cached password that was trying to access the internet for the license manager!

mpocanac

We had a similar problem after a password change. Apparently it was bad credential information being cached in the ProgramData on multiple Orion servers (main and APE).

Fix was to do the following on all APEs and main server:

1. Stop all Orion services on server

2. Run a CMD prompt on server as Administrator

3. Navigate to c:\programdata\solarwinds\installers

4. Run the following installers to remove the components in this order:

CollectorInstaller.msi
JobEngine.msi
JobEngine.v2.msi

5. Run them again to perform an installation in this order:

JobEngine.msi (Typical installation)
JobEngine.v2.msi (No options)
CollectorInstaller.msi (No options)

6. Restart all services

randolfini

I wanted to chime in here. Similar scenario, I used a service account for the APE as it was in another domain. I forgot to document the account and had to reset the password in AD. I updated in Solarwinds GUI (NPM, SAM) but it was apparently cached and locking the account out. Path I took to resolve.

Support asked me to:

On the Add polling engine server:

Stop the v2 Job Engine service.
Browse to hidden folder: C:\ProgramData\SolarWinds\JobEngine.v2\Data\
Rename JobEngine35.sdf to JobEngine35.sdfOLD
Next, make a copy of JobEngine35 - Blank.sdf and rename it to JobEngine35.sdf
Next, do a right click on JobEngine35.sdf, select Properties. Make sure it is NOT set to Read-only.
Next, restart the v2 Job Engine service.
Monitor if tr\monitoring still locked.
If yes, please go thru the following KB: https://support.solarwinds.com/Success_Center/Network_Performance_Monitor_(NPM)/Orion_is_locking_me_out_of_active_directory

This did not resolve the problem. I follow the KB and performed the SQL query to find what was using WMI " Select Caption, IP_Address FROM Nodes WHERE ObjectSubtype = 'WMI' ", this did not resolve either.

Finally followed Mpocanac's steps above on the APE only and that resolved.

Thanks!

mu5ic92

For anyone who might still be running into this issue. The problem I was having with one of my NPM users was actually the Web Performance Monitor (WPM). The user has recorded transactions that would run daily and log into a website using his credential. Log into the server where you have WPM installed and deleted or adjust the transaction from within the recorded. Start Menu -> WPM Recorder. Login and connect to the NPM server using your network credentials if you use them.

ecklerwr1

I've done this on accident with WPM too... it took me a little while to figure it out too and was a pain.

bobmarley

This happened to me the other day as well. One of the other administrators in Solarwinds was doing her monthly password changes for the SAM transactions she runs and

after she changed the password in SAM the account kept locking up.

Turns out the same account was also being used in our lab Solarwinds and it had not been changed yet - so was using the old password and thus locking the account.

-Don't forget your 'other' environments if you have them.

vjerez4129

That worked for me thanks for the help! i was completely stumped, because i deleted the AD account from everything i could think of in Orion

ray_lbs

While I didn't try the above solutions, I had to finally change the credential that was stored in Orion to use a different account so it would stop locking out the original. I did try the SQL query, but more things were failing than succeeding, so it was not helpful at all. 3 hours later, it's still attempting to use the original account from time to time. Orion should automatically kill all the jobs in the queue for nodes using a credential when it's changed.

ecklerwr1

I think this happens to all of us at some point or another. We use service accounts that change on a much longer time period for anything automated in Orion. The last time this happened to me it was WPM which I had set up a series of actions in the player and it was using my credentials... of course when I change my password after 60 days WPM then proceeded to start locking my account every hour... bummer!

jbillington01

I've attempted all of the methods listed here and still no luck. I have a user who had been using their own AD account for polling of some devices. I believe the user recently changed their password and now they are getting locked out every few minutes. I removed the user from the credentials manager, killed all tasks utilizing the users credentials. The user is still getting locked out. How can I find out which nodes are using this users credentials so that I can switch it with the newly created service account. (We don't have SAM the module).

mlong3

I have had the same problem with my domain admin account continuing to lock after changing my password.The lockouts were showing on our prod Orion server. I checked all the credentials, services, db, jobs, and couldn't find the cause. I contacted SolarWinds support and they couldn't resolve it either. There were also locks on our High Availability server for Orion. When I shut the HA server down, the locks still continued. The source still appeared to be the Orion Prod server. Here was my cause: The Solarwinds HA application encrypts and stores the credentials of the user who monitors and changes DNS during failover. (I had checked the credentials and HA settings, but the credentials were blank, so I assumed that this wasn't the source.) Unfortunately, the password isn't automatically changed in HA when changed in A.D., so the account kept locking out. We needed an Orion service account, that never expires, and is in the DNSAdmins group in A.D.

To change the HA DNS credentials, go to Settings, High Availability Setting. On the "High Availability Deployment Summary", select the Commands drop down and select "Edit Pool". On the "Set Up High Availability Pool" page, click the Next button. On the "DNS Settings" page, enter Orion service account in "User Name", the "Password" and then select "Test". This should complete successfully.

mesverrum

Cannot say how many of my engagements I have to press them on creating a service account BEFORE we start setting things up. "Oh we can just use my creds here for now" is inevitably going to leave you in this position.

solaradmin

What other monitoring tools are you using? Did someone else use the service acocunt.... Or its being used as the SQL service account. I would shut down the Orion server. Verify the account lockouts stop

Account lockouts are the most difficult things to troubleshoot. There are tools from Microsoft to assist a

solaradmin

https://www.microsoft.com/en-us/download/details.aspx?id=18465

solaradmin

Sorry, just read the once a day part. It sounds like a scheduled task. Check those.

viorelp

Steps 1-6 help me resolve my similar problem, after I changed the password for the account use SW modules. Thank you!

RaviK

You may also want to check credentials configured for Network Insight example Cisco Nexus devices, ASA's and others. Note these maybe different than the NCM credentials.

jdbrp

This worked like a champ after hours of trying everything else...