File under #2020:
You are required to change all Orion-related passwords in response to a security breach or some other event. The affected credentials may include the following list, and possibly more:
This write-up will focus on the first four types of accounts, which should all be domain-based accounts in Active Directory, unless you are running Orion services as the default NETWORK SERVICE security principle.
* You may find that your Orion services are running as NETWORK SERVICE instead of a service account. Per SolarWinds documentation, this is a supported configuration.
Let's skip to the spoiler: in this kind of scenario, the least impactful and most secure approach is to create new domain-based accounts instead of changing passwords.
Step 1: Orion Service Account
Step 2: WMI Monitoring Accounts
Step 3: Mailbox Accounts
How are your alert emails and job notification emails sent from Orion? If you have created an Exchange mailbox for your original service account to send alerts, here is another best practice to consider: use a separate account with its own mailbox. When using the same account for email and for the application itself, you increase your risk of an accidental account lockout bringing down the whole application. This has potential to happen any time the SMTP settings are changed for the Orion platform, and any time an NCM job's email notifications are set.
Does this one require a brand new account and brand new mailbox? I'll leave that up to you and your security team. A password reset may be deemed sufficient. If new accounts are mandated across the board, then be prepared to help recreate any inbox rules that your alert recipients have configured in their mail clients.
Step 4: Final Validation and Cleanup
These final steps may vary depending on how your environment is configured and what your policies around account lifecycle require.
Q: Why shouldn't we just change the passwords?
This can certainly be done, but there is actually more work involved, more potential disruption, and ir provides less guarantee of mitigation if any of your accounts were ever compromised. At a very high level, the process for this approach would include the following steps.
(Temporarily stopping the use of WMI credentials in Orion is required because the accounts will instantly get locked out and cause a LOT of errors after you change the password in Active Directory and the monitors keep trying to use the old password.)
Notes / To-Do:
We will update this document ASAP with more details:
2020/12/16 12:40 EST - Added to list of credential types in situation description.
2020/12/16 15:03 EST - Added link and note about support for running services as NETWORK SERVICE.
2020/12/16 20:54 EST - Added more information regarding Exchange mailbox accounts.
Any update on when we get information on this one: "WMI monitoring accounts that do not require administrator privileges"
This would be especially helpful information to document better. I'm surprised @aLTeReGo and the SAM team haven't provided anything about this and for security best practices for implementing: SAM templates that leverage powershell, AppInsight for Active Directory and other templates that could be used to monitor a Windows Active Directory environment
I have not yet had a chance to automate changing of the WMI credentials on nodes, but that can certainly be done. (Thankfully I'm not in a place that I must change them, but am going to do so anyway.)
The best starting point would be to create a custom Orion report with the following SQL query and review it. This will show you all of your nodes grouped by which WMI credentials they use. Of course, it shows the saved name of the credential, and not the credential itself.
select Credential.Name, nodes.Caption, nodes.NodeID, nodesettings.SettingName, NodeSettings.SettingValue from Nodes join NodeSettings on nodes.NodeID = Nodesettings.NodeID join Credential on nodesettings.SettingValue = Credential.ID where nodes.ObjectSubType = 'WMI' and nodesettings.SettingName = 'WMICredential' order by Credential.Name, nodes.Caption, nodesettings.SettingValue asc
This will help you grasp the scope of change that needs to be made, and similar SQL can be used for the automation part.
When you refer to the 'automation' part, do you mean SQL can be crafted to change the assigned Windows Credential for each node via this method?
This would really help me personally, since I'm at a place where I've got a broken install - the website throws errors when I attempt to edit any WMI node, regardless of credential used. If I could just do this via the DB, I'd have one less problem to deal with. I'm waiting on support, but it's been three days and I haven't heard squat from them.
Thanks for any info.
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process.