I'm very excited to announce that SolarWinds Server Configuration Monitor (SCM) 1.1 is now available for download! This release expands on SCM 1.0 capabilities, both giving more detail for each change detected, and adding a new data source that can be analyzed for changes:

 

  • Detect “Who made the change” for files and registry
  • Detect changes in near real-time
  • Deploy PowerShell scripts and track changes in the output (with links to additional example scripts)
  • Set baselines for multiple nodes at once

 

Who made the change? In near real-time

SCM 1.0 is good at detecting changes in your Windows files and registry, but it didn't tell you who made the change, leaving you to do some additional investigative work. SCM 1.1 adds "who made the change" by leveraging our File Integrity Monitoring (FIM) technology, which also detects changes in near real-time -- a double benefit. Near real-time allows us to catch changes almost as they happen, and gives us a separate record for each change, even if changes are happening in rapid succession.

 

Turning on "Who made the change"

After you install or upgrade to SCM 1.1, you can easily turn on the "Who Made the Change" feature for the servers you want to monitor via a wizard:

  • From the "Server Configuration Summary -> What's New Resource," click the Set Up "Who Made the Change" Detection button
  • From the "All Settings -> Server Configuration Monitor Settings -> Polling Settings Tab," click the Set Up Who Detection button

Either way, it starts the "Who Made the Change" wizard.

The first step tells you about what happens when you turn on "Who Made the Change" detection:

The second step allows you to define the server exclusion list and turn on the feature:

Once you press Enable Who Detection, SCM will push out FIM driver to the agent(s) and turn it on, so file and registry changes will be monitored in near real-time rather than polled once a minute as in SCM 1.0. You can always come back and change the exclusion list or turn off "Who Made the Change" later.

 

Where to see "Who made the change"

You can see who made the change (user and domain) in a number of places, represented by the person icon.

  • SCM Summary: Recent Configuration Changes resource
  • Node Summary: Configuration Details and Recent Configuration Changes resources
  • Node: Content comparison, note the time I added to the file matches the time SCM shows the file changed.

Alerting

When building an alert, you can filter on "Who made the change" and add it to the text of your alert.

 

Reporting

The out-of-the-box SCM report includes "Who made the change" data.

 

Deploy and monitor the output of PowerShell scripts

Everyone's environment is different, and SCM could never monitor everything you want to "out-of-the-box." So, we added the ability to deploy and execute PowerShell scripts and compare the output over time. Now, configuration monitoring is only limited by your imagination and scripting super powers.

 

Adding a new script

I created a new Profile for this test, but you can add scripts to your current Profiles too.

First, create a new Profile and click Add to add a new element.

To add a PowerShell script configuration element:

  1. Choose PowerShell script as your Element type.
  2. Paste your script into the box.
  3. Click Add to add the element to the profile, then add again to save the profile.

Deploy and enjoy!

Once your new (or modified Profile) is ready, you can deploy it to one or more agents. From Server Configuration Monitor Settings > Manage Profiles, select the profile and click assign, then pick the servers you want, and walk through the wizard. SCM will deploy the scripts and start executing them on schedule.

Comparing the output

Comparing the output of the script over time works like any other source (file, registry, asset info) in SCM. You can set baselines and see changes in the content comparison. As you can see, the entire output of the script is captured and stored.

Mix and match elements in profiles

Don't forget -- one of the great things about SCM is you can mix and match elements in a single profile. Mix and match registry setting, multiple files, and PowerShell scripts into a single profile to monitor interesting aspects of your configurations.

 

Check Out Some Cool PowerShell Examples by Kevin

SolarWinds' own Technical Community Manager KMSigma put together some awesome examples of what SCM can do: Manage and Monitor PowerShell Scripts

Keep a lookout in our SCM forums for more PowerShell script examples in the future, and feel free to post your scripts too.

 

Set/Reset baselines for multiple nodes at once

Our early customers in large environments were limited to setting/resetting baselines one node at time, which was very painful when the dozens or hundreds of servers were updated (like a Windows update), so we addressed it quickly in this release. Now from the Server Configuration Monitor Settings screen, you can pick multiple servers, see a quick summary of the number of baselines you'll be updating, and then reset the baselines to the current output -- easy as 1-2-3.

What's next?

Don't forget to read the SCM 1.1 Release Notes to see all the goodness now available.

 

If you don't see the features you've been waiting for, check out the What We're Working on for SCM post for a list of features our dedicated team of configuration nerds and code jockeys are already researching. If you don't see everything you've been wishing for, add it to the Server Configuration Monitor (SCM) Feature Requests.

I’m pleased to announce the General Availability of Log Analyzer (LA) 2.0 on the Customer Portal.  You may be wondering what Log Analyzer is. The artist formally known as Log Manager for Orion has undergone a transformation. It has evolved past its former life as a 1.0 product and become Log Analyzer 2.0. Log Analyzer was selected after extensive research to better understand what our users would call a product that solves the problems our tool solves based on our feature set. I hope you like the new name!

 

This release includes Windows Event Support, Log Export, Log Forwarding and Rule Improvements as well as other items listed in the Release Notes.

 

 

 

Windows Events

As a System Administrator, closely monitoring Windows Events is vital to ensuring your servers and applications are running as they should be. These events can also be hugely valuable when troubleshooting all sorts of Windows problems and determining the root cause of an issue or outage. While there are vast array of Windows Events categories, the three main categories you'll likely focus on when troubleshooting are the Application (events relating to Windows components), System (events related to programs installed on the system) and Security (security related events such as authentication attempts and resource access). Trawling through Windows Event Viewers to find the needle in the haystack on individual servers can be a laborious task. Having a tool such as Log Analyzer can be a real life saver when it comes to charting, searching and aggregating these Windows Events. Thanks to the tight integration with Orion, you can view your Windows Events alongside the performance data collected by other tools such as NPM and SAM. Worth noting that you can also add VMware Events into the mix, thanks to the latest Virtualization Manager (VMAN) release.

 

In order to start ingesting Windows Events with Log Analyzer, you need to install the Orion Agent on your Windows device. Windows Event Forwarding is also supported, so if you prefer to forward events from other nodes to a single node with the Orion agent installed, that's an option too. By default, we collect all Windows Application and System events, along with 70 of the most common Windows Security Events. You can view more information on setting up Windows Event Collection here.

 

Once you have the agent installed and added the node(s) to Log Analyzer, you'll see the Events within the Log Viewer. Events are automatically tagged with Application, System or Security tags. Predefined rules are also included out of the box which tag events such as Authentication Events, Event Logs Cleared, Account Creation/Lockout/Deletion, Unexpected Shutdowns, Application Crashes and more.

 

 

Windows Events are also supported in PerfStack, enabling you to correlate performance data with Windows Events. For example, you can see below there are memory spikes on a SQL Server, with some corresponding Windows Events and Orion Alerts. Drilling into the Windows Events you can clearly see there is insufficient system memory which is causing the Node Reboot and SQL Server Insufficient Resources alerts.

 

 

Log Forwarding

Log Analyzer shouldn't be seen as a dead end for your log data. There may be times when you need to forward import syslog/traps to another tool such as an Incident Management or SIEM for further processing/analysis. This release includes a new 'Forward Entry' rule action which enables you to forward syslog/traps to another application. You can keep the source IP of the entry intact or replace with Orion's IP address:

 

 

 

Log Export

When troubleshooting problems it's often necessary to share important log data with other team members, external vendors or attach to a helpdesk ticket. You can now do so thanks to the new Export option within the Log Viewer.

 

 

 

Rule Improvements

We've added some pre-populated dropdown menus for fields such as MachineType, EngineID, Severity, Vendor and more to make it even easier to create log rules. It is now also possible to adjust the processing order of the rules.

 

 

The team is already hard at work on the next version of LA, as you can see covered here in the What We're Working On post. Also, please keep the feedback coming on what you think and what you would like to see in the product in the Feature Requests section of the forum.

Virtualization Manager (VMAN) 8.4 is now available and can be downloaded from your customer portal. In recent releases, we brought you VMware vSAN monitoring, container support, and better centralized upgrades to your deployment overall.

 

 

VMware Event Monitoring, Correlation, and Alerting

 

As a virtualization admin, it's a primary concern to track the many changes that occur in dynamic and often automated virtualization environments. While many virtualization vendors tout that the simplicity of their solution alleviates the need for admins to worry, I err on the side of caution. With VMware event monitoring, you now have real-time access to alert and correlate VMware's alarms, health checks, events, and tasks to issues in your environment. Ephemeral events such as vMotions are now easily tracked, and if you also have Log Analyzer, you can tag them for future cataloging.

Looking at my VMware Events summary, there are quite a few warning and critical events in the last hour. Filtering down to the warning events to do deeper inspection, I can see four of them are warning me of a failed migration for virtual machine DENCLIENTAFF01v

Clicking on one of these events allows me to drill in to get more context. Clearly, I need to look at the configuration of my vMotion interface.

Clicking "Analyze Logs" allows me to have better filtering and is also where I would configure processing rules to start configuring real-time alerting on these VMware events. Yes, event collection is real-time, and as a result, your alerts configured on these events are also triggered in real-time. If you want to be alerted to host connection changes, or when vMotions are triggered when they aren't supposed to be, you now can be alerted immediately.

 

For those of you who have Log Analyzer, you have even more troubleshooting tools that play very nicely with this VMAN feature. Are you looking to visually see occurrences of this event over time? Easy. Click "Analyze Logs" to navigate to the Log Viewer. Your Log Viewer will differ in that you'll have a visual graph to see how many times this event has occurred over the specified time period. In the example below, I increased the time to two hours, and searched for "vMotion." In addition, I've used the tagging feature to tag all events like this with a "vMotion" tag.

So how do I correlate this to problems? By using PerfStack dashboard.

After troubleshooting your issues, simply save the PerfStack project and put that project on your NOC view for future visibility.

 

Deeper Dives and Other Features

 

For a more in depth look at the VMware events feature check out these documents. Let me know if you have use cases that require real time alerting, monitoring and reporting so we can consider putting them in as OOTB content.

 

For those of you who are curious what we have for those users who do not need VMware event visibility check out these documents for more details:

 

Next on the VMAN Roadmap

 

Don't see what you're looking for here? Check out the WHAT WE'RE WORKING ON FOR VIRTUALIZATION MANAGER (UPDATED NOVEMBER, 2019)  post for what our dedicated team of virtualization nerds and code jockeys are already looking at. If you don't see everything you've been wishing for there, add it to the Virtualization Manager Feature Requests

 

This version of VMAN is compatible with the legacy VMAN 8.1 appliance; however, all the newly available features are only on VMAN on the Orion Platform. If you're using the appliance on your production VMAN installation, I recommend that you consider retiring the appliance at your earliest convenience to reap all the benefits of the new features we are developing for VMAN on Orion. If you cannot retire the appliance for any reason, I'm very interested in your feedback and reasons, and would love to see them listed out in the comments below.

Helpful Links

Anyone who knows me knows that I’m a fan of PowerShell. “Fan” is a diminutive version of the word “fanatic,” and in this instance both are true. That’s why I was so excited to see that PowerShell script output is now supported in Server Configuration Monitor (SCM).

 

Since SCM’s release, I’ve always thought it was a great idea to monitor the directory where you store your scripts to make sure they didn’t vary and to validate changes over time, even going in and reverting them in case there was a change without approval. However, that part was available in the initial release of SCM. Using PowerShell with SCM, you can monitor your C:\Scripts\*.ps1 files and get notified when any deviate from their baselines.

 

Using PowerShell scripts to pull information from systems you’re monitoring is only limited by your scripting prowess. But let me say this plainly: You don’t need to be a scripting genius. The THWACK® members are here to be your resources. If you have something great you wrote, post about it. If you need help formatting output, post about it. If you can’t remember how to get a list of all the software installed on a system, post about it. Someone here has probably already done the work.

 

Monitoring the Server Roles

Windows now handles many of the “roles” of a machine (Web Server, Active Directory Server, etc.) based on the installed features. There never was a really nice way to understand what roles were installed on a machine outside the Server Manager. This is especially true if you’re running Windows Server Core because it has no Server Manager.

 

Now, you can just write yourself a small PowerShell script:

Get-WindowsFeature | Where-Object { $_.Installed } | Select-Object -Property Name, DisplayName | Sort-Object -Property Name

 

…and get the list of all features displayed for you.

 

Name                      DisplayName

----                      -----------

FileAndStorage-Services   File and Storage Services

File-Services             File and iSCSI Services

FS-Data-Deduplication     Data Deduplication

FS-FileServer             File Server

MSMQ                      Message Queuing

MSMQ-Server               Message Queuing Server

MSMQ-Services             Message Queuing Services

NET-Framework-45-ASPNET   ASP.NET 4.7

NET-Framework-45-Core     .NET Framework 4.7

NET-Framework-45-Features .NET Framework 4.7 Features

NET-WCF-Services45        WCF Services

NET-WCF-TCP-PortSharing45 TCP Port Sharing

PowerShell                Windows PowerShell 5.1

PowerShell-ISE            Windows PowerShell ISE

PowerShellRoot            Windows PowerShell

Storage-Services          Storage Services

System-DataArchiver       System Data Archiver

Web-App-Dev               Application Development

Web-Asp-Net45             ASP.NET 4.7

Web-Common-Http           Common HTTP Features

Web-Default-Doc           Default Document

Web-Dir-Browsing          Directory Browsing

Web-Dyn-Compression       Dynamic Content Compression

Web-Filtering             Request Filtering

Web-Health                Health and Diagnostics

Web-Http-Errors           HTTP Errors

Web-Http-Logging          HTTP Logging

Web-ISAPI-Ext             ISAPI Extensions

Web-ISAPI-Filter          ISAPI Filters

Web-Log-Libraries         Logging Tools

Web-Metabase              IIS 6 Metabase Compatibility

Web-Mgmt-Compat           IIS 6 Management Compatibility

Web-Mgmt-Console          IIS Management Console

Web-Mgmt-Tools            Management Tools

Web-Net-Ext45             .NET Extensibility 4.7

Web-Performance           Performance

Web-Request-Monitor       Request Monitor

Web-Security              Security

Web-Server                Web Server (IIS)

Web-Stat-Compression      Static Content Compression

Web-Static-Content        Static Content

Web-WebServer             Web Server

Web-Windows-Auth          Windows Authentication

Windows-Defender          Windows Defender Antivirus

WoW64-Support             WoW64 Support

XPS-Viewer                XPS Viewer

 

This is super simple. If someone adds or removes one of these features, you’ll know moments after it’s done because it would deviate from your baseline.

Monitoring Local Administrators

This got me thinking about all manner of other possible PowerShell script uses. One that came to mind immediately was local security. We all know the local administrator group is an easy way to have people circumvent security best practices, so knowing who is in that security group has proven difficult.

 

Now that we don’t have those limitations, let’s look at the local admins group and look at local users.

 

Get-LocalGroupMember -Group Administrators | Where-Object { $_.PrincipalSource -eq "Local" } | Sort-Object -Property Name

 

Now, you’ll get returned a list of all the local users in the Administrators group.

ObjectClass Name                         PrincipalSource
----------- ----                         ---------------
User        NOCKMSMPE01V\Administrator   Local
User        NOCKMSMPE01V\Automation-User Local

Now we’ll know if someone is added or deleted. You could extend this to know when someone is added to power users or any other group. If you really felt like going gang-busters, you could ask for all the groups, and then enumerate the members of each.

 

Local Certificates

These don’t have to be relegated to PowerShell one-liners either. You can have entire scripts that return a value that you can review.

 

Also, on the security front, it might be nice to know if random certificates start popping up everywhere. Doing this by hand would be excruciatingly slow. Thankfully it’s pretty easy in PowerShell.

 

$AllCertificates = Get-ChildItem -Path Cert:\LocalMachine\My -Recurse

# Create an empty list to keep the results

$CertificateList = @()

ForEach ( $Certificate in $AllCertificates )

{

    # Check to see if this is a "folder" or a "certificate"

    if ( -not ( $Certificate.PSIsContainer ) )

    {

        # Certificates are *not* containers (folders)

        # Get the important details and add it to the $CertificateList

        $CertificateList += $Certificate | Select-Object -Property FriendlyName, Issuer, Subject, Thumbprint, NotBefore, NotAfter

    }

}

$CertificateList

 

As you can see, you aren’t required to stick with one-liners. Write whatever you need for your input. As long as there’s output, SCM will capture it and present it in a usable format for parsing.

FriendlyName : SolarWinds-Orion
Issuer       : CN=SolarWinds-Orion
Subject      : CN=SolarWinds-Orion
Thumbprint   : AF2A630F2458E0A3BE8D3EF332621A9DDF817502
NotBefore    : 10/12/2018 5:59:14 PM
NotAfter     : 12/31/2039 11:59:59 PM

 

FriendlyName :
Issuer       : CN=SolarWinds IPAM Engine
Subject      : CN=SolarWinds IPAM Engine
Thumbprint   : 4527E03262B268D2FCFE4B7B4203EF620B41854F
NotBefore    : 11/5/2018 7:13:34 PM
NotAfter     : 12/31/2039 11:59:59 PM

 

FriendlyName :
Issuer       : CN=SolarWinds-Orion
Subject      : CN=SolarWinds Agent Provision - cc10929c-47e1-473a-9357-a54052537795
Thumbprint   : 2570C476DF0E8C851DCE9AFC2A37AC4BDDF3BAD6
NotBefore    : 10/11/2018 6:46:29 PM
NotAfter     : 10/12/2048 6:46:28 PM

 

FriendlyName : SolarWinds-SEUM_PlaybackAgent
Issuer       : CN=SolarWinds-SEUM_PlaybackAgent
Subject      : CN=SolarWinds-SEUM_PlaybackAgent
Thumbprint   : 0603E7052293B77B89A3D545B43FC03287F56889
NotBefore    : 11/4/2018 12:00:00 AM
NotAfter     : 11/5/2048 12:00:00 AM

 

FriendlyName : SolarWinds-SEUM-AgentProxy
Issuer       : CN=SolarWinds-SEUM-AgentProxy
Subject      : CN=SolarWinds-SEUM-AgentProxy
Thumbprint   : 0488D26FD9576293C30BB5507489D96C3ED829B4
NotBefore    : 11/4/2018 12:00:00 AM
NotAfter     : 11/5/2048 12:00:00 AM

 

FriendlyName : WildcardCert_Demo.Lab
Issuer       : CN=demo-EASTROOTCA-CA, DC=demo, DC=lab
Subject      : CN=*.demo.lab, OU=Information Technology, O=SolarWinds Demo Lab, L=Austin, S=TX, C=US
Thumbprint   : 039828B433E38117B85E3E9C1FBFD5C1A1189C91
NotBefore    : 3/30/2018 4:37:41 PM
NotAfter     : 3/30/2020 4:47:41 PM

Antivirus Exclusions

How about your antivirus exclusions? I’m sure you really, really want to know if those change.

 

$WindowsDefenderDetails = Get-MpPreference

$WindowsDefenderExclusions = $WindowsDefenderDetails.ExclusionPath

$WindowsDefenderExclusions | Sort-Object

 

Now you’ll know if something is added to or removed from the antivirus exclusion list.

C:\inetpub\SolarWinds
C:\Program Files (x86)\Common Files\SolarWinds
C:\Program Files (x86)\SolarWinds
C:\ProgramData\SolarWinds
C:\ProgramData\SolarWindsAgentInstall

Trying to find this out by hand would be tedious, so let’s just have SCM do the work for you.

 

This is all just a sample of the power of PowerShell and SCM. We’d love to know what you’ve got in mind for your environment. So, download a trial or upgrade to the latest version of SCM. Be sure to share your excellent scripting adventure so the rest of us can join in the fun!

In part 2 of "What's New in SAM 6.8" we are going to discuss the improved Cisco UCS monitoring that is shipping with SAM 6.8

(If you were looking for part 1 it is over here: SAM 6.8 What's New Part 1 - AppInsight for Active Directory )

Those of you who have been using SAM with NPM for a while are probably already aware that some support for UCS monitoring is possible in Orion. UCS support has been re-written to be utilized by any combination or standalone deployment of SAM, VMAN or NPM Additionally we added a new overview resource that let's you visualize your UCS environment. We fleshed out the hardware health support to include all the pieces. Fabric Inter-connects, Chassis, Blades and any rack mount UCS servers that you have managed under UCS. Finally we added a widget to let you see native errors and failures from UCS via the API. If you are using Cisco UCS in a Hyper-converged (HCI) configuration or hosting your critical virtualization resources in UCS then the new monitoring we have added is going to be a big win for you!

 

Get started by adding your Cicso UCS Manager node. In the Add a node wizard, click  'Poll for UCS' and enter your credentials.

 

 

Once you are successfully polling the UCS Manager some new widgets will become available:

 

Overview and UCS Errors and Failures

 

Chassis Overview

 

 

Blade hardware health

 

 

New layer added in AppStack!

AppStack let's you see the relationship between your Cisco UCS resources and the VMs and Applications running on them.

See end to end status from containers and applications all the way to the storage at the foundation of your UCS stack!

 

Out of the box alerts and reports:

 

Hardware Alerts:

 

 

Cisco UCS Entity Report

 

 

That wraps up our quick tour of this great new feature in SAM 6.8... As always, if you like what you see or have a question or a comment please feel free to contribute below.

You can also submit a feature request Server & Application Monitor Feature Requests

If you are curious about what we are planning for future releases jump over to the public road map What We're Working On For Server & Application Monitor (Updated November, 2019)

 

Here are some additional useful links related to SAM:

SAM 6.8 is now available - Following up to our previously released AppInsight for SQL, Exchange and IIS... The latest installment of AppInsight is here and it wants to make your life easier when it comes to monitoring Active Directory. In addition to performance counters and event logs, detailed information about Replication, FSMO Roles and Sites is provided out-of-the-box

 

To get started there are a couple ways to get AppInsight for Active Directory applied to your domain controller nodes:

You can either use "List Resources" on a node you know to be a domain controller or you can run a network sonar discovery and we will find your DCs for you!.

 

 

 

Perf-counters and events are still here but we took the time to add some new ones and also improve the grouping presentation. User and Computer Events, System Events, Replication Events, Policy Events and Logon Events are all neatly grouped together to make it easy to find what you are looking for.

 

Click to EnlargeClick to EnlargeClick to Enlarge

 

Replication: If replication isn't working, your Active Directory isn't working. Keep an eye on replication and get alerted if anything goes wrong. In addition to status we are representing direction and site location. You can also expand any given DC to see more detail about it's configuration.

 

Click to EnlargeClick to Enlarge

 

FSMO Roles at a glance: When something is wrong with a particular DC it can be helpful to know what roles it holds. Hover over the pill to expand the role description. Filters are also available at the top of the resource to allow you to focus on servers of a particular type of role.

 

 

Site Details: This widget provides a detailed overview of your sites including a view into related Links and Subnets. The widget also allows for quick searching to zero in on a specific item.

 

 

Alerts objects specific to AppInsight for AD

 

 

So that wraps up our quick tour of this great new feature in SAM 6.8... Don't forget to check out part 2 of what's new in SAM 6.8 SAM 6.8 WHAT'S NEW PART 2 - Enhanced Support for Cisco UCS Monitoring

 

As always, if you like what you see or have a question or a comment please feel free to contribute below.

You can also submit a feature request Server & Application Monitor Feature Requests

 

If you are curious about what we are planning for future releases jump over to the public road map What We're Working On For Server & Application Monitor (Updated November, 2019)

 

Here are some additional useful links related to SAM:

 

Thanks for stopping by!

Filter Blog

By date: By tag:

SolarWinds uses cookies on its websites to make your online experience easier and better. By using our website, you consent to our use of cookies. For more information on cookies, see our cookie policy.