1 2 3 4 Previous Next

Product Blog

47 Posts authored by: aLTeReGo

We've just wrapped up the Server & Application Monitor 6.1.1 Service Release, meant to address any major outstanding issues identified in SAM 6.1 since its official release. Now it's time once again to turn our attention to the future of Server & Application Monitor. With that said, below is a list of items the team is currently working on.

 

  • AppInsight for IIS
  • Optional Agent for Monitoring Windows Applications and Servers that addresses the following needs:
    • Allows for polling host and applications behind firewall NAT or proxies
    • Polling node and applications across multiple discrete networks that have overlapping IP address space
    • Allow for reliable and secure encrypted polling over a single port
    • Support low bandwidth, high latency connections
    • Polling nodes across domains where no domain trusts have been established
  • Application stack integration and visualization (E.g. visual mapping through the entire application stack to help identify root cause of performance and availability issues)
  • Performance & Scalability Improvements
  • Web Based Advanced Alert Manager
  • Packet-level Traffic Analysis and Classification
  • Web Based Syslog and Trap Management
  • Unmanage Behavior Improvements
  • Disk Volume Capacity Forecasting
  • Mandatory Custom Properties
  • Support for SQL Server 2014

 

PLEASE NOTE:  We are working on these items based on this priority order, but this is NOT a commitment that all of these enhancements will make the next release.  We are working on a number of other smaller features in parallel.   If you have comments or questions on any of these items (e.g. how would it work?) or would like to be included in a preview demo, please let us know!

Server & Application Monitor 6.1.1 Release Candidate is now available for all SAM customers under active maintenance through your Customer Portal. Release Candidates are fully supported and upgradable to the final release when available. If you are experiencing any of the issues outlined below, we recommend downloaded the SAM 6.1.1 Service Release and upgrade now.

 

Fixed Issues and Additions in this Release

This release of SAM includes the following fixes and additions:

Version 6.1.1

Fixed Issues:

  • An issue where certain Asset Inventory resources were displaying volumes that were not appearing in the web console has been fixed. a
  • The Exchange database file size default limit was increased in size from 50 GB to 1 TB, and can now be configured manually in the following file: SolarWinds.APM.BlackBox.Exchg.Collector.dll.config.  The 50 GB limit made it difficult to determine proper thresholds. b
  • An issue where AppInsight for Exchange incorrectly calculated the limit for any mailbox database on Exchange 2010 Standard Edition at 50GB has been resolved. c
  • An issue where a PowerShell scripts are unable to run because user accounts and the Exchange server are from different domains has been resolved. d
  • An issue where AppInsight for Exchange threw an error citing, Exchange Version Mismatch, has been resolved. e
  • A conflict between:
    • Windows Scheduled Task Manager and templates of the same name causing the former not to function properly, has been fixed. f
    • AppInsight applications and templates of the same name causing the former not to function properly, has been fixed. m
  • An issue where certain Asset Inventory resources were showing duplicate records due to external updates has been fixed. g
  • AppInsight for Exchange now has the ability to poll all mailboxes within a domain forest. Prior to this fix, AppInsight for Exchange could only poll root mailboxes.  
  • A memory leak has been fixed. This leak occurred during the attempted initialization of certain performance counters resulting in performance counters endlessly attempting to start. i
  • An issue where some physical servers being monitored were falsely reporting a child Warning status has been fixed. j 
  • AppInsight for Exchange now supports Windows 2008 (non-R2). k 
  • A timeout setting has been increased for Linux script execution which now prevents a script execution error. l
  •   A Linux script execution error has been fixed. n, p, q
  • After  upgrading to SAM 6.1, an issue where the variables, ${Threshold-Statistic-Critical} and ${Threshold-Statistic-Warning} no longer populated, has been fixed. o
  • An issue where certain Asset Inventory resources were showing duplicate records due to external updates has been fixed. g
  • AppInsight for Exchange now has the ability to poll all mailboxes within a domain forest. Prior to this fix, AppInsight for Exchange could only poll root mailboxes.
  • A memory leak has been fixed. This leak occurred during the attempted initialization of certain performance counters resulting in performance counters endlessly attempting to start. i
  • An issue where some physical servers being monitored were falsely reporting a child Warning status has been fixed. j
  • AppInsight for Exchange now supports Windows 2008 (non-R2). k
  • A timeout setting has been increased for Linux script execution which now prevents a script execution error. l

 

Table of Fixed SAM Issues

The following table provides the internal Development ID numbers and external support ID numbers for fixed SAM issues as well as new feature requests in this release. Search in the support ID number column for the number assigned to your support case.

 

SuperscriptSupport ID numberDevelopment ID number
a591714321074
b586749318056
c603164328359
d591412320755
e593737321838
f, m594985, 602168323456, 328549
g576182312166
h581448314906
i490700255402
j577567312188
k596422324735
l599218326004
n, p, q594888, 583631, 601187323753, 325219, 327131
o598659325679

We here at SolarWinds are continuously looking to improve our products in both functionality and user experience. Failover Engine (FoE) as you can imagine, doesn't get a tremendous amount of feedback from the Thwack community. This is because FoE is akin to the spare tire sitting in the trunk of your car. You hardly ever think about it until you need it. With that in mind, I've compiled a few thought provoking questions that I hope will engage those of you in the community to think about how you use FoE. This should help to give us a better understanding how and where we can improve FoE in the future,

 

What has your experience been like Installing/Upgrading FoE?

In an Failover Engine LAN Configuration How do you Maintain the Standby Host?

Are your Failover Engine member servers joined to the Domain?

How do you prefer to manage administrative tasks in Failover Engine?

What is the primary reason your Orion server is down?

How much redundancy is enough for your environment?

At first glance, Server & Application Monitor (SAM) 6.1 might sound like it's a "minor" release. However, with the mountain of new features we've managed to cram in, 6.1 is anything but minor. In previous blog posts I discussed Windows Scheduled Tasks and JSON/XML monitoring, as well as Sustained Threshold Conditions that can be used to squelch nuisance alerts. Despite more than half a dozen additional new features in this release I haven't even talked about yet, such as new SOAP Monitor, Drag & Drop resources (that's right, I said drag and drop), and a new Web-Based Report Scheduler, much of the buzz surrounding SAM 6.1 has centered around AppInsight for Exchange. This is likely due to the success of AppInsight for SQL in the SAM 6.0 release, coupled by the tease that was my previous blog posting entitled "Introducing AppInsight for Exchange - Server & Application Monitor 6.1 Beta 2 Sneak Peek". In that post I gave readers a very early glimpse into AppInsight for Exchange, that barely scraped the surface of what this new application monitoring capability provides. So today I'll attempt to satisfy some of that curiosity by showcasing some of the other functionality included in AppInsight for Exchange.

 

I first cut my teeth back in the day of Exchange 5.5. Since that time I've seen tremendous improvements in Exchange scalability, reliability, and performance. As a consequence of these improvements however, Exchange has become significantly more complex to manage, monitor, and maintain. Simply isolating a performance bottleneck in your Exchange environment can be akin to playing a bad game of "Where's Waldo".

 

AppInsight for Exchange ends the madness by centrally consolidating all information about each mailbox database and its copies across all mailbox servers in the Database Availability Group, into a single Mailbox Database Details view. It is within this Mailbox Database Details view where you will find all relevant information pertaining to that specific individual mailbox database on the server, as well as all other servers where a copy of that database resides.

 

Information including last full and incremental backup, number of mailboxes in the database, average mailbox size, and default storage quotas applied at the mailbox database are all easily at hand.

Database Details.png
Database size and disk io.png

The Database Details view also contains multi-server performance, health, and availability information that should make troubleshooting common mailbox database issues a breeze.

 

For example, in the screenshot to the left you can easily identify where the mailbox database resides on each servers file system, the size of that mailbox database, and how it relates to the amount of free space on each servers volume. You can also identify disk I/O performance issues across servers by seeing the Disk Queue Length, Latency, and total IOPS for the volumes on each server where a copy of that database resides.

 

Similar resources also appear in this view for Transaction Logs, showing additional detail such as the total number of transaction log files, as well as their cumilitive total size on disk.

 

All this information allows you to easily spot problems before they start. There's nothing worse, or more preventable than a database dismounting because the volume it's located on has run out of space. With AppInsight for Exchange, now you can be proactively alerted and take corrective action before it impacts your users.

Now let's say your mailbox database was running out of space. Where do you go, and what do you do now? You could move the mailbox database to a different volume that has more space. If you have more unallocated storage you could even extend the volume. Both of those options require heavy lifting, and likely some downtime.

 

What if you could easily identify the offending user mailboxes that are taking up a large percentage of space in the database? You could then either hunt those users down and ask them them to clean up their mailbox, or move them to another mailbox database that has more available space.

 

The "Users By Mailbox Quota Used" and "Users By Mailbox Size" resources allow you to view each and every user mailbox, its total size, amount of space all attachments in the mailbox are consuming, total number of attachments, and percentage of quota used. This information is available in each mailbox database view, as well as across the entire Exchange environment.

 

You can even spot dormant mailboxes easily within the same resource by viewing the "Last Accessed" date.

Users By Mailbox Quota Used.png

From this resource you can drill down even further into the User Mailbox Details view. Here you can see the quota limits applied to the individual users' mailbox, the Active Directory Organizational Unit where this user resides, and can even click their Primary SMTP Address to notify them that some mailbox cleanup is required.

 

Before you do that though, you might want to get a better understanding of how this user is using their mailbox. Perhaps they recently received several very large email attachments that they could move off onto the file server. Maybe this user regularly receives a large volume of incoming email. This could be normal given their job function, or indicative of ineffective SPAM filtering.

 

With AppInsight for Exchange you can easily visualize each users historical mailbox usage, identify trends such as the growth of a user's mailbox over time, and the total size of all attachments within the users mailbox over the same period. You can also gain insight into the volume of mail sent and received by the user each day, both internally and externally.


This information allows you to make informed decisions before extending users mailbox quotas or adding additional storage to the Exchange server. This information can also be included in alerts that give helpdesk staff a heads up as users approach their quota limit.

Mailbox Details.png
Total Mailox and Attachments Sizes.pngReceived Mail.png

What if the problem you're facing was the other direction? Instead of a massive influx email or attachments driving the users mailbox size, it was malicious activity the end user wasn't even aware of? The Users By Messages Sent resource helps identify mailbox abuse caused by potential trojans, botnets, or otherwise unscrupulous activity. Should your users mailbox be taken over by such mass mailing marauders, AppInsight for Exchange makes identifying this unusual traffic a trivial affair.

 

AppInsight for Exchange also allows you to report on the mobile devices being used in your environment, the operating system version running on those devices, as well as the last time any device was used to connect to Exchange via ActiveSync.

 

This information is available on any individuals Mailbox User Details view. It is also available as an out of the box report that lists all mobile devices in use in your organization and their respective owner.

 

Until now, SAM 6.1 has been available only to a select number of beta participants, but that's no longer the case. As of today, all current SAM customers under active maintenance can download and install the official SAM 6.1 Release Candidate simply by signing up here. Upgrading of your existing production Server & Application Monitor installation is also fully supported. So give AppInsight for Exchange, or any of the over a dozen other improvements in this release a go, and tell us what you think!

Users By Messages Sent.png

Synced Devices.png

 

SAM 6.1 RC button.png

In my previous blog post I introduced you to several cool new features we've been working on for the next release of SolarWinds Server & Application Monitor (SAM). From sustained status conditions that are sure to help squelch nuance alerts, Windows Scheduled Task monitoring, and JSON, SAM 6.1 was already shaping up to be a pretty awesome release.

 

It's now time to take the wraps off SAM 6.1 Beta 2, and just in time for Christmas. If you've been wondering what to get yourself for the holidays, or fear that after all the presents have been unwrapped you'll have amassed a years supply of tube socks, silk ties, and ugly sweaters, you needn't worry. We have just the thing that's sure to put a smile on your face this holiday season.

Sign-up to Download SAM 6.1 Beta 2.png

 

AppInsight for Exchange

 

The overwhelmingly positive feedback we received from the community regarding AppInsight for SQL since its debut in SolarWinds Server & Application Monitor 6.0 has been so phenomenal that we just couldn't stop there. The next obvious choice to get the full AppInsight treatment had to be Microsoft's Exchange. Both SQL and Exchange are complex business applications that are at the very center of most organizations' IT universe. In almost all cases, end users directly (or indirectly) interact with these applications on a continual basis. Be it for basic internal and external communications, data entry, billing, ordering, etc. SQL and Exchange tend to touch so many individuals both inside and outside the organization, that it's imperative that their health, performance, and availability is continuously monitored.

 

If you're not at all familiar with AppInsight for SQL, or the AppInsight concept, below is an excerpt from one of my previous blog posts in which I attempt to explain it.

 

AppInsight provides a whole new level of application monitoring detail that was previously very difficult, if not impossible to achieve using Application Templates. AppInsight is not a direct replacement for Applications Templates but rather an entirely new monitoring concept within SAM. Application Templates remain the primary method for quickly monitoring virtually any commercial, open source, or home grown application imaginable. In contrast, AppInsight is more akin to an entirely new product deeply embedded within SAM; designed from the top down to solve common, yet complex problems for a specific application, rather than merely a new feature.

 

Discover


As with AppInsight for SQL, monitoring your Exchange Mailbox Servers with SAM is a fairly simple, straightforward affair. For existing nodes currently managed via WMI simply click List Resources from the Node Details view and select Microsoft Exchange Server directly beneath AppInsight Applications. The same is also true for any new Exchange servers added individually to SAM and managed via WMI using the Add Node Wizard.

 

Exchange Servers can be added both individually using the methods above, or en masse using the Network Sonar Discovery Wizard. Both one-time and scheduled re-occurring discovery of Exchange servers in the environment using Network Sonar are fully supported. Either method will allow you to begin monitoring your entire Exchange environment in record time.

 

Please note that AppInsight for Exchange has been designed exclusively for Microsoft® Exchange 2010 and Exchange 2013 Mailbox Role servers. This option will not appear for nodes running previous versions of Exchange or servers running other Exchange roles, such as the Client Access role.

AppInsight for Exchange List Resources.png
Network Sonar AppInsight for Exchange.png
Network Sonar Scheduled Discovery Results - AppInisght for Exchange.png

Configure

 

AppInsight for Exchange uses PowerShell to collect virtually all information from the Exchange server. As such, PowerShell 2.0 must be installed on the local Orion server or Additional Poller that the node is assigned to. PowerShell 2.0 must also be installed on the Exchange Server being monitored. Windows 2008 R2 includes PowerShell 2.0 by default. If you're already running Orion on Windows 2008 R2 or greater and plan to monitor Exchange running on a Windows 2008 R2 server, you needn't worry about the PowerShell 2.0 requirement. Microsoft has taken care of that for you.

 

Beyond simply having PowerShell installed, Windows Remote Management (WinRM) must also be configured. Both locally on the Orion server, and on the remotely monitored Exchange Server. Fear not though; we've made this process incredibly simple and completely painless.

 

After discovering the Exchange mailbox servers running in your environment and choosing to monitor them, you may find them listed in the All Applications tree resource on the SAM Summary view in an "Unknown" state. This is likely due to WinRM having not been configured on either the local Orion server or the remotely monitored Exchange mailbox server. Clicking on the AppInsight for Exchange application that is in an "Unknown" state from the All Applications resource launches the AppInsight for Exchange configuration wizard.

Zero Config Basic.png

The AppInsight for Exchange configuration wizard will prompt you for credentials to configure and monitor the remote host. By default, credentials used to manage the node via WMI are selected. However, under some circumstances, such as using the local administrator account to manage the node, these permissions may not be adequate for monitoring Exchange. If that is the case, you can select from the list of credentials available from your Credential Library, or enter new credentials for AppInsight for Exchange to use. The account used for AppInsight for Exchange should have Exchange Admin Role permissions.

 

Once you've selected existing, or defined new credentials for AppInsight for Exchange to use, simply click "Configure Server". The configuration wizard will do the rest. It should only take a minute or two and you'll be up and monitoring your Exchange mailbox server. Easy peasy, and no agent required.

 

So what exactly is this magic "Configure Server" button doing anyway? Well nothing that couldn't be done manually with a bit of effort. Quite simply the "Configure Server" button pushes a self signed certificate to the Exchange Server and configures WinRM to function in a secure encrypted fashion between the two hosts. Steps for manually configuring your Exchange Server, as well as creating a least privilege user account for monitoring your Exchange mailbox servers using AppInsight for Exchange will be available in the SAM Administrators Guide once SAM 6.1 is officially released.

 

Monitor

 

An Exchange environment is typically comprised of multiple Mailbox Databases. Much like SQL databases, each mailbox database has its own independent status that tells the administrator how that database is currently being used (or not used in the case of databases that are "Dismounted"), as well as the health of that database. In the Mailbox Database Status resource, located on the Application Details view we see all of the databases running on this Exchange server. All but one is in a "mounted" state, but the SAMDB03 database does not appear to be running on its "preferred" server, as designated by its Activation Preference. For smaller environments where two or more Exchange Servers running in a DAG (Database Availability Group) are sitting next to each other, this might not be an issue. For larger distributed environments, losing track of where your mailbox databases are running can lead to end users complaining about email performance problems or worse. For example, If your office was headquartered in Boston, but have a DR facility in your satellite office in Shanghai, the last thing you want is all the traffic from the users in the Boston office traversing the WAN to access their mailboxes from the Shanghai server.

Mailbox Database Status.png
As obvious as that sounds, this can and does occur for simple, sometimes seemingly stupid reasons. For example, applying Windows Updates to the Exchange server in Boston, but failing to move those mailbox databases back over from Shanghai after the reboot. It's important to know which server in the DAG your mailbox databases are mounted, and be notified when they're not mounted on the appropriate server.
Replication Status Checks.png

The Replication Status Checks resource, also on the Application Details view, checks all aspects of replication and replay status to provide a complete overview of the mailbox server in the Database Availability Group (DAG). This allows you to proactively monitor continuous replication, the availability of the Active Manager, and the health and status of the underlying cluster service, quorum and network components, to name a few.

 

In the event of a replication status check failure you will be notified both visually through the UI, as well as through your normal alerting mechanisms. Clicking on the "More" link for any failed Replication Status Check displays the full details of that failure.

Each mailbox database on your Exchange server is a time bomb ticking down until ultimately it runs out of space. Be it from mailbox database limits imposed by the Standard Edition of Exchange, NTFS file size limitations, or simply running out of free space on the volume where the mailbox database is stored, it's only a matter of time before the mailbox database hits the wall. When that time comes, it's certain to negatively impact any and all users whose mailboxes reside on that mailbox server.

 

To stay ahead of the game it's imperative that you have a good understanding of how your mailbox database size relates to these limitations. This will allow you to be proactive in your approach to managing your mailbox databases, and the individual mailboxes that reside within.

Mailbox Database Size and Space Use.png

Alerting

 

This is precisely the kind of information that the Mailbox  Database Size and Space Use resource provides. Not only does this resource list all mailbox databases managed by the mailbox server, their current size on disk, and the amount of white space remaining within the database, but the linear gauge also shows the percentage of the mailbox databases usage as it relates to such things as remaining free space on the volume, and file size limits imposed by Exchange edition/version or the NTFS file system. This information is then available for reporting, trending, and of course, out of the box alerts so you can be notified proactively and avoid such crisis altogether.

 

This was just a tiny glimpse into a few of the powerful new capabilities included with AppInsight for Exchange. If you'd like to try it out for yourself, don't hesitate! Sign-up here to download SAM 6.1 Beta 2 today. You need only own an existing license of SolarWinds Server & Application Monitor, and be under active maintenance to participate.

 

Your feedback (both positive or negative) is what we thrive upon. It serves either as confirmation that we're on the right track, or that adjustments and improvements need to be made. Either way, it's what helps us to build great software.

AppInsight for Exchange Out of the Box Alerts.png

So here we go again! Time to kick off the next release of SolarWinds Server & Application Monitor with the first official beta, and grab a quick first glimpse into several of the cool new features we've got lined up for SAM 6.1.

 

Not to be confused with SAM 6.0.1, the Service Release containing many important bug fixes for SAM 6.0 and available for download now through the Customer Portal, SAM 6.1 is the next major "feature" installment in the series. If you'd like to participate in the SAM 6.1 beta you can do so by signing up here. Feedback is crucial during the beta phase of development because there's still plenty of time to make important tweaks and adjustments that can make all the difference in the final release. If you've never participated in a SolarWinds beta before, now is a great time to start. Not only do you get to play with all the great new features first, but it's also an excellent way to help shape the future of the product.

 

Windows Scheduled Tasks

 

At one time or another, every systems administrator has had to rely (albeit sometimes begrudgingly) on Windows native Task Scheduler to automate some routine process. Be it automating backups, disk defragmentation, antivirus scans, etc., the Windows Task Scheduler has undoubtedly played an important role in ensuring your infrastructure is properly maintained. However, even as the Windows Task Scheduler has improved over the years, real-time visibility into the success or failure of those tasks across the enterprise has remained, for the most part, an enigma.

 

In SolarWinds never ending pursuit to provide greater levels of visibility into the critical componentry that make up your IT infrastructure, we sought to resolve this visibility gap by introducing the Windows Scheduled Task Monitor as part of the SAM 6.1 beta release.

 

Elegant in it's simplicity, SAM's Windows Scheduled Task Monitor for the first time provides you with at-a-glance access to the state and status of the scheduled tasks configured on your Windows hosts. In addition to simply seeing what tasks have been configured on the host, their current state, and their last run result, SAM 6.1 includes a new pre-configured out-of-the-box alert which will notify you of any task execution failures that occur. You will also find new web based reports that allow you to view all scheduled tasks configured across all servers in your environment, as well as a dedicated Task Failure Report you can view or have emailed to you on a regular basis.

Windows Scheduled Tasks.png

 

When monitored, you will find the Windows Schedule Task resource pictured above on the Node Details view of the monitored server. This is because Windows Scheduled Tasks are not applications in the conventional sense. As such, they are treated somewhat special in SAM and given a prominent resource of its own amongst other host specific information on the Node Details view.

WIndows Scheduled Task Add Node.png

Several options are available to enable SAM's new Windows Scheduled Task monitor. When adding a new, or listing resources on an existing WMI managed node, you will be provided an option to select Windows Scheduled Tasks. The same as you would for volumes or interfaces.

 

If enabling this feature one node at a time isn't your speed, you also have the option of leveraging the Network Sonar Discovery Wizard. The Network Sonar Discovery Wizard allows you to quickly and easily enable the Windows Scheduled Task monitor en masse across all Windows hosts in your environment, or surgically enable this feature only on a select group of nodes.

 

Both one-time discovery, and scheduled reoccurring discovery options are available to enable the Windows Scheduled Task monitor. If using the scheduled discovery option you will have granular level control over which hosts the Windows Scheduled Task Monitor is enabled, as seen in the screenshots below. Hint: If the image is too small, click on it to zoom in and see the full size image.

 

The new Windows Scheduled Task Monitor in SAM 6.1 supports monitoring tasks configured on Windows 2003, 2003R2, 2008, 2008R2, 2012, and 2012R2.

Windows Scheduled Tasks - Scheduled Discovery.pngWIndows Scheduled Task Network Sonar Discovery.png

JSON

 

Web Services APIs such as JSON are the glue that bind modern applications together, usually across different servers, allowing for the exchange of information between them. As end users become reliant upon applications built on these web services, it becomes increasingly more important to monitor those applications to ensure they're functioning as expected. The simplest, and most obvious method for monitoring those applications is to query the back-end server directly, using the same web service API method that the front-end web application would use. From the server's response we can determine the web services availability (up/down), latency (response time), as well as validate the content returned as a result of that query.JSON.png

From within the HTTP/HTTPS Component Monitor settings, you will find three new options (Host Request, Content Type, and Request Body) that allow for the monitoring of restful web service API's, such as JSON and XML. Three new methods (Put, Post, and Delete) are also provided, in addition to the existing "GET" method that has historically been the default and only method available for the HTTP/HTTPS User Experience Monitors prior to SAM 6.1.

 

Sustained Thresholds

 

Last, but certainly not least, 6.1 includes additional improvements to how thresholds are handled in SAM. While tremendous strides were made to how thresholds are calculated in the SAM 6.0 release with the introduction of the Threshold Baseline Calculator, that feature served to provide meaningful context to already collected data. In other words, to answer the proverbial question "What's normal for my environment?" and then suggest recommended warning and critical thresholds based on that information; however, as anyone who's been monitoring IT infrastructure for a while will tell you just because a threshold was crossed once, doesn't mean it's a significant issue that requires immediate attention.  After all, who enjoys being woken from their slumber at 3am to a nuisance alarm telling you that the % Processor time on one of the servers spiked momentarily. If the alert requires no action on your behalf, then more than likely it wasn't worth you waking up for. Alert notifications should be about providing actionable information that requires some level of user intervention to resolve. While some metrics, such as the amount of free space remaining in your SQL database might only get worse over time, thus requiring immediate attention when it dips below a reasonable limit, other metrics can vary wildly from one poll to the next. This is where sampling can play an important role in reducing, or even eliminating the number of nuisance alerts that flood your inbox on a regular basis.

 

In SAM 6.1 you will find new options for defining sample criteria for both warning and critical thresholds associated with each monitored metric of an application. By default, both warning and critical thresholds are evaluated after a single successful poll. This is the exact same behavior as all versions of SAM prior to 6.1. In addition to the single poll evaluation, you will find options for defining criteria for multiple consecutive polls, as well as a method for defining the number of samples that must exceed the threshold for a configured sample size before the condition is met and the status of that component monitor is changed.

 

Sustained Thresholds.png

 

Sustained conditions in SAM 6.1 can be defined independently for both warning and critical thresholds to provide maximum flexibility. Both "X Consecutive Polls" and "X out of Y Polls use a sliding window approach to evaluating thresholds. After each poll, the conditions defined for the threshold are evaluated based on the bounds of the sample size. Put simply, that means that after each poll a new sample is collected and added to the evaluation, while the oldest sample is removed from evaluation. Below, I provide two examples. The first example on the left demonstrates the "X consecutive polls" method. In the left column I show the numerical value collected from the poll (the sample). In the right column I show the status of the component as defined by the sustained condition. The "Sample Size" in this example is "3", meaning that three consecutive polls/samples must exceed the threshold of "80" before the status should change to "Warning".

 

Warning = Greater Than 80 for 3 Consecutive PollsWarning = Greater Than 80 for 3 out of 5 Polls
Polled ValueStatus
65UP/Green
77UP/Green
88UP/Green
85UP/Green
89Warning
83Warning
46UP/Green
81UP/Green
22UP/Green
Polled ValueStatus
65UP/Green
82UP/Green
34UP/Green
95UP/Green
88Warning
90Warning
35Warning
25Warning
15UP/Green

 

The second example demonstrates the "X out of Y polls" method. While the "Sample Size" for evaluation in this example is "5" polls, any three of those 5 polled samples must exceed "80" before the status of this component would change to "Warning". Using the same sliding window approach as the first example, with each successive poll a new sample is collected, while the 6th sample is dropped from evaluation.

 

While somewhat similar functionality has existed within the Advanced Alert Manager for some time now, aiding in reducing the number of nuisance alarms, each individual component monitor that has unique threshold criteria has required its own separate alert definition. Not only is this a tedious and time consuming process to initially setup and configure, but it also necessitates the additional overhead of managing and maintaining what can be an unruly number of alert definitions.

 

Sign-up Now

 

We'd love to get your feedback on these new features. So tell us what you think in the comments section below, or better yet, sign-up here to download the latest SAM 6.1 beta and try them out for yourself!

 

Please note that you must currently own a copy of SolarWinds Server & Application Monitor that is under active maintenance to participate in the SAM 6.1 beta.

To receive updates on the SAM roadmap, JOIN thwack and BOOKMARK this page.

 

The mad scientists of the SAM team have now fully recovered from the marathon release that was SAM 6.0; and as such it is time once again we turn our attention to the future of SAM. Below is a listing of several of the new features they're brewing up in the laboratory.

 

  • AppInsight for Exchange
  • JSON Monitoring
  • SOAP Monitoring
  • Independent Thresholds for Node CPU Utilization, Memory Usage, Response Time, and Packet Loss
  • Windows Scheduled Task Monitoring
  • User Definable Sustained Status Condition Alerting
  • Optional Agent for Monitoring Windows Applications and Servers
  • Application stacked integration and visualization (i.e. visual mapping through your entire application stack to help you answer the question, "Why is my app slow?")

 

PLEASE NOTE:  We are working on these items based on this priority order, but this is NOT a commitment that all of these enhancements will make the next release.  We are working on a number of other smaller features in parallel.   If you have comments or questions on any of these items (e.g. how would it work?) or would like to be included in a preview demo, please let us know!

Please join us for a monthly product update from the SolarWinds Product Management team. The team will cover what’s new, what’s coming, what we’re thinking about for future releases, tips & tricks, and cross-product capabilities.


Each session will be hosted by one or more product managers, and will be very collaborative. We want to hear your thoughts, questions, and requests.

On August 21, we’ll do a sneak peek into our progress with NPM Vnext, and we’ll discuss new developments with our Systems portfolio. We’ll set aside plenty of time for your thoughts, opinions, and questions.

 

IF YOU WOULD LIKE TO ATTEND THIS WEBCAST, PLEASE REGISTER HERE!

I believe Bronx summed it up perfectly when referring to this release as SAM 6.OMG; because that's precisely the kind of reaction and enthusiasm we receive anytime we have an opportunity to show it off. Those lucky few who've had an opportunity to play with some of the early betas first hand have been champing at the bit in eager anticipation for the opportunity to upgrade their production installations of SAM to version 6.0. Well that time has finally arrived.So without further ado I would like to extend a personal invitation to all Server & Application Monitor customers under active maintenance to sign-up here to download the SAM 6.0 Release Candidate.

 

Please note that Release Candidates are provided as early availability access to the latest release. They are treated no differently than GA releases, in that Release Candidates are fully supported in your production environment, and the SolarWinds technical support team is available to aid and assist with any problems you might encounter.

 

If you haven't been following the SAM beta blog postings here in the Product Blog, I'll give you a brief rundown of some of the exciting new features included in the SAM 6.0 Release Candidate.

 

Real-Time Event Log Viewer

 


The Real-Time Event Log Viewer, as the name suggests, is a web based version of Windows own Event Viewer. This tool allows you to troubleshoot and diagnose application and server related issues occurring on the remotely monitored host in real-time, without the need to remotely connect to the server, log-in, and launch the Windows Event Viewer. Once you've isolated and identified the cause of the issue, you can then easily create a Windows Event Log Monitor from within the Real-Time Event Log Viewer so you can be alerted if this issue occurs again in the future. Simply select the event(s) you'd like SAM to continuously monitor, click Start Monitoring, and the wizard begins. It's just that easy.

 


Asset Inventory

 


In this 6.0 release we extend SAMs hardware health monitoring, introduced in v5.0, to include a much broader array of additional server asset inventory information. This allows SAM to collect all relevant information necessary for asset depreciation, insurance, and support. However, this feature also provides a wealth of additional information you'll likely find useful in your everyday tasks. Some such examples include reporting on software installed for license tracking, or what operating system updates have been applied to the server; by whom, and when they were applied.

 

Information from SAMs Asset Inventory collection can even be programmatically synchronized with other CMDBs using the Orion SDK. If you don't already have a dedicated CMDB, the inverse is equally true. SAM 6.0 allows you to either manually or programmatically populate and report upon non-pollable information in the Orion database, such as the original purchase price, purchase date, rack location, etc. Need to track and report upon other asset information? No problem. Add additional fields to track whatever information you need, such as "in service date",  who the server was originally purchased from, or anything else for that matter.

Server Warranty Expiration.png

 

Custom Asset Information.png

In my previous blog post regarding server asset inventory collection, cwestwater suggested that it would be helpful if SAM was able to automatically track and maintain the servers warranty status and warranty expiration dates. And we agreed. So in addition to gathering all kinds of useful asset inventory information about the physical and virtual machines SAM 6.0 is monitoring, SAM will also monitor the warranty status of your physical servers. Provided your Orion server has access to the internet, this is all done automagically by periodically checking the status of each servers warranty against Dell, HP, and IBM's online warranty validation servers. This allows SAM to alert you when your servers warranty is due to expire. You'll also find a new Server Warranty Summary resource (pictured above) that provides at-a-glance warranty status information for servers where the warranty has expired, is due to expire soon, and those next closest to expire.

AppInsight forSQL


Much of the buzz surrounding the SAM 6.0 release is associated with a radical new concept we've dubbed "AppInsight". Beginning with Microsoft SQL Server, AppInsight's focus is to provide unparalleled monitoring depth and visibility around a given application, while simplifying discovery and reducing overhead associated with continuous monitoring.

 

It is our belief that SAM can provide a truly multi-vendor "best of breed" application and server monitoring experience, previously only possible by deploying numerous dedicated point products. Unlike these "point products" however, AppInsight is a tightly integrated solution built into SAM to provide a single pane of glass view into the health and status of all your servers, and the applications that run on them.

 

For example, AppInsight for SQL provides visibility into who's connected to the SQL Server, how long they've been connected, and from where (host name or IP address) they're connected. Need to know who's actively using the SQL server, not just who's connected, before taking it out of service for maintenance? No problem. Within that very same resource you will discover exactly how long each of user has been idle.

 

AppInsight for SQL gives a unique view into your database's storage usage, allowing users to easily understand the size of each database file, the percentage of white space in the file, as well as calculating in the free space remaining on the volume if auto-grow is enabled. This allows AppInsight to alert you before your database runs out of space, regardless if it's a fixed database size running out of white space, a database file that's configured to auto-grow to a configured limit, or even if the database is configured to auto-grow until it runs out of space on the drive. No matter how your database's are configured, AppInsight for SQL has you covered.

 

Whether you're new to SQL, or a seasoned veteran DBA, the SQL error log is one of the first places you're likely to turn your attention to identify common issues with Microsoft SQL. The SQL Error log contains important troubleshooting information such as, client connectivity issues, backup failures, end of recovery after a restart, DBCC Events, reconfiguration of server or database options, non-yielding schedulers, stack dumps, autogrowth failures, delayed IO events, etc. With AppInsight for SQL this information is right at your fingertips, located conveniently on the Application Details view of each monitored SQL instance.

Top 10 Databases By Active User Connections.png
File By Size.pngSQL Errors.png

Speaking of backups, and other regularly scheduled reoccurring tasks like database shrinks, AppInsight for SQL also includes SQL Agent Job Monitoring. Never again be caught blindsided because a scheduled SQL Agent Job failed to run properly. Better still, understand when, and for how long, SQL Agent Jobs are running to minimize their impact during business hours. These are just a few of the features included with AppInsight for SQL in SAM 6.0.

 

So how is AppInsight for SQL licensed anyway? What does it cost? Is it another module? An add-on?

 

AppInsight for SQL is built-in to SAM 6.0 and does not require the purchase of any additional modules or add-ons. Despite monitoring well over 100 unique SQL Server instance metrics, and more than two dozen key performance indicators for each database, AppInsight for SQL consumes a mere 50 component monitor licenses per-SQL Server instance. Now that's news everyone can be excited about!

SQL Agent Job Status.png

Processes and Services.png

Threshold Baseline Calculator


Believe it or not, there's still plenty of additional new features included in the SAM 6.0 RC I still haven't covered yet. One of these features is the all new Threshold Baseline Calculator. For a several years now we've pre-populated warning and critical thresholds within the application templates included in SAM. These thresholds have been based upon the individual vendors best practices for the given application. There are however many circumstances where this "one size fits all" approach to thresholding simply isn't appropriate, or accurate. The most typical reason why "one size fits all" thresholds doesn't always "fit all" is due to size and scale of an organization and their application deployment.

 

Let's take the example of Message Queue Length in Microsoft's Exchange for an organization with 10,000 or more employees. It might not be unusual for an organization of this size to see the number of messages queued for submission rise above 100 or more on occasion. However, if a small business of  25 or fewer employees saw the same Message Queue Length, at or around 100 messages queued for submission, it would be likely be an indication of a serious issue with the transport service failing to deliver messages properly. That's because for an organization of 10,000 or more employees, having 100 messages simultaneously sent or received within a given time frame isn't too terribly difficult to fathom. On the other hand, for an organization of only 25 employees, that same feat would be extremely unusual. This is just one example of how thresholds for performance metrics don't always "fit all".


For this reason (and countless more examples just like it) the Threshold Baseline Calculator in SAM 6.0 was born. Located from within the Application Editor, directly next to any threshold field you will find two new options. The first, "Use Latest Baseline Thresholds" calculates and applies recommended warning and critical thresholds for the selected metric based upon the last seven days of statistics collection (the baseline). Some might consider this a "blind faith" approach to thresholding that more closely resembles a random number generator, but fret not. There's a science behind where those numbers came from; and for all that detail and more you need only click the "Latest Baseline Details" link that lies just beneath the "User Latest baseline Thresholds" button.

 

http://thwack.solarwinds.com/servlet/JiveServlet/showImage/2-207334-11847/Threshold+Basline+Calcuator.png

 

Upon clicking the "Latest Baseline Details" link (pictured above) you will see a chart which displays the occurrences of statistic values returned by the selected component. This is a visual representation that allows you to easily digest all of the values collected during the baseline period to gain an understanding of what is considered "normal" for this metric in your environment. You have the option of selecting/deselecting typical business hours (8am-6pm Monday-Friday) as well as nights and weekends, or view the cumulative total for entire time period. Overlaid on the chart are the color coded red and yellow banding that represent how the recommended warning and critical thresholds relate to the baseline data. This provides a quick and easy way of determining the number of times you would have been alerted during the baseline period depending how your thresholds were defined.

 

Latest Baseline Details.pngIf embracing your inner statistician isn't your thing, you can also choose the more familiar "Metric over time" from the top most tabs, which displays the typical historical chart view with visual warning and critical thresholds included.

 

Thresholds are calculated using standard deviation, as represented by the sigma "σ" symbol in the table header above. By default 2 standard deviations are used for warning, and 3 standard deviations are used for critical thresholds above or below the mean/average depending upon which operator is selected (greater than/less than). Clicking on any number in the table provides you the option of setting the selected value as either the warning or critical threshold. This also dynamically updates the visual yellow and red banding representing warning and critical thresholds in the chart above the table. At any time you may click the "Reset to Recommended Thresholds" link to return values selected in the table, and represented in the chart above to those recommended by the Threshold Baseline Calculator. You may also click the "Cancel" button to revert any changes made and return to the Application Editor.


Armed with a better understanding provided by the Threshold Baseline Calculator of what is typical/average for a given metric you can make educated, informed decisions as to how warning and critical thresholds should best be defined for your environment.

 

We think you'll agree that this has shaped up to be an outstanding release of Server & Application Monitor. If you've been one of the many patiently waiting for this moment to come, we'd like to welcome you to sign-up here to download the SAM 6.0 Release Candidate now.

 

Please note that the SAM Release Candidates is available exclusively to those running previous versions of SAM or APM under active maintenance.

It's a rare thing these days to see a product essentially reinvent itself. Most product improvements across the industry tend to be more evolutionary than revolutionary; and with Server & Application Monitor 6.0 beta 3 (sign-up here) we are definitely aiming our sights towards the latter. Since we very first sat down at the drawing board to begin design on SAM 6.0, our mantra has been "question everything". We wanted to provide a tremendous level of additional application monitoring depth, while simultaneously simplifying the entire experience of initial setup and configuration of application monitoring; we also wanted to reduce overhead associated with configuration changes made in the environment. As result of this endeavor AppInsight was born.

 

AppInsight is a new application monitoring technique we are introducing as part of the Server & Application Monitor 6.0 release. Beginning with Microsoft SQL Server, AppInsight provides a whole new level of application monitoring detail that was previously very difficult, if not impossible to achieve using Application Templates. AppInsight is not a direct replacement for Applications Templates but rather an entirely new monitoring concept within SAM. Application Templates remain the primary method for quickly monitoring virtually any commercial, open source, or home grown application imaginable. In contrast, AppInsight is more akin to an entirely new product deeply embedded within SAM; designed from the top down to solve common, yet complex problems for a specific application, rather than merely a new feature.

 

AppInsight for SQL

 

Microsoft's SQL Server is a ubiquitous beast at the very core of many organizations critical business applications, both large and small. Unfortunately, gaining the necessary visibility into various different aspects of the SQL Server instance, or worse yet, the databases themselves, that effect the performance characteristics of the applications that rely on the database server has proven challenging without the necessary tools to provide visibility.

 

With SAM 6.0 we knew we wanted to deliver comprehensive SQL database server monitoring in a simple, intuitive, easy to use manner that would aid anyone, from novice systems administrators to veteran DBAs alike, identify both common and complex SQL issues in their environment.

 

Discovery

 

The AppInsight experience, as with most things,starts at the beginning during discovery. When adding nodes individually all available SQL 2008, 2008R2, and SQL 2012 Server instances found on the host are listed. As you can see in the screenshot to the right, you can select each individual instance you'd like to monitor the same as you would for volumes or interfaces.

 

If adding nodes individually isn't your speed, AppInsight for SQL has also been tightly integrated into the Network Sonar Discovery wizard. This allows you to easily scan an entire subnet, group of subnets, IP address ranges, or a list of individual IP addresses, to quickly discover hosts on your network and any SQL Server instances installed on those hosts. Regardless of which method you choose, monitoring your SQL server instances is as easy as checking the box.

Add Node - AppInsight.png
Network+Sonar+-+AppInsight.png

Once you've enabled AppInsight for SQL monitoring for any SQL Server instance(s), you will see them listed within the same All Applications resource alongside the template based applications that you're monitoring with SAM today.

 

A new icon adorns AppInsight for SQL instances in the All Applications resource, making them easily distinguishable from traditional template based applications.

AppInsight Dashboard

 

When you click on any AppInsight for SQL listed in the All Applications tree, you will be taken to a new, radically overhauled version of the Application Details view that's been designed from the ground up to serve as the dashboard for that Microsoft SQL Server instance.

 

The very first thing you're likely to notice when accessing this new Application Details view are the the newly redesigned chart resources that combines information, such as status and current value normally found within the Components resource, with those metrics historical values charted over time. This allows you to see not only how the application is performing now, but how that performance compares to historical trends without needing to drill deeper into the Component Details view.

All Applications - AppInsight.png

There is such an enormous amount of valuable information on this view that it has been categorized and broken up logically across multiple resources; a theme carried over from the Asset Inventory subview I discussed in my previous SAM 6.0 Beta blog post. This prevents anyone from becoming too overwhelmed with all the information shown, while also making the diagnosis of root cause easier. For example, if you're troubleshooting a potential memory issue on your SQL Server, then all information related to memory is contained within the Memory resource. There are similar resources for information such as latches & locks, connections, cache, paging, buffers, and more.


Memory.png

Database Details

 

AppInsight for SQL doesn't merely gather information about the SQL Server instance itself but also individual database managed by that SQL Server as well. Contained within the Application Details view, you will find the All Databases resource that displays status and basic size information for the databases running on this SQL instance. As databases are created on the SQL server, AppInsight will automatically begin monitoring them. Conversely, when databases are deleted from the SQL server they will disappear from the All Databases resource. Click on any of the databases listed and you'll be taken to the Database Details view which contains all information specific to that database.

All databases.png

Database Index Fragmentation.png

 

The Database Details view helps to identify various database performance issues, such as those associated with poorly performing indexes through the top 10 clustered and non-clustered indexes by fragmentation resources.There's also a wealth of other valuable information contained in this view that will help you to determine which tables consume the most storage space, the number of rows in each table, white space remaining in the database, as well as resources to help isolate disk I/O contention issues for both database and transaction log files.

Database - Tables by Size.png
Expensive Queries


What do you do when "Sally" in accounting complains that the ERP system, that's critical to the business and relies on Microsoft SQL, is "slow", but you've exhausted all avenues trying to isolate the cause of the issue? SAM's AppInsight for SQL may be telling you that the issue is CPU related, but how do you determine the actual cause? This is where having insight into the most expensive queries executing on the SQL server becomes absolutely essential. Within the Application Details view you will find the Top 10 Expensive Queries resource that shows the the most expensive queries running across all databases for that SQL server instance. Additionally, you will find the same Top 10 Expensive Queries resource on the Database Details view that shows the list of most expensive queries running against that specific database. If those resources don't provide an adequate level of detail to help isolate the issue there's also a dedicated Queries subview where you can filter on a specific database, time frame, host, or user. From this view you can quickly identify that the cause of the high CPU utilization on the SQL server, and the ERP systems poor performance, are caused by "Jan" in the Finance department running her quarterly reports. That's actionable information everyone can use.

Most Expensive Queries.png

Download Details

 

There's still a ton more AppInsight for SQL goodness that I haven't yet shown. If you'd like to see it in action for yourself sign-up here to participate in the SAM 6.0 beta. Note: You must already own a license of Server & Application Monitor that's currently under active maintenance to participate in the Beta.

As anyone who's been regularly following the thwack blogosphere might tell you, SolarWinds Server & Application Monitor (SAM) has an impressive track record of packing a ton of really great features into each and every release; and SAM 6.0 is no exception. In fact, some might argue that we're upping the ante with this release to a whole new level. And to any naysayers amongst you I say, you ain't seen nothing yet.

 

No different than in previous releases, SAM's elite army of robot zombie developers have been diligently and tirelessly working on a lengthy list of new features and improvements for this SAM 6.0 release, some of which are now ready for beta testing. If you would like to participate in the SAM 6.0 beta and begin playing with some of the new features, the process is very simple; sign-up here. The only requirement is that you must be an existing Server & Application Monitor product owner under active maintenance.

 

Real-Time Windows Event Log Viewer

 

Regardless of whether you're new to systems management, or you're a war torn veteran with the battle scars to prove it, most likely the first place you turn whenever trouble starts brewing in your Windows environment is the Windows Event Log. Whether it's application exceptions, user lockouts, failed backups, services stopping, or anything else that happens to go bump in the night, your first clue as to what went wrong, and when, can almost always be found in your tried and true Windows Event Log.

 

However, until now, your only recourse for pouring through the mountain of events in the Windows Event Log has been to either Remote Desktop (RDP) into your server to open the Windows Event Log Viewer locally, or launch the Windows Event Log Viewer on your workstation and connect to the servers Event Log remotely. Both of these options introduce needless additional steps and precious time wasted when trying to quickly troubleshoot the root cause of the issue. Neither of these options allow you to proactively monitor for these kinds of events in the future, should they occur.

 

In SAM 6.0 we wanted to bring the same power to your fingertips that we provide with the Real-Time Process Explorer and Windows Service Control Manager, to the Windows Event Log. To that end, the Real-Time Windows Event Log Viewer was born. Found in the Management resource on the Node Details view, the Real-Time Windows Event Log Viewer allows you to browse through all the events in your Event Log, as well as filter for specific events in a particular log. You can filter based on event severity, such as Error, Warning, Information, etc.  You can also filter on Event Source to quickly isolate the specific event(s) you're looking for.

Real-Time Windows Event Log Forwarder.png

Once zeroed in on the event(s) you're interested in, you can click on the event itself to see the full details of the event, including the full message details. Want to be alerted if this event occurs again in the future? No problem. Simply click Start Monitoring and you will be walked through a wizard for creating a Windows Event Log Monitor and assigning it to a new or existing Application Template. It's just that easy.

 

Asset Inventory

Ok, I know what you're going to say. Inventory collection and reporting is neither cool nor exciting. I get it. However, regardless of your environment size, keeping an up-to-date inventory is a necessary evil normally required by the accounting department or upper management for asset depreciation tracking, insurance purposes, and/or lifecycle management and renewal. It's normally an arduous annual or quarterly task that involves a tremendous amount of manual labor to compile, and has questionable value beyond satisfying those higher up the ladder.

 

In SAM 6.0 we strived to make inventory data not only valuable to upper management, but also those responsible for managing and maintaining the infrastructure. Expanding beyond the typical fixed asset information required by the business, SAM 6.0 allows you to answer key IT questions that help drive your business decisions. Whether you're constantly running low on memory in your server and need to know if you have any free slots for additional RAM, or need to report on software installed for license compliance or security purposes, SAM has you covered. Has your vendor notified you of a major issue identified in a specific driver or version of firmware? With SAM 6.0 you can easily see which machines are affected.

Inventory List Resource.png

 

You can enable Asset Inventory collection for a node by clicking List Resources from the Management resource of the Node Details view. It will also appear listed when adding the node through the Add Node Wizard. To enable collection, simply check the box next to Asset Inventory as depicted in the image on the left. Click submit to save your changes and inventory data for that node will collected.

Asset Inventory collection can be enabled for both physical and virtual assets alike, and functions independently of Hardware Health monitoring, or any other status monitoring for that matter. That means you don't need to monitor your volumes or interfaces for them to be included as part of an inventory report.

 

Asset Inventory collection can be enabled for any Windows host managed in SAM, including Windows servers and workstations. Stay tuned for more information on inventory collection for Linux/Unix and VMware hosts.


Once Asset Inventory has been enabled for the node, you will find a new Asset Inventory sub-view that appears on the Node Details view for that node. Clicking on the Asset Inventory sub-view tab takes you to a dedicated asset inventory view which includes all relevant inventory data collected for that node.

When you access the Asset Inventory sub-view you may at first be overwhelmed by the sheer volume of information being collected. Fortunately all this information is logically categorized and grouped together into independent resources that can be moved or copied to the Node Details view, or any other node based sub-view if you desire.

 

Anyone who's Polling Engine capacity is teetering dangerously on the edge might be worried what kind of load all this newly collected inventory information is going to place on their poller. Fortunately inventory data doesn't need to be collected with the same degree of regularity as status information. As such, Asset inventory collection by default occurs only once a day, but can be configured to occur weekly, bi-weekly, or even monthly depending on your needs.

 

Concerned about licensing? Don't be. Asset Inventory Collection is not licensed separately, and does not count against your SAM component monitor licenses. It's included as part of the Node License. 

Graphics and Audio.png

Asset Inventory Subview.png

 

Below are just a few examples of some of the new Asset inventory resources contained in this view. If you'd like to check them out for yourself in your own environment I encourage you to sign-up here to download the SAM 6.0 beta. We'd love your feedback!

New Asset Inventory Resources

 

Memory

Memory Modules.png
Processors
Processors.png
Software Inventory
Software Inventory.png
Drivers
Drivers.png
Logical Volumes
Logical Volumes.png
Network Interfaces
Network interfaces.png
Removable Media
Removeable Media.png

The dust is still settling after the blowout success of the SAM 5.5 release earlier this week, but that doesn't mean it's time to rest on our laurels. In fact we're hoping the next release is even bigger than the last! With that said, below is a list of some of the bigger items the mad scientists in the lab looking at cooking up.

 

  • Improvements in SQL Server Database Monitoring
  • Server Asset & Inventory Management
  • Real-Time Windows Event Log Viewer
  • Statistical Baselining with Automatic Recommended Threshold Calculations
  • Disk I/O Performance Metrics for Monitored Processes and Services
  • Web Services Monitoring for SOAP & JSON
  • Host Header Support for HTTP/HTTPS User Experience Monitors
  • Update all Remaining Charts to New Charting Library
  • User Auditing of SAM Configuration Changes and Remediation Actions
  • Additional macros for script monitors
  • Reduce product license key management overhead associated with major version upgrades and maintenance renewals

 

PLEASE NOTE:  We are working on these items based on this priority order, but this is NOT a commitment that all of these enhancements will make the next release.  We are working on a number of other smaller features in parallel.   If you have comments or questions on any of these items (e.g. how would it work?) or would like to be included in a preview demo, please let us know!

Server & Application Monitor 5.5 Release Candidate has been available for a few weeks, and now RC3 is available for download through the customer portal for all existing SAM product owners under active maintenance. Throughout the beta process I've written about several new features that expand SAM's capabilities beyond simply monitoring applications. Another example of this, is newly added support for monitoring network interface performance of Windows servers via WMI. Once an exclusive feature of Network Performance Monitor, known primarily for monitoring switches, routers, firewalls, network load balancers, and other network gear, NPM has been the industry’s de facto standard in network performance monitoring for a generation of network engineers the world over. Now SAM 5.5 gains some of this NPM goodness by providing users visibility into their Windows servers network performance, and allowing them to diagnose and troubleshoot server network issues that commonly result in latency, packet loss, and overall poor application performance.

 

When you add a Windows Node to SAM via WMI, or List Resources on an existing WMI managed node, a listing of network interfaces installed in the server is presented. Select the interfaces on the server you want to monitor and you're done. It's just that simple, with no need to enable or configure SNMP.

List Resources WMI Interfaces.png

  • The listing of interfaces you see in SAM mirror those shown in the Windows Network Control Panel of the server being monitored.

Windows Network Interfaces.png

  • Once monitored, a listing of these interfaces and their current utilization is shown on the Node Details view of the managed Windows host in SAM.

Current Percent Utilization.png

Interface Details.png

 

Clicking on any interface in this resource takes you to the Interface Details view, where general information about the selected network interface, such as MAC address, interface type, and link speed can be found. In this view you'll also notice a plethora of network interface performance information. Interface utilization, Min/Max/Average bits per-second, packets per-second, as well as errors and discards, both current and historical, are all available to help aid in diagnosing Windows server and application performance issues.

Min/Max/Average bps In/OutPercent UtilizationTotal Bytes TransferedErrors and Discards
MinMaxAvgBps.png

Percent Utilization.png

Total Bytes Transfered.png

Errors and Discards.png

 

The best news of all is that network interface monitoring for Windows machines via WMI is essentially free, as it's included as part of the node license. Now what does that mean exactly? If you're already managing a windows host via WMI with SAM, then monitoring its network interfaces does not consume any additional licenses. What does that mean for those running both SAM and NPM on the same server? Well it means you can continue to monitor your Windows network interfaces via SNMP, or opt to monitor them via WMI. The choice is yours. Windows interfaces monitored via SNMP will continue to count against your NPM licensed interfaces, while those monitored via WMI will not. If you're an existing NPM product owner nearing your interface license limit and you don't yet own SAM, this is a great way to reclaim some of those interface licenses while adding tremendous value to your organization.


The best kind of properties are custom

 

A few months ago NPM 10.4 introduced us to a radically overhauled Custom Property Editor, that made a complete web transformation from its previous Win32 underpinnings. With the release of SAM 5.5, what was once the exclusive domain of nodes, volumes, and interfaces, has now been expanded to include applications at the request of many in the community.

Application Custom Properties.png

Application Template Editing Custom Properties.png

As one might expect, these application custom properties can be used in the Advanced Alert Manager, Report Writer, Groups, filters etc, the same as other custom property types are used throughout the product.

 

Once the custom properties have been defined you can designate values to your assigned applications through the Custom Property Editor found under Settings. Similar to other custom property types, this is normally the easiest way to modify custom property values for applications en masse. Alternatively, you can edit specific applications and define their individual custom properties using the application editor as pictured on the left.

 

These are two more great new features you can begin using today by downloading Server & Application Monitor 5.5 RC3 from your customer portal. The Release Candidate can be used to upgrade your existing SAM installation, but also includes a temporary 30 day license if you'd prefer to play with these new features in a lab environment.

If you've been chomping at the bit to upgrade to SAM 5.5 now is your chance to get the goods before everyone else. Simply sign-up here to download the SAM 5.5 Release Candidate.

 

If you've been following the Product Blog closely the past few months you'll no doubt be familiar with several of the major new features we've been working on for our next release of Server & Application Monitor. In my previous posts I showcased features such as: T

 

 

One area however, that's seen continual improvement with each successive release but doesn't get the same level attention as many of the top billed new features are usability enhancements.

 

At SolarWinds usability and user experience aren't merely philosophies, but rather they're a way of life. It's about ensuring that what you need, is where you need it, when you need it. Many of you have no doubt worked with some of SolarWinds usability experts in the past. Reviewing early prototypes, mockups, or talking through concepts, helping us to flesh out design flaws before even a single line of code is written. For those of you who may not have taken part in SolarWinds usability studies in the past, don't let these opportunities pass you by. We take your feedback very seriously, and it's instrumental in shaping the future of our products. Below are just a few examples where your feedback has had direct influence on this release of SolarWinds Server & Application Monitor.

 

Point - Click - Remediate

 

Management Resource Node Details.png

More than a few of you have realized the value of the Real-Time Process Explorer, introduced in SAM 5.0, but found it far too buried on the Application Detail view. Being on the application details view also meant that using this awesome feature was limited to those machines which already had application templates assigned to them. This made using the Real-Time Process Explorer to monitor new processes under certain circumstances difficult or impossible.

 

With that feedback in hand our usability experts set out to do what they do best. The result of that labor as you will see In the SAM 5.5 Release Candidate is a new Management resource on your Node Details view. Many of these management functions you're already familiar with, as they were previously located on the Node Details resource. This new dedicated Management resource however, has been extended to provide quick access to SAM management functions like the Real-Time Process Explorer, as well as the new Windows Service Control Manager, and server Reboot action.

 

These buttons appear dynamically in the Management resource on the Node Details view where applicable, depending upon device type. Windows hosts display all three management functions, the Real-Time Process Explorer appears for Linux/Unix hosts running NET-SNMP, and all three are hidden from view for other devices such as Cisco routers and HP switches. By virtue of making the software smarter, and options more accessible, it's inherently easier and more intuitive to use.

 

Quick Link Service Control Manager

 

When we set out to design the Windows Service Control Manager for SAM 5.5, the concept was fairly straightforward; provide a method for remediating common application issues.A listing of all services available on the managed node is presented when the Service Control Manager is opened, allowing for those services to be stopped, started, or restarted, depending on their current state. This same interface would also serve as a quick access method for creating new Windows Service component monitors in SAM. What became apparent after our user research is this introduced an unnecessary additional step for those services already monitored by SAM. In other words, if I'm already monitoring said service, then why do I need to open the Windows Service Control Manager to alter the service state? That was an excellent point and one we chose to remedy by adding the service remediation actions to the Component Details resource as pictured below for any Windows service currently monitored by SAM. This subtle change reduces mean time to resolution when application issues occur, by ensuring that what you need is readily at hand.

Stop-Restart Monitored Service.pngStart Monitored Service.png

 

Bringing Order to The Chaos

 

Structure and organization are cornerstones of managing anything of substantial size and importance, and applications and the templates that monitor them are no exception. The SAM 5.2 release saw a radical overhaul of the template editor, which itself was a direct result of many hours spent talking to users like yourself. One item however, that didn't make the cut for that release was reordering components within an application or template. Why, you might ask, would I want to reorder components? Templates and applications are dynamic in that users update them periodically as their monitoring requirements change. Unfortunately until now, whenever a new component monitor was added to an application template, or assigned application, this component was tacked-on to the bottom of the list with no ability to change its order. What we found in our usability research was that users wanted the ability to organize their component monitors in order of importance and impact on the monitored application. This would bubble critical issues up to the top of the page, and leave lesser important or less likely causes to the bottom. By implementing this feature in SAM 5.5 we further speedup the mean time to resolution by reducing time spent page scrolling, and allowing users to order components according to impact and priority. Another win for you, and the SolarWinds User Experience Team.

 

Reorder Component Monitors.png

 

These are just a few examples of some of the new usability enhancements included in this release, purpose built to make your job a little easier, and there's still plenty more features I haven't had a chance to blog about yet. Such as...

 

  • Windows Network Interface Monitoring via WMI
  • Exchange Web Service User Experience Monitor
  • RPC Fetching Method option for the Windows Event Log Monitor
  • Application Custom Properties
  • Private Key Authentication for Linux/Unix/Nagios Script Monitors
  • Independent Email Send/Receive Credentials for POP3/IMAP4/MAPI User Experience Monitors

 

to name only a few. If you'd like to kick the tires on some of the new features and usability enhancements included in the SAM 5.5 Release Candidate sign-up here. Release Candidates are fully supported and can be used to upgrade your existing installation of SAM. The download also includes a 30 day evaluation license key if you're not yet ready to upgrade. Either way, we'd love your feedback.

Microsoft's Windows Management Instrumentation, better known as WMI, is powerful remote API that's available with all Windows desktop and server operating systems since Windows NT 4.0. It's enabled by default and requires no configuration to utilize. It's this simplicity, coupled with the vast array of valuable system information that makes it invaluable for agentlessly monitoring Windows performance and collecting system inventory information remotely.

 

However, WMI is not all rainbows and lollipops. There are a few notable areas lacking in its implementation. For example, unlike SNMP, WMI is not very latency friendly. It's a protocol best suited to local area networks, or WANs where latency normally averages less than 100ms. That makes the WMI protocol less than ideal for remotely monitoring hosts via satellite, or heavily congested and highly latent, long distance, low bandwidth WAN links. In these scenarios it's recommended to monitor these remote hosts using SNMP, or deploy a dedicated Orion Instance to this remote location where polling can occur locally; leveraging the Enterprise Operations Console to provide a single pane of glass view for your entire organization.

 

WMI Protocol Security

 

I'm often asked about the security of the WMI protocol. While it's true that the majority of implementations lack payload encryption, the most meaningful component, the credentials used to gain access to the remote computer are fully encrypted using NTLM, or Kerberos. This means that even in the most insecure environment, the most sensitive information a potential hacker intercepting your packets is likely to uncover is your servers' current CPU or memory utilization. Hardly top secret stuff. The image below shows what a raw WMI packet looks like when SAM is monitoring the IIS World Wide Publishing Service "W3SVC" remotely.

 

Service Name.pngPacket Capture RAW.png

 

In the packet capture above you can see the actual WMI query starting with "SELECT", all the way down to the name of the service being monitored by SAM. Again, this isn't really anything to be too concerned about. After all, 294 billion emails, many of which contain far more sensitive information than how much free disk space you have remaining on your server C:\ drive, are sent completely unencrypted via the internet every day.

 

However, for environments where regulatory mandate dictates the highest standard of network encryption requirements, such as the Department of Defense, there are a variety of different methods for encrypting WMI communications. Most commonly used are IPSEC Policies built into Windows, which serve to ensure that each and every packet communicated between these two hosts are fully encrypted, and safe from prying eyes. It's not something I'd recommend for every environment, but chances are good that if you fall under a similar regulatory mandate then you're already all too familiar with many of the options available to you for encrypting communications between two hosts.

 

Restricting WMI Port Usage

 

Undoubtedly the most common question posed to me as Product Manager of both Server & Application Monitor, and Patch Manager, both of which heavily utilize WMI, is what port WMI uses to communicate with a remote host over the network? This is a somewhat loaded question, since WMI isn't limited to any particular port at all. In fact, the closest, best, and most accurate answer I can provide is that WMI is built on DCOM, and leverages the RPC Portmapper service for dynamic port allocation. Put simply, this means that WMI uses TCP Port 135 to initiate communication with the remotely managed host, then switches to any random high port anywhere between TCP port 1024-65535.

 

WMI Packet Capture.png

 

This is normally the point in the conversation where the person I'm speaking with grimaces and moans, uttering that there's no way the individual in charge of network security is going to be "okay" with opening 64512 TCP ports on the Firewall or router's access control list, simply to monitor their servers or push patches. Essentially this would be the equivalent of having no firewall at all!

 

Not to worry, there are a variety of different methods that can be used to limit the range of ports WMI uses to a much more meager number. It's important to note however, that Microsoft recommends that you don't limit your WMI port range down too tightly. Each allowed port in the designated port range allows one concurrent WMI connection. So the more applications you have that leverage WMI, the more ports you're likely to consume. I've personally had long-term success limiting the WMI port range down on SAM managed nodes to as few as 10 ports, but your mileage may vary. So for the purposes of this blog post I'll stick with Microsoft's recommended 100 ports. There are some additional notes you should keep in mind while reading this post and before implementing any changes.

 

Prerequisites

 

  • Consider your port range carefully before proceeding to ensure there are no conflicts with ports already in use by critical business applications in your environment. For instance, if you plan to use ports 5000-5100 as provided in these examples, you might not realize that some of these ports overlap with other applications such as FileMaker Pro. While this shouldn't be a major issue, it will reduce the total number of ports available to WMI for concurrent connections. This however does become vitally important to pay close attention to if you have plans to reduce the WMI port range to only a handful of ports. It's probably worth spending a little time determining which ports are being used in your environment by monitoring your network traffic using something like our NetFlow Traffic Analyzer. Similarly, you could use the Network Configuration Manager to report on all open ports that your Windows hosts are listening on.

 

  • Group policy can take as long as 90 minutes to update a clients configuration. If you plan to use method #2, or implement method #3 below via group policy, it's important to remain patient. You can always verify a clients configuration using Regedit and validating the contents of the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Rpc\Internet key. If you have easy access to the machines receiving their configuration settings via group policy, you can also force a policy refresh by opening a command prompt on the host and running the "gpupdate /force" command.

 

  • These changes require rebooting the remote host before they take effect. In the case of workstations, you may be able to rely on your users to shutdown their machines at the end of the day and turn them back on when they return. For servers, you will more than likely have to wait for a maintenance window, such as Patch Tuesday before you're allowed to reboot and these changes take effect.

 

  • Neither SAM nor Patch Manager require WMI. It's a powerful, but completely optional component to both products. In the case of SAM, there are SNMP and RPC alternatives for almost every instance where WMI is available. Patch Manager includes optional WMI Providers that allow for the immediate deployment of patches to end hosts, eliminating the need to wait for a regularly scheduled phone home event from the client to check for new updates. Regularly scheduled patch deployments however, have no reliance on WMI.

 

  • WMI Ports are opened (listening) only as needed, based on an initial request sent via TCP 135. So while the examples below demonstrate allocating a range of 100 ports to WMI, a port will only be opened for use once the connection has been properly authorized and authenticated during that initial RPC request.

 

  • Be sure you've made provisions for allowing TCP Port 135, as well as, the range of ports you've chosen to allocate to WMI to the Windows Firewall exclusion list. For added security, you can optionally limit this exclusion to the source IP address of your SAM or Patch Manager server.

Windows Firewall Protocols and Ports.png Windows Firewall Scope.png

 

 

Method 1 - Modify The Registry Directly

 

Perhaps the simplest, and most straightforward way to limit the port range WMI uses is directly through the registry of the remotely managed host. This can be done manually following the steps below.

 

  • Add the Internet key under:  HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc
  • Under the Internet key, add the values "Ports" (MULTI_SZ), "PortsInternetAvailable" (REG_SZ), and "UseInternetPorts" (REG_SZ).

In this example ports 5000 through 5100 inclusive have been arbitrarily selected to help illustrate how the new registry key can be configured. For example, the new registry key appears as follows:

 

Ports: REG_MULTI_SZ: 5000-5100
PortsInternetAvailable: REG_SZ: Y
UseInternetPorts: REG_SZ: Y

 

WMI_Regedit.png

  • Restart the server. All applications that use RPC dynamic port allocation use ports 5000 through 5100, inclusive. In most environments, a minimum of 100 ports should be opened, because several system services rely on these RPC ports to communicate with each other.

 

I’ve also attached the WMI Ports 5000-5100.zip that packages these changes into an easy to use single registry file. Simply extract the archive, and double click on the "WMI Ports 5000-5100.reg" file to make the above changes on the machine and reboot.

 

Method 2 - Active Directory Group Policy Administrative Template

 

Alternatively, these settings can be centrally configured and deployed via group policy to any machine within the domain. While the manual method referenced above may be suitable for a small handful of machines, such as those in your DMZ, it simply doesn't scale should you need to make these changes to hundreds or possibly thousands of managed hosts.

 

For instances such as this I’ve packaged an Active Directory Administrative Template, WMI Group Policy.zip,that can be used to configure the WMI port range through Active Directory. It's important to note that this Administrative Template can only be used with Windows 2008 and greater functional role level domains, though any machine joined to the domain, from Windows 2000, XP, through Windows 8 and Server 2012 will respect these policy settings.

 

Once extracted, copy the “WMI.ADMX” file to the “%systemroot%\PolicyDefinitions” directory on your domain controller. Then copy the “WMI.ADML” file to the “%systemroot%\PolicyDefinitions\en-US” directory. If you share your administrative templates across multiple domain controllers and would like these to be available to all DCs you can alternatively place WMI.ADMX in the “%systemroot%\sysvol\domain\policies\PolicyDefinitions” and the “WMI.ADML” in the “%systemroot%\sysvol\domain\policies\PolicyDefinitions\EN-US” directories.

 

Now that you've extracted the Administrative Template into the appropriate directories, you can utilize it in the same manner as any other group policy setting. From within the Group Policy Management Editor, either open an existing group policy that’s already applied to the devices you’d like this change applied to, or create a new group policy. Right Click on the Group Policy and click “Edit”.

Group Policy Management.png

Within the Group Policy Management Editor expand “Administrative Templates” – “Networking” – “RPC Static Pool” and double click on “RPC Static Pool Definition”.

Group Policy Editor.png

Inside the RPC Static Pool Definition select the “Enabled” radio button to apply the following settings to the objects this group policy applies to.

 

“Use Internet Ports” = “Y”

“Ports Internet Available” = “Y”

“Define Internet ports for RPC below” = “5000-5100”

RPC Static Pool Definition.png

When done click “Apply”. This group policy change should take effect within 90 minutes for all nodes contained in the OU.

 

Method 3 - Create Your Own Group Policy Setting

 

The last method for distributing these policy changes that I'll discuss in this blog post is creating your very own Group Policy setting. It's perhaps not as elegant or neatly packaged as the Administrative Template method above, but it does have the distinct advantage of working on older Windows 2003 Domain Controllers. Also, unlike Administrative Templates you can follow essentially the very same steps below on any Windows machine that's handy and define a local RPC port range for that workstation. This means you can play around using this method without needing to implement any changes on your Domain Controller. Once you're comfortable with these settings however, you can implement them via Group Policy in a similar manner to distribute them across your domain.

 

  • Before we begin you'll first want to make a backup copy of the "sceregvl.inf" file located in "C:\WINDOWS\INF". This will allow you to quickly and easily revert any changes made should you need to for any reason.
  • Next, you'll need to take ownership of the "sceregvl.inf" file by right clicking on the file, selecting "Properties", then selecting the "Security" tab and clicking "Advanced". Inside the "Advanced Security Settings", select the "Owner" tab click the "Edit" button to select your Domain "Administrators" group, for which you should already be a member. Lastly, click "Apply" and "OK" to save these settings.
  • While still inside the security properties of the file we need to ensure the Domain Administrators group has all the necessary permissions required to modify the file. To do so, select your Domain Administrators group from the list of "Group or user names" and select both "Allow" check boxes next to "Modify" and "Write" under "Permissions for Administrators" as pictured below.

Take Ownership sceregvl.inf.pngModify Permissions sceregvl.inf.png

 

  • Now that security permissions have been properly modified, it's time to get down to business. Start with launching Notepad as the Administrator by holding down the "Control" key on your keyboard while right clicking on Notepad and selecting "Run as administrator". Once launched, open "C:\WINDOWS\INF\sceregvl.inf" from within Notepad using the "File" menu.

Open Notepad as Administrator.png

  • Next we'll need to add a few lines to the top section of the "sceregvl.inf"configuration file, just above the other registry keys as pictured below. These configuration settings can be copied as written here, then pasted directly into the "sceregvl.inf" configuration file. Once completed don't forget to save the file.

 

MACHINE\SOFTWARE\Microsoft\Rpc\Internet\Ports,7,RPC_Ports,4

MACHINE\SOFTWARE\Microsoft\Rpc\Internet\PortsInternetAvailable,1,RPC_PortsInternetAvailable,2

MACHINE\SOFTWARE\Microsoft\Rpc\Internet\UseInternetPorts,1,RPC_UseInternetPorts,2    

Edit sceregvl.inf.png

 

  • Before we can start playing with our new policy settings we first need to re-register the "scecli.dll" with Windows. To do this, you'll first need to open a command prompt window as Administrator in the same manner we launched Notepad in the previous step. Right click on the "Command Prompt" while holding down the "Control" key and select "Run as administrator". Then run the command "regsvr32 scecli.dll' to re-register "scecli.dll". When executed, you should receive confirmation that the DLL was successfully registered.

regsvr32 scecli.dll.png

 

  • Similar to the Administrative Template method above you can now configure these settings in the same manner as any other group policy. From within the Group Policy Management Editor, either open an existing group policy to an organizational unit that already contains the devices you’d like this change applied to, or create a new group policy. Right Click on the Group Policy and click “Edit”.  Within the Group Policy Management Editor expand “Windows Settings” – “Security Settings” - "Local Policies"  and select "Security Options". There you will find these new configuration options which can be defined using the following suggested settings.

 

“RPC_Ports” = “Y”

“RPC_PortsInternetAvailable” = “Y”

“RPC_Ports” = “5000-5100”

RPC_Ports.pngGroup Policy Settings.png

 

Show me the Money

 

Regardless of which method you choose, the end result is essentially the same. Instead of WMI using any random TCP port greater than 1024, it's now confined to a limited range of ports that both you and the security czar will unanimously agree strikes the perfect balance of safe and functional. The image below demonstrates this result in a packet capture. You can still see the initial request via TCP Port 135 then switch to TCP Port 5002. This is because there are already two other WMI connections to this host using TCP Port 5000 and 5001. The WMI port range recycles ports as connections are terminated or timeout; therefore, provided your port range is adequately sized, there should be no concern of port exhaustion.

 

Packet Capture Result.png

Filter Blog

By date:
By tag: