Back in April of 2015, NPM 11.5 was released and with it came a brand new Web-based Alert Engine in the Orion Platform. At the time, and ever since, one of the most valuable capabilities of this new engine was the ability to dynamically alert on multiple objects based on their own individually assigned thresholds. Setting individual thresholds for things like CPU Utilization, Percent Memory Utilization, Packet Loss, Response Time, Interface Errors, and Interface Utilization was a game changer to a lot of alerting schemas that allowed us to reduce our custom property footprint, as well as the complexity of the alert definitions. However, a glaring "omission" was that the thresholds made available for Volumes were not presented to the alerting engine (or so we thought). This was a bit mind-boggling, and talking to other MVPs, seasoned SW Admins, and SW employees over the years, I had never heard differently, so the assumption was cemented as a "missing item that requires a work-around". (On a side note, I am 42% sure that jbiggley was behind a very well orchestrated and elaborate trolling to keep me in the dark on this capability, but I digress...) But today, I'd like to present the solution that was hiding in the background this entire time, to save future admins the discomfort of maintaining "Disk_Crit" custom properties.
NOTE: As tait.cyrus mentions in the comments below:
A problem with using "Volume Capacity Forecasting" is newly added nodes will not have any forecasting data for several days, so you will not be able to get any volume alerts from newly added nodes (until forecasting volume data becomes available which can be somewhere in the 1-3 days time period) since volume thresholds won't appear until the forecasting data appears.
The problem arises from the fact that "Volume Capacity Forecasting" is a database 'view' made up of data from various other tables including a database JOIN of a forecasting table so until there is forecasting data available for a volume on a node, "Volume Capacity Forecasting" will not show anything so no threshold data will be available and thus no ability to generate volume alerts on the node. It would have been preferable that an appropriate database JOIN would have been used that would have shown threshold data even when forecasting data was not yet available. I did submit an incident on this and was told this is a known 'feature'. I have requested that this be changed so volume thresholds show up immediately allowing volume alerts to be immediately generated in newly added nodes.
tl;dr - Be aware that this solution has limitations on new volumes!!!
Background: Node and Interface metric thresholds are added to the alerting engine in a very intuitive way:
However, volume thresholds are obviously not:
The key was to take a step back and look at the alerting object options, there you shall find your salvation in the form of a "Volume Capacity Forecasting" object (as opposed to the intuitive "Volume" object type):
Which then presents those valuable thresholds!
From there, you need to setup a "Double Value Comparison" in the trigger:
And then create a comparison between the current and threshold values, respectively:
Which will then trigger on Volumes where their current percent utilization exceed the threshold you have defined on that specific volume:
For reference: thresholds are edited per object by editing the object's properties, and looking at the bottom of the page: (Pro Tip: you can edit multiple objects at once from the "Manage Nodes/Entities" page)
Verified via SQL search on the "VolumesForecastCapacity" view in the database:
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community.
More than 150,000 members are here to solve problems, share technology and best practices, and directly
contribute to our product development process.
Learn more today by joining now.