Tuning Noisy Alerts: SQL Notebooks
When SQL Sentry is first deployed and begins to alert on the conditions that are shipped with the product, you can get a lot alert emails that can be at times overwhelming. This short post is all about tuning those noisy alert conditions.
If you are using Azure Data Studio, hopefully you are aware of SQL Notebooks. SQL Notebooks are great for many things and useful in certain situations. As a DBA, I created SQL Notebooks to give to Junior or Mid-Level DBAs to collect certain diagnostic information for troubleshooting or the beginnings of root cause analysis. SQL Notebooks allow you to save the results to the Notebook making the exchange of data between the teams a simple process. Depending on what Kernel you use the Notebook could also be a PowerShell Notebook. Notebooks are not just to be used in Azure Data Studio but also be used VS Code depending which editor you prefer.
The Notebook attached to this post is designed for Alert mining when going through the process of tuning the alerting from SQL Sentry. Each code block has a corresponding text block describing what the result set to be expected. By default, SQL Sentry retains a year’s worth of alert data, so it is recommended that you limit your time range to the last 30-90 days. As you go through cycles of alert tuning, be sure you set the start times for after each tuning cycle, as the older alert data will still be there.
Here is a screen shot of the Notebook from Azure Data Studio.
Here is a breakdown of each section in this SQL Notebook. Each of these queries will return a count for each condition and how many times those conditions have been fired. This will help you determine which conditions to target for tuning.
- Global General Condition counts with ActionType of SendEmail
- General Condition counts by target with ActionType of SendEmail
- Global Advisory Condition counts - Send to Alerting Channels/Health Score
- Global Advisory Condition counts - Send Email
- Advisory Condition counts by target - Send to Alerting Channels/Health Score
- Advisory Condition counts by target - Send Email
I hope this helps in removing “alert noise” and ensuring getting value from SQL Sentry.