For the past 30 days, I've been base-lining metrics, fine-tuning thresholds, and working on alerts. I'm trying to reduce the number of alerts so that we only receive alerts when we are required to 'take action', and reduce the number of events so they're more meaningful. I've made good progress, however, I'd really like to be able to view a trend of alert and event volume. Ideally, something simple like # of events/hour and # of alerts/hour over the period of 1 day, 1 week, 1 month, etc. would work.
Has anyone created something like this, and/or have any thoughts on the best way to go about it?