Reason for Alert:
DB guys reindex once a week. To check all is well on reindex we run a transaction that uses the newly indexed data. If all goes well, wait until next week and run again. If all does not go well and Alert signals a warning, critical or down... then the actions of alert contact all necessary and DB gets fixed or reindexed.
3 separate step level Alerts: One for detecting Warning, another for Critical and one for Down. The reset for the Warning is an UP status, the reset for Critical is an UP or Warning status, and for Down it is a Critical, Warning or UP status.
If Alert reports warning, critical or down... it has done its job, BUT... because it only runs the one time a week (a business cost decision), it is not able to reset until the following week if the condition no longer exists. In addition, if the condition is the same the following week... the alert does not trigger any actions, and the DB guys dont know there is a problem to fix, thus creating a bigger mess.
Because of the cost of business.... the Alert can only run once a week after the DB is indexed, how then does one reset the Alert in the reset logic or by some other means that will not negate future SolarWinds support?
Hi Paul, I discussed with our Dev and problem is not so much with alerting but with the fact that transaction stays in that same state for a week. They can think of two options:
1) If transaction is executed manually, user can unmanage transaction once he gets its status. That should remove alert. Once he needs to run it again, he can remanage it. He can do this by scheduling unmanage if he has always the same time for transaction run.
2) Have a scheduled SQL script that resets transaction status back to UP or better “Unknown” every X minutes/hours/…
Hope that helps,
Thank you for the response Peter...
Here's the thing...
Answer #1 We do not run this manually so this is not a possibility. It is however, what we end up having to do when the transaction starts running at the wrong time because somewhere in the week someone thought it would be neat to run... thus getting it off its schedule. We un-manage and remanage the day before we need it to run. Not a great solution.
Answer #2 SQL script - you are saying to run a script that will directly reach in the Orion DB and change the status field for the alert... NOT a kind of thing we like to do here. Directly updating fields created by SolarWinds is something we stay away from... to much at stake. I thought you support guys did not like us customers directly manipulating things in the DB. Are we not on the same track on this? looking at the same sheet of music? on the same page? : )
Was hoping that the WPM product would continue to be developed and there would be some additional features to the alerting reset trigger... like running SQL scripts within the application. Seriously... are there not going to be any more enhancements to WPM?
yes you are correct, that generally it is not recommended to touch DB, so your approach is correct. We are working on next release of WPM. I understand it looks like we are not, since the last release was some time ago, but we want to further develop WPM. Therefore I would recommend you to create a FR on WPM ideation page - Web Performance Monitor (formerly SeUM), so other users can vote and comment in it.
While unlikely that most may not have the situation of a transaction running only once a week, hence not having the ability to reset an alert until the following week... and also having the problem of no alert actions occurring because the next run resulted in the same status... here is one answer to the situation. Use the escalate feature for each alert action and set to alert again in 7 days if the status remains the same. Thanks to Xander Snyder for his thoughts on this problem.
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process. Learn more today by joining now.