Hello, what you experienced is given by the alert migration concept. Let me explain briefly to understand how to fix that.
The legacy alerts had unique identifier "AlertDefId" in AlertDefinitions. Once the alerts are migrated to the new structures, migration creates the record in AlertConfigurations with the same identifier and disables the old alert (not deletes as we wanted to have fallback to revert the alert possibly in future). This causes the fact, that the alert won't be visible in old Alert Manager (simply hides the alerts, which are present in AlertConfigurations) and is ignored by the old Alerting engine.
Additionally the migrator looks to the new structures if it's present there and if so, than ignore it. The original intention for not removing the alerts was to let potentially the customers to revert specific alerts to the old engine when the alert won't be behaving as expected.
So what happened to you:
You deleted the migrated alert, which caused loss of inter-version versions reference, so the migrator doesn't know it must skip this alert. Additionally the old alert appeared in the old alert manager again. The bad is, that there's not tracked the "IsMigrated" flag and it's just driven by the table matches.
So what you can do here:
- I'd prefer to open c:\Program Files (x86)\SolarWinds\Orion\AlertManager.exe . You'll see here just alerts, which have no reference in new structures (you deleted them.) If those were originally succesfuly migrated, the definitions should be disabled. Then you can safely delete all you can see and those won't be migrated anymore. The "fix" will be permanent.
- If you don't like previous advice, before you'll run the Configuration wizard next time, open c:\Program Files (x86)\SolarWinds\Orion\ConfigurationWizard.exe.config with notepad, find following key,
<add key="AlertMigrationEnabled" value="true"/>
switch to false and save it. If you already had opened CW application, close it and reopen.
I hope it helps you to understand.
Thanks Jan I appreciate the response.
I'll go the recommended route using the AlertManager.exe tool.
Just wanted to follow-up with what we did here.
I did delete all of the rules listed in the old AlertManager.exe tool on our primary poller. However, it only listed the first 50 rules from the AlertDefinitions table. We still had well over 200 obsolete alert rules listed in the table. The only option from here was to delete the data in the table.
First we looked at how many alert rules there were in the table, and noted the row count.
SELECT * FROM AlertDefinitions
Order by AlertName Asc
Then we backed up the table by copying the data from the old table into a new one called "AlertDefinitionsBackup"...
SELECT * INTO AlertDefinitionsBackup FROM AlertDefinitions
Next we deleted only a few of the rows (alerts that we KNEW were not in use any longer), then looked in the Orion web console to make sure there weren't any problems.
We did this by first selecting a 'group' of alerts that were confirmed obsolete and got the row count from the query result set.
[We had several rules that started with "All Servers" in the title that were obsolete. This is just an example from our own environment.]
SELECT * FROM [NetPerfMon-Serverv10].[dbo].[AlertDefinitions]
WHERE AlertName LIKE '%All Servers%'
Order by AlertName Asc
Then, we deleted the records...
DELETE FROM AlertDefinitions WHERE AlertName LIKE '%All Servers%'
We went slow at first, confirming smaller deletes before wiping out the entire table.
After confirming that things looked ok finally we deleted all rows in the AlertDefinitions table.
DELETE FROM AlertDefinitions
Things still appear to be good to go. No alert issues at all.
We have a backup of the table in case a problem is discovered (especially if there is a problem with running Config Wizard again).