I think you might want to open a support ticket for this one.
I have the same problem, did you get a solution?
This error occurs when the SWAlertActions8.dll or SWAlertEngine8.dll is not registered.
To resolve this error:
1. Close all SolarWinds programs, including both the Advanced Alerts Manager and System Manager.
2. Browse to C:\Program Files\SolarWinds\Orion\.
3. Right-click SWAlertActions8.dll, and then click Open With....
4. Click Open With....
5. Select Select the program from a list, and then click OK.
6. Click Browse..., and then browse to C:\WINDOWS\System32.
7. Execute regsvr32.exe.
8. Register SWAlertActions8.dll.
9. Repeat the above for SWAlertEngine8.dll
10. Open the Orion service Manager, and restart the Orion Network Performance Monitor service, and the Alerting engine.
If this doesn't work which I have seen happen - go to Add/Remove Programs - right click on the Orion Core installation and select repair, after that allow it to run (the config wizard will run afterwards) when this completes, run the same repair on the Orion NPM install.
When this has finished you should be able to make changes/edit/create new alerts in Advanced Alert manager.