When you're editing the built in Alert Message field in an advanced alert the changes are not reflected until you completely go through saving the entire alert.
It would be amazing if you could somehow commit the changes to the Alert Message field in order to test the alert actions right away rather than having to back out of the entire alert wizard