Alerts not working

After upgrading to 8.1 and moving to a new server my alerts no longer work. Manually firing the 'test alerts' works fine so I know the settings are right, its just not working when a node goes down. Has anyone else seen this?