This article will help you to quickly isolate the issue when you have any difficulty / issues with Orion Alerts as below .
You can quickly check your current installed environment and if you still have any further issues can open technical ticket with support.
Most common issues with Alerts are :
My Alerts are not working
I am not receiving emails from Alerts
I am not receiving emails from certain Alerts
Which logs i should check for the Alerts
I have Alerts issues after migration of Orion
I have Alerts issues after an upgrade
Your check list:
Check Alerting Service / all Orion Services
Check MSMQ Q [Must be 0]all Polling Engines
Check polling Engine DB Updated
Check CPU / Memory [SQL / Orion ]Servers
Check any pending reboot (after Windows and AV update )
Clear pending Subscriptions (if migrated Orion)
Check Polling Graphs (ICMP / SNMP)
Check SMTP Server
Check if the issue is with a specific Alert
Check if the issue is with a specific Alert Action
Check if the issue is with Certain Condition with Alert
Check Alerting Service DEBUG Logs for any Error
Check you are not using SQL Express Edition
Check your SQL and Orion Free Disk Space
Check Alerting Service / all Orion Services
RDP Orion Server and then Open Orion Service Manager > Make sure all the Orion Services are UP and running
No Service flapping
Check MSMQ Q [Must be 0] on All Pollers
Check your MSQM on the Orion Primary Server and all the Polling Engine (should not have any Q there and stays 0 )
Open computer Management and check
Check your Polling is even working :
Go to Settings > All Settings > Polling Engines
( All Polling Engines are Green )
(Database Sync )
Check CPU / Memory [SQL / Orion ]Servers
Check CPU / Memory on the Orion Pollers and SQL Server to make sure there is no resources constraint issues.
For more details please see this post
My Orion Website & Server is very slow / CPU & Memory Spike /Polling gaps what should i check quickly ?
Check any pending reboot (after Windows and AV update )
powershell - How can I check for a pending reboot? - Stack Overflow
If you think there is any reboot is needed please proceed - Consult with your Windows team
Clear pending Subscriptions (if migrated Orion)
If you have just migrated Orion Server application / Database.
https://support.solarwinds.com/SuccessCenter/s/article/Clear-Information-Service-Subscriptions
Check Polling Graphs (ICMP / SNMP)
Go to the Few Nodes (SNMP / ICMP ) Check the Last Poll Time is UP )
Check the Hardware Health polling
Check Node Polling Details
Check graphs are updating for SNMP and ICMP / Interfaces / CPU / Memory
Check you are not using SQL Express Edition
Please check if you are using SQL Express Edition database (DB will lock near 10 GB )
Alert will stop if this limit will reach up to the free limit for more details please see the KB post below.
Quick Orion database health check guide
Check SMTP Server
Please verify the SMTP Is working perfectly and you have received an email for the test
You can also use below method to
Verify poller have access to the SMTP Server on the configured port
From the affected poller please check if you are able to access your SMTP server.
Please Navigate following path
C:\Program Files (x86)\SolarWinds\Common\SWEmailNotify.exe
Run
SWEmailNotify.exe
Enter SMTP / IP Or Hostname
Click "Test Gateway"
Example:
You will receive following message for Successful connection
"Successfully connected to 192.168.1.50 220 mysmtpserver ESMTP
For Unsuccessful
Error Connecting to 192.168.1.13 the attempt to connect timed out
Testing your Alerting area
Please Note: During testing you will receive many emails / and Alerts under (Active Alerts ) Please consider this and only use your own email avoid Alerting for others.
Or you my can inform to the team before testing as they may receive Alerts / emails from Orion System which can be ignored during the testing phase.
Check if the issue is with a specific Alert
Verify Orion Alerting functionality is working
Please create a Brand New Alert ( Do not copy / clone )
Alerts and Activity > Alerts > Manage Alerts > Add New Alert >
Alert Condition Node / Status / is equal to / UP
Once on the Alert Action > use the Log the Alert to a File >
Create a folder Alerts on (Orion Server C drive )
Add the folder location into the Alert and file name (see screen shot )
C:\Alerts
Add Variable into the Message
----------
An issue has been detected at ${N=Alerting;M=AlertTriggerTime;F=DateTime} on ${N=SwisEntity;M=MachineType}
device named ${N=SwisEntity;M=Caption} (IP: ${N=SwisEntity;M=IP_Address}, DNS: ${N=SwisEntity;M=DNS})
View full device details here: ${N=SwisEntity;M=DetailsUrl}.
View full alert details here: ${N=Alerting;M=AlertDetailsUrl}
Click here to acknowledge the alert: ${N=Alerting;M=AcknowledgeUrl}
This message was brought to you by the alert named: ${N=Alerting;M=AlertName}
The node is monitored by the polling engine ${N=SwisEntity;M=Engine.ServerName}
Complete the Alert and check
Go to the folder location and check the Alert file which is just created by the Alert Action
Success ? This means Alerting Engine is working correctly -
Now Add the Email Action to the same within the same Alert and test .
If you have failure enable DEBUG level to the AlertingService.V2.log and Check the logs if you see any ERROR
Which logs you should check for Alerts
Enable DEBUG to the AlertingService
Open log Adjuster > On Orion Server > change the Level to DEBUG > Apply
Setup live log for AlertingService.V2.log file (See below how to check live logs )
View live Orion log activity Kiwi Log Viewer
C:\ProgramData\SolarWinds\Logs\Orion
AlertingService.V2.log
Now Re produce your issue . (Check if you see any ERROR in the logs ?)
Also please try this .
Disable All the Alerts and ONLY keep enable the New Alert you have created above
Example Case:
Alerts Not Working after trying above steps we found below Error.
For Example:
2019-12-04 14:32:32,369 [61] ERROR SolarWinds.Orion.Core.Alerting.Service.ActionsResolverInternal.PendingExecutionActions - Action ID: 430, ActionType: Email, Title: Send an Email/Page (Alert: ${NodeName} is ${Status}), Description: To: ${Node.Email} <br/>From: noc@abc.com<br/>Subject: Alert: ${NodeName} is ${Status}, Enabled: True, Order: 1 failed. alertActiveId: 7198 alertObjectId: 1770. Error: At least one recipient is required, but none were found.
Cause:
We found the Alerts are imported from old System using 3rd party tool (Which is not supported ) And Custom properties were not correct caused the issue under Alerting Action
Using Custom Properties sending Alert emails
Resolution:
Go the Alerts > Manage Alerts > Action Manager and correct the Recipient Variables for Custom Properties as explained in the above post.
In case of Failure (Collect Full Orion Diagnostics and Open Support Ticket )
Still need assistance from Solarwinds Support
Tips and Tricks on opening a Support Ticket with SolarWinds
