Add the alert name in the AlertService.log file

Whew!! You're giving me a heart attack. For some unknown reason, I checked the AlertService.log file and noticed this message was occuring every minute ... I was freaking out that it was the weekend and the log would be

HUGE if I didn't figure it out quickly.

2013-02-22 14:30:02,105 [AlertCheckingThread] WARN Error - Error in UpdateRowsThatAreTriggered() - GetNetObjects

System.Data.SqlClient.SqlException (0x80131904): Ambiguous column name 'NodeID'.

at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)

at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)

at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()

at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)

at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()

at System.Data.SqlClient.SqlDataReader.get_MetaData()

at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)

at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)

at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)

at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)

at SolarWinds.Orion.Common.SqlHelper.ExecuteReader(SqlCommand command, SqlConnection connection, SqlTransaction transaction, CommandBehavior behavior)

at AlertingEngine.CheckAlert.UpdateRowsThatAreTriggered()

Luckily it started today, but I was having a hard time determining which alert was causing the error .. because you CAN'T tell from the log file. So, I started disabling the alerts I thought I had changed today & FINALLY figured out which one it was.

The culprit was with an APM:Application alert. I had a condition group of "..when NONE apply" and a condition of 'Last Booted has changed'. Once I removed that condition the errors stopped getting logged.

I know I'm a newbie, but how can this be? I guess I need to write an alert for when the AlertService.log file changes.

How to determine which alert contains an SQL error / bug.

Find more posts tagged with

Alert

prodfr

debug

alert_lab

log_data

Status: None

Comments

marunderwood

Happened again yesterday. Am I the only one writing custom sql alerts that have syntax errors? Maybe the system should also force you to cllick the button to check the SQL syntax, if it changed. This time, I didn't notice the problem (because I normally don't check this log file) for 12 days and it was logging an error every 5 minutes. Also, the method (I assume) is 'UpdateRowsThatAreTriggered' is referencing the Trigger Query. To find the alert, I wrote a report over the AlertDefintion table searching for one that triggered every 5 minutes (ExecuteInterval = 300) and the query contain ' Node' (upper(TriggerQuery) like ' NODEID'). The complete sql is below.

Error log contained:

2013-11-25 17:24:35,463 [AlertCheckingThread] WARN Error - Error in UpdateRowsThatAreTriggered() - GetNetObjects

System.Data.SqlClient.SqlException (0x80131904): Ambiguous column name 'NodeId'.

at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection) ...

Report SQL:

select
ad.AlertDefID,
ad.AlertName,
ad.Enabled,
ad.ExecuteInterval,
ad.LastError, ad.LastErrorTime,
ad.TriggerQuery
, ad.ResetQuery
, ad.ResetQueryDesign

from AlertDefinitions ad
where
    Enabled = 1 /*-- True*/
/* ad.AlertName like '%high cpu load%'
--   and ad.LastError like '%ambiguous%'
--   and ad.LastErrorTime is not null
*/
        and ad.ExecuteInterval = 300
        and UPPER(ad.ResetQuery) like '% NODEID%'
order by
ad.lasterrortime desc,
ad.AlertName

marunderwood

Thanks yaquaholic. How about a vote?

highstone351

Everyone:

This is caused by the conversion of NPM alerts that had "NodeID" as an undocumented column in the alert. When the conversion utility tried to convert it to the Web Version, it could not. Instead of disabling the alert and telling us about the problem, it simply stated that one alert could not be converted. It did not say why or how. If you have looked at the database lately, you will notice a lot of changed columns. Many dbo's and their columns have been deprecated to new ones. There is no documentation about the changes.

This should have been done since it would be easy for DEV to just list old dbo's or columns now = <this new one>.

One has to delete the OLD alert and create it using the Web page. Before you delete it, see if you can bring it up and copy the logic and any OID information you might need.

Support will show you how to look for any problem alerts, etc.

I hope that this helps.