We've had numerous instances in SEUM 1.5 where playbacks stall but the status never changes to Unknown. All our recordings play back from 3 separate US locations, every 5 minutes. Periodically, certain playbacks "stall", and stop playing back entirely until manually restarted with the Play Now button. The GUI shows the last date/time the playback ran, but even if it's days/weeks ago, the status is still Green, since that was the last known status before it went dark. This is a serious issue for us, because our alerting looks for agreement among all 3 playbacks for either Red (down) or Gray (unknown). When all 3 playbacks are in either of those two states, we start paging engineers. If any one of them is Green (up), we do not. Stalled transactions create a critical blind spot. Previously, Solarwinds provided a SQL script to detect and alert on stalled events (>15 minutes without updates). However, it's still a manual process to login and address the stalled playback. During that time, we are blind. We are scaling up our infrastructure to meet the needs of the app. In the meantime, my questions are:
1) Shouldn't the "Unknown" status be written to the database when monitored objects fail to update? Is this a bug or do I misunderstand the status taxonomy?
2) Can weplayb mitigate this with a SQL job that sets the playback status to "Unknown" when the following alert logic is tripped?
Trigger Query = SEUM: Transaction
ON SEUM_TransactionsAlertsData.TransactionId = SEUM_Transactions.TransactionId
ON SEUM_Agents.AgentId = SEUM_Transactions.AgentId
WHERE ( SEUM_TransactionsAlertsData.IsEnabled = 1
AND DATEPART(year,SEUM_Transactions.LastDateTimeUtc) > '1900'
AND DATEADD(second, (3 * SEUM_Transactions.Frequency), SEUM_Transactions.LastDateTimeUtc) < GETUTCDATE()
AND SEUM_Agents.ConnectionStatus = 1