Alert Randomly Corrupt

Randomly this week (no recent upgrades or anything) we've noticed a ton of alerts aren't working. we recreate the exact alert, and it works. The only change is all of our alerts were disabled then enabled, but why would this corrupt them? We do this all the time.

Alerts which were triggering last week are now corrupt. This is a major issue; we are testing every alert one by one because we are not sure of the scope of this.

Has anyone ever seen something like this before? I've opened a ticket with SolarWinds already but typically takes several days before i get anything meaningful from support

version 2024.1

Find more posts tagged with

Alert

npm

HCO

Accepted answers

All comments

chad.every

That is odd behavior for sure. Not entirely sure what might cause it but I've seen issues with a sub-service that SW uses called 'RabbitMQ' that is tied to a handful of things in the platform. You can consider if you want to go this route before Support gets back with you, but you can reset this service by following "Option 2" in the following article

support.solarwinds.com/.../Full-RabbitMQ-Reset-in-Orion-Platform-products

adam.beedell

Something like this happened in one version in 2023. Did you upgrade up a few at once?

Defieguy@91

@acurrent

please keep monitoring number of elements getting polled on each poller and polling completion rate.

what you are facing is might be because of above said reason and have a look if alerts which are getting missed are belong to same polling engine or different one.