Why are jobs queuing up in the billions? see attached. I also verified this using performance monitor on the server itself. Polling percentage was 98% but >2 billion jobs queued.
Jobs Queue is performance counter with type INT32. In case counter is decremented and original value is zero, it goes into negative value. But because it's signed int and it can't be negative that's why this value rollover and counter has its maximum value 2,147,483,647.
Explanation: someone restarted all counters with lodctr.exe, or there could be some synchr. issue inside jobengine with increment/decrement counter value.
Anyway this is harmless behavior and it has no impact on your system. As Rob already mentioned, restart service will fix it.
Ok. So I restarted the job engine V2 service and that cleared out the counters and the issue. The issue was that, none of the Application monitors were polling. Which is what prompted me to check the performance counters and I found the Job Engine V2 Jobs queue way out of wack.
I'll open up a case and get to the bottom of this. Thanks for the help. I'll post a root cause if or when we find one.
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process.