Major issues with APM and SNMP

My APM instance seems to very unreliable, after a reboot of the Orion server, it runs perfectly for around two weeks and then ends up in a state where on any typical day nearly half of my monitored applications are sporadically down (HTTP monitors, SQL server process monitors via SNMP, etc.) for 5 minutes at a time (i.e. one scan fails, and then the next works perfectly) .. this has been the case for nearly a year now, and APM alerts have long been ignored as completly unreliable. NPM itself shows no particular issues and I've never seen a false positive yet.

I have tried adjusting the polls per second tuning to both the recommended setting and to half the reccomended setting, however neither seem to solve the issue permanently... the only thing which will solve the problem once it occurs is a complete restart of the Orion server.

Are there any APM-specific logs in which I can look to see the cause of the issue?Or any APM tuning parameters/config files I can adjust?

I am currently running around 136 APM component monitors (licensed APM250), but this issue is really hampering my extension of APM to other applications.

NPM version is 9.1 SP4

APM version is 2.0 SP2

Find more posts tagged with

SNMP

orion

apm

Accepted answers

All comments

FormerMember

Polls per seconod tuning won't impact APM. Have you considered upgrading to APM 2.5? We made some significant changes to how APM polls, so it may help.

TGhosh

I feel your pain. Ive done EVERY update and service pack since APM 2.0 came out.

I've gone through the WMI troubleshooting guide frontwards and backwards.

I've scoured these message boards day after day looking for a solution.

I've had to temporarily pause my Alert Engine for weeks now. Basically, my NOC is running with its pants down around its ankles.

I've worked with support on a couple of cases which have been left with open status for WEEKS now.

Sorry for the rant, but I've paid thousands of dollars for this so called Upgrade to Solarwinds' new products (APM2, NPM9, EOC)...for the most part, the slick interfaces have been a welcoming change... But what good is all that if the product cant accurately and reliabily be used as an Enterprise Monitoring System?!?

Once again, i'm sorry for the rant, but my job is on the line. I want a refund of my purchase. I am willing to repurchase the product at a sever discount some time in the future (once all the bugs have been worked out).

My management has given me carte blanche to bring in the best tools for my Operations center, please dont make me regret my choice to stay with Solarwinds!!

FormerMember

TGhosh,

I've looked at all of your cases. If you are getting alerts when an application goes "unknown", you might try reconfiguring the alert to fire only when an app is actually "down" rather than "not up".

You could also disable only those alerts that are causing problems. Disabling the whole engine seems like an extreme response.

Only one of your cases is still open, and Dev has looked at it. We've not been able to reproduce your issue so far. We know it's not a general bug because we have close to 1000 users who have installed APM 2.5 already and they're not seeing the same problem.

Your issue seems to be focused on WMI, and many times, the problem is specific to a customer's environment. Having been in this business for many years managing mulitple products, I can tell you that WMI is sometimes a challenge, irrespective of the network management system.

In any case, You should be receiving a contact from Support asking for more info. To go forward, we'll yneed more detailed information. We will continue to work with you, but some problems are hard to diagnose, so we appreciate your patience in helping us to resolve it.

TGhosh

Disabling specific alerts wasn't an option since the false reporting was varied across all the nodes in a particular APM template.

I've explained to support that my environment may be a very limited usage case since our SQL server DB's are running on 64 bit Windows Server 2008. From what i've been reading the Server 2008 platform has had significant changes to WMI.

I've made it clear with my support tech that I am willing to invite your developers into my Solarwinds server via screen-share so they can see first hand what the problem is. Unfortunately, the amount of noise being generated by APM2.5 right now is reducing my team's confidence level in the accuracy of the product.

I am open to any suggestions you may have to help mitigate these issues in my particular environment. Please feel free to contact me offline.

dbaggette

I am also experiencing the same issue and have a ticket open with support that has been escalated to the developers. Our WMI APM templates continuously go to an unknown state after upgrading to APM 2.5. We never saw this condition in 2.0. Interestingly enough, my DB's are also running on 64 bit server 2008.

FormerMember

I've made it clear with my support tech that I am willing to invite your developers into my Solarwinds server via screen-share so they can see first hand what the problem is. Unfortunately, the amount of noise being generated by APM2.5 right now is reducing my team's confidence level in the accuracy of the product.
I am open to any suggestions you may have to help mitigate these issues in my particular environment. Please feel free to contact me offline.

I looked at your case, and it was suggested that you extend your timeouts so that we can see what would happen. We're still waiting on a response. Without that, your case is stalled. Please respond to your ticket, not here on Thwack. If you respond here, I can't update your ticket.