Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Too much DATA and how do we make APM do something simple - back to basics

Hi all, I have a fairly big Orion install, but i made the mistake of turning it into a network and application system by adding the APM module to all of my 5 pollers.... big mistake... the APM has taken over the thing and destroyed database performance. This is due to a couple of major feature flaws in the product that I am hoping can be looked at by solarwinds (and i will mention below). However, in the short term, can someone help me purge the data that is collecting in my APM tables... specifically: APM_ProcessEvidence_HourlyAPM_ProcessEvidence_Detail Can I just empty these tables with the following commands in database manager “Truncate Table [APM_ProcessEvidence_Hourly]” And “Truncate Table [APM_ProcessEvidence_Detail]” Vv These 2 tables are very large, even though i have daily maintenance plans, and I have process detail retentions et to 1 day, and hourly set to 7 days. Only where the problems stem from... I want to use APM as a simple addition to the availability monitoring solution on my network, I do not actually want to know anything about performance of my processes or services. (The reason or this, is that I do most of my monitoring remotely through firewalls and the likes, so i am limited by SNMP, so i cannot do any clever performance counters using WMI until Microsoft sort out their port allocations.) Within the current APM, i select a process-via-SNMP, create a template, and assign it to a server. If that server has multiple instances of that process, APM will collect data about every instance of that process, even if maybe there are 50 instances running (e.g. cron on a unix server). All i actually want to know, is whether there is any instance of it at all, or maybe just a quick count of the number of processes and record that. I do not want to collect ANY performance stats into the database, and do not even want APM to pull them back. I think if there were a way to select whether an assigned APM template is for availability of a process OR application performance management, then it would solve just about all of my issues. I think i would then have a very smooth running highly performing Orion environment. But as it stands, APM is bringing my environment to its knees, and affecting the core network monitoring. Please comments from any would be great, and a quick solution to purging the APM tables above would be super!!!! Happy new year!!!

Find more posts tagged with

apm

Accepted answers

jiri.tomek

Hello,
you can try one last thing.

DELETE FROM [APM_ProcessEvidence_Detail] WHERE Name = 'YOUR_CRON_PROCESS_NAME'

This will delete all records for cron processes. I've just tried it on my machine and availability data for affected components are still available so I'm able to generate reports and I can see availability charts. I'm just not able to see performance graphs.

You can try this approach. Backup your database, run this query and check if you have all required reports and data available. If yes, you can run this query periodically to keep table clean. However be careful, this is not official and supported way and you should know that some problems may occur with APM after this. I would try this for a while on some test machine first.

I put this solution here just for your specific scenario and can't take any responsibility for it as it is not tested enough on our side.

All comments

jiri.tomek

Hello,

If you want to know whether specific process is running, APM has to pull some data from target machine and store them in database. If you really want just to know if process is running or not, you can purge [APM_ProcessEvidence_Detail] and [APM_ComponentStatus_Detail]. Then all you get is status of process (up/down/warning/...).

But remember that this is not supported scenario and there may be some errors in web console, missing charts and so on. You should make backup of your data before purging. Though it should give you data you need.

Even if you purge those tables, APM still have to collect data and store them in database. You can try to repeatedly run some SQL script that purges those tables periodically but data collection will be still there.

nicksmith

Hey

These are exactly the issues i have been having with APM which I have deployed to a number of our customers - thanks for providing this post.

It seems APM under-performs on large networks or complex environments.

Will be keeping an eye on this post.

FormerMember

thanks for the reply..

could I please log a feature request, to enable a user to select if one wants to pull back the information on application performance, or whether they just want availability data? it should not be too difficult to implement, and it would dramatically improve the entire performance of my Orion implementation. There must be something under the hood that can be done.

I want to know if cron is running on 100 servers. this means i have an application template assigned to these 100 servers, that just has the process "cron" via SNMP within it. However, within the database, and within Orion, each of those servers has 20+ cron processes, so I am constantly writing useless performance DATA for 2000 processes, which is killing the database due to the way Orion writes polling info tot he DB

Please help me find a performance related work around this, to help me sort this out as it is really killing my environment... i currently have:

Orion SLX
4 additional pollers (more to come hopefully)
APM ALX, plus modules on ALL pollers
IP SLA module (SLAX)
NTA – unlimited
Eng toolset
NCM DL3000 (possibly going to rise)

this environment is on its knees, and i truly believe the main cause is APM......

Can you also confirm if i have the purge syntax correct? and if i have to stop all pollers to do this purge? and is there any linked data elsewhere int he database that might be affected, or is it all in this table?

cheers

dan

MarieB

HI Dan--

I'll make sure that the product manager sees this and logs it as a feature request.

jiri.tomek

Hello Dan,
regarding your posts at , one thing you can do right now is to write Linux script (I suppose it should run on Solaris) that checks if there is at least one cron instance and returns this as statistic. This way you will get "only" as many records as many servers you have. This can help your database a lot.

You can also try SQL purge queries mentioned above. “Truncate Table [APM_ProcessEvidence_Detail]” should be enough because [APM_ProcessEvidence_Hourly] table will be empty if you run this query more often than once per week (data retention period for Detail->Hourly). Deleting data from [APM_ProcessEvidence_Detail] can affect charts and will definitely affect alerts but these "inconsistency" states should last only until next poll when new evidence is generated.

Data from this table are not linked to any non-APM data (it won't affect your NPM), data are linked to records in [APM_ComponentStatus_Detail] and to some views. Stopping pollers is not required, truncate command is atomic and locks table so there should be no collision with possible writes.

You said you want option to not collect data from polls. This is possible to develop but then you would have no historical data for those components. You would have only current status available. Then if a component goes to DOWN status and then after some time back to UP, you wouldn't notice until you set an alert on current status. Is this the way how you would like it?

jiri.tomek

Hello Dan,
I suggested my workaround as a temporary solution. You are not able to force APM not to store data right now but you can at least purge that data once they are in DB. I know that it won't solve "data writing" performance issue but it can keep table small so operations with it are faster. As I noticed, our PM have taken your request as a feature request so maybe it will occur in some future release. But with current APM you can't get rid of evidence writes. Sorry.

FormerMember

I have now marked it back as the answer, you are completely correct... I hope the PM agrees to include this, as it will drastically increase APM scalability for me...