This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Data Collection Issue: java.lang.OutOfMemoryError

We've been running Profiler for the last year and upgraded to version 5 a few months back without any issues. Recently though we have started to get collection issues for our Virtual environments.

Initially we had all polling done by the profiler server but we have since added an additional poller. We are now getting the error message of:

Data Collection Issue : java.lang.OutOfMemoryError: GC overhead limit exceeded

I thought this may be due to too many VMs being monitored so I have removed the test and DR ESX environments from the poller and just left our production cluster being monitored by the poller but we are still getting this error message on some of teh ESX servers.

Anyone know what this means or how to resolve it?

 

Thanks in advance!

    • If you see the java.lang.OutOfMemoryError, increase the maximum memory setting.

       

      • Windows:
        1. Stop the ProfilerAgent service.
        2. Make a backup copy of the ProfilerAgent.ini.
        3. Change the following parameter in the ProfilerAgent.ini from:
          param01 = -Xmx67108864 > param01 = -Xmx268435456
        4. Restart the Profiler Agent.
      • Unix:
        1. Stop the Agent Service by /etc/init.d/pragent stop.
        2. Open /etc/init.d/pragent
        3. Change the following line from:
          EXT_ARGS="-Xrs -Xms16777216 -Xmx134217728 -Djava.awt.headless=true -Djsnmp.ignoreV1V2PduSizeLimit=true -Djava.net.preferIPv4Stack=true"

          to:


          EXT_ARGS="-Xrs -Xms16777216 -Xmx268435456 -Djava.awt.headless=true -Djsnmp.ignoreV1V2PduSizeLimit=true -Djava.net.preferIPv4Stack=true"
        4. Save the pragent start script.
        5. Start the agent by /etc/init.d/pragent start.

          Note: The memory can be increased from the default 64MB to:
          512MB - 536870912 (512*1024*1024)
          256MB - 268435456 (256*1024*1024)
          128MB - 134217728 (128*1024*1024)

      Also you may check the memory setting for tomcat.exe in Task Manager at startup. If tomcat.exe memory spikes considerably, increase memory (to 512MB).

      1. Open a command prompt at E:\program files\tek-tools, inc\profiler server\bin
      • Issue the command net stop profilerwebserver
      • Issue the command uninstall_tomcat.bat
      • Issue the command notepad rm_service.bat and make the following change (notice -Xmx512m; prefixed to -Dcatalina....)
        rem Set extra parameters
        "%EXECUTABLE%" //US//%SERVICE_NAME% --JvmOptions "-Xmx512m;-Dcatalina.base=%CATALINA_BASE%;-Dcatalina.home=%CATALINA_HOME%;-Djava.endorsed.dirs=%CATALINA_HOME%\common\endorsed;-Ddbcp" --StartMode jvm --StopMode jvm
        rem More extra parameters
        set PR_STDOUTPUT=%CATALINA_HOME%\logs\stdout.log


      • Issue the command install_tomcat.bat
      • Issue the command net start profilerwebserver
  • Actually, this error stems from the Garbag Collector component of the java memory.

    Stop the agent service and edit the Storage Manager Agent.ini file (may be named SolarWinds.Storage.Agent) found in <install dir>\Storage Manager Agent\

    Add the following entry after "EXT_ARGS=":

    -XX:-ReduceInitialCardMarks

    So that it look like this:

    EXT_ARGS=-XX:-ReduceInitialCardMarks -Xrs -Xms67108864 -Xmx1024m -Djsnmp....

    Save the file and restart the agent.  

    If you continue to see this error, please open a support ticket at www.solarwinds.com/support/ticket/

  • Thanks Garrett.

    This looks more like my config as I don't have the Profileragent.ini file on my server.

    I'll try these changes over teh weekend and let you know how this goes. :)