Tuning JVM Performance with OpenTelemetry Metrics

Tuning the Java Virtual Machine (JVM) is essential for squeezing the best performance out of your applications. It helps reduce latency, optimize memory usage, and better use your hardware. But to tune the JVM effectively, you need to know what’s happening inside it, especially around garbage collection, memory usage, and threads.

That’s where OpenTelemetry (OTel) comes in. As a vendor-neutral observability framework, OpenTelemetry gives you direct visibility into the internals of your application, including the JVM. You can collect and export runtime metrics with minimal overhead and use them to pinpoint performance issues.

In this post, we’ll show you how to use OpenTelemetry metrics to monitor garbage collection, memory usage, and thread performance in your Java application. Plus, we’ll explain how to export those metrics to SolarWinds Observability SaaS for easy visualization and alerting.

Initial Setup

For this walkthrough, we’re using a sample Java application that simulates JVM load through a thread pool processing memory-intensive tasks. The demo app creates memory and CPU load by processing tasks with random memory allocation (100KB–1MB) and execution time (50ms–150ms) across four worker threads, making it ideal for surfacing real JVM behavior under variable load, such as garbage collection activity and thread utilization. You can try it yourself using the demo in this GitHub repository.

Before we get started, here’s what you’ll need to follow along:

Java 11 or later: The SolarWinds agent supports Java 8 through Java 21 (check the official documentation for the latest version support).
A Java application running on the JVM: This can be any Java app you’re currently running or testing, or you can try it out with the demo used in this guide.
A SolarWinds Observability SaaS account: You’ll be sending metrics from your app to this platform.

Familiarity with JVM startup options and environment variables: You’ll need to modify startup flags to enable the agent and set configuration.

Get SolarWinds Observability SaaS OTel Endpoint and API Token

As mentioned above, we will instrument our application with OTel and send our metrics to SolarWinds Observability SaaS. To do this, you will need to find your OTel endpoint and create an API token for data ingestion. Follow these steps.

https://my.na-01.cloud.solarwinds.com/

The xx-yy part of the URL (in the above example, that's na-01) indicates the data center for your organization. You will need this when determining your OTel endpoint.

Navigate to Settings > API Tokens.

<img style="max-height:615px;max-width:820px;" alt=" " src="https://us.v-cdn.net/6038570/uploads/communityserver-blogs-components-weblogfiles/00/00/00/00/69/4743.pastedimage1749662642476v1.png" />

On the API Tokens page, click Create API Token.

Specify a name for your new API token. Select Ingestion as the token type. You can also add tags if you wish. Click Create API Token.

Copy the resulting token value.

Configure the Application for OpenTelemetry Instrumentation

Before instrumenting your application, you will need to add the appropriate configurations and dependencies.

Add OpenTelemetry Dependencies

First, edit pom.xml to add the necessary dependencies:

<dependency>   <groupId>io.opentelemetry</groupId>   <artifactId>opentelemetry-api</artifactId>   <version>1.26.0</version> </dependency> <dependency>   <groupId>io.opentelemetry</groupId>   <artifactId>opentelemetry-sdk</artifactId>   <version>1.26.0</version> </dependency> <dependency>   <groupId>io.opentelemetry</groupId>   <artifactId>opentelemetry-exporter-otlp</artifactId>   <version>1.26.0</version> </dependency> <dependency>   <groupId>io.opentelemetry.instrumentation</groupId>   <artifactId>opentelemetry-instrumentation-api</artifactId>   <version>1.26.0</version> </dependency> <dependency>   <groupId>io.opentelemetry.instrumentation</groupId>   <artifactId>opentelemetry-instrumentation-api-semconv</artifactId>   <version>1.26.0-alpha</version> </dependency> <dependency>   <groupId>io.opentelemetry.instrumentation</groupId>   <artifactId>opentelemetry-runtime-metrics</artifactId>   <version>1.26.0-alpha</version> </dependency>

</code><h3 id="mcetoc_1itg1gn9b4"><a id="Configure_OpenTelemetry_to_Export_Data_to_SolarWinds_Observability_SaaS" name="Configure_OpenTelemetry_to_Export_Data_to_SolarWinds_Observability_SaaS"></a>Configure OpenTelemetry to Export Data to SolarWinds Observability SaaS</h3>Next, create a configuration class to set up OTel with authentication and the endpoint for SolarWinds Observability SaaS.<code>// src/main/java/com/solarwinds/jvmmonitoring/OpenTelemetryConfig.java

package com.solarwinds.jvmmonitoring;

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.semconv.resource.attributes.ResourceAttributes;
import io.opentelemetry.instrumentation.runtimemetrics.BufferPools;
import io.opentelemetry.instrumentation.runtimemetrics.Classes;
import io.opentelemetry.instrumentation.runtimemetrics.Cpu;
import io.opentelemetry.instrumentation.runtimemetrics.GarbageCollector;
import io.opentelemetry.instrumentation.runtimemetrics.MemoryPools;
import io.opentelemetry.instrumentation.runtimemetrics.Threads;

public class OpenTelemetryConfig {
private static final String SERVICE_NAME = "jvm-monitoring-demo";
private static final String ENDPOINT = System.getenv().getOrDefault("OTLP_ENDPOINT", ""); // Blank default
private static final String API_TOKEN = System.getenv().getOrDefault("OTLP_INGESTION_TOKEN", ""); // Blank default

public static OpenTelemetry initialize() {
// CONFIGURE THE OTLP METRIC EXPORTER
OtlpGrpcMetricExporter metricExporter = OtlpGrpcMetricExporter.builder()
.setEndpoint(ENDPOINT)
.addHeader("authorization", "Bearer " + API_TOKEN)
.build();

// SET UP THE SDK WITH THE EXPORTER
Resource resource = Resource.getDefault()
.merge(Resource.create(Attributes.of(
ResourceAttributes.SERVICE_NAME, SERVICE_NAME,
ResourceAttributes.SERVICE_VERSION, "1.0.0"
)));

SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder()
.setResource(resource)
.registerMetricReader(PeriodicMetricReader.builder(metricExporter).build())
.build();

// INITIALIZE THE OPENTELEMETRY SDK WITH JUST METRICS
OpenTelemetrySdk openTelemetry = OpenTelemetrySdk.builder()
.setMeterProvider(sdkMeterProvider)
.build();

// REGISTER RUNTIME METRICS
BufferPools.registerObservers(openTelemetry);
Classes.registerObservers(openTelemetry);
Cpu.registerObservers(openTelemetry);
GarbageCollector.registerObservers(openTelemetry);
MemoryPools.registerObservers(openTelemetry);
Threads.registerObservers(openTelemetry);

return openTelemetry;
}
}

Notice that this configuration expects you to set two environment variables at runtime. These are related to your SolarWinds Observability SaaS endpoint.

OTLP_ENDPOINT is the HTTP endpoint URL for OTLP ingestion with SolarWinds Observability SaaS. It will look like https://otel.collector.xx-yy.cloud.solarwinds.com:443, but you should replace xx-yy with your specific data center code (such as na-01).
OTLP_INGESTION_TOKEN is the value of the API token created in the previous step.

Instrumenting the JVM

With your application configured to send OTel data to SolarWinds Observability SaaS, you can instrument it to capture important metrics. The updated TaskProcessingSystem class looks like this:

package com.solarwinds.jvmmonitoring;

// IMPORT OTEL-RELATED DEPENDENCIES
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.metrics.LongCounter;
import io.opentelemetry.api.metrics.Meter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;

public class TaskProcessingSystem {
 private static final Logger logger = 
 LoggerFactory.getLogger(TaskProcessingSystem.class);
 private static final int NUM_THREADS = 4;
 private static final int SHUTDOWN_TIMEOUT_SECONDS = 15;
 private static final AtomicBoolean isShuttingDown = 
 new AtomicBoolean(false);
 private static final int DEFAULT_RUNTIME_SECONDS = 20;

// OTEL COMPONENTS
 private static OpenTelemetry openTelemetry;
 private static Meter meter;
 private static LongCounter tasksProcessedCounter;
 private static LongCounter taskProcessingTimeCounter;

public static void main(String[] args) {
 // INITIALIZE OTEL
 openTelemetry = OpenTelemetryConfig.initialize();
 meter = openTelemetry.getMeter("com.solarwinds.jvmmonitoring");

 // CREATE METRICS
 tasksProcessedCounter = meter.counterBuilder("tasks.processed")
 .setDescription("Number of tasks processed")
 .setUnit("1")
 .build();

 taskProcessingTimeCounter =
 meter.counterBuilder("tasks.processing.time")
 .setDescription("Total time spent processing tasks")
 .setUnit("ms")
 .build();

// Parse runtime duration from command line args
 final int runtimeSeconds = parseRuntimeDuration(args);
 logger.info("Starting Task Processing System with {} threads, runtime: {} seconds", NUM_THREADS, runtimeSeconds);

 // Create a thread pool with a bounded queue
 ThreadPoolExecutor executor = new ThreadPoolExecutor(
 NUM_THREADS, NUM_THREADS,
 0L, TimeUnit.MILLISECONDS,
 new LinkedBlockingQueue<>(1000),
 new ThreadPoolExecutor.CallerRunsPolicy()
 );

// Add shutdown hook for graceful termination
 Runtime.getRuntime().addShutdownHook(new Thread(() -> {
 logger.info("Shutdown signal received, initiating graceful shutdown...");
 shutdownGracefully(executor);
 }));

// Add timer for automatic shutdown after specified runtime
 new Thread(() -> {
 try {
 Thread.sleep(runtimeSeconds * 1000);
 logger.info("Runtime duration of {} seconds reached, initiating shutdown...", runtimeSeconds);
 System.exit(0);
 } catch (InterruptedException e) {
 Thread.currentThread().interrupt();
 }
 }, "shutdown-timer").start();

// Create task processor
 TaskProcessor processor = new TaskProcessor();

try {
 // Submit tasks to the executor
 while (!isShuttingDown.get()) {
 executor.submit(() -> {
 try {
 long startTime = System.currentTimeMillis();
 processor.processNextTask();
 long processingTime = System.currentTimeMillis() - startTime;

 // RECORD METRICS
 tasksProcessedCounter.add(1);
 taskProcessingTimeCounter.add(processingTime);
 } catch (InterruptedException e) {
 Thread.currentThread().interrupt();
 logger.debug("Task processing interrupted");
 } catch (Exception e) {
 logger.error("Failed to process task: {}", e.getMessage());
 }
 });

 // Small delay to prevent overwhelming the system
 Thread.sleep(10);
 }
 } catch (InterruptedException e) {
 logger.info("Main thread interrupted, initiating shutdown...");
 Thread.currentThread().interrupt();
 } finally {
 shutdownGracefully(executor);
 }
 }

…
}

Now, you can run the application for 60 seconds and then see the telemetry metric data sent to SolarWinds Observability SaaS. Run the following commands in a terminal:

$ mvn clean package
$ export \
 OTLP_ENDPOINT=https://otel.collector.xx-yy.cloud.solarwinds.com:443
$ export \
 OTLP_INGESTION_TOKEN=replace-with-your-token
$ java -jar target/jvm-monitoring-demo-1.0-SNAPSHOT.jar 60Confirm Data Ingestion With SolarWinds Observability SaaS

After running your application several times, it's time to confirm that the application is sending telemetry data to SolarWinds Observability SaaS. To do this, log in to your account and navigate to Analyze > Metrics.

On the Metrics page, search for metrics related to JVM. This returns a substantial list of metrics captured by your application.

You can click on any individual metric to see a visualization. For example, <a style="background-color:#fff0cf;" href="https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmgcduration">jvm.gc.duration</a> shows the duration of JVM garbage collection actions.

Or, you can look at <a style="background-color:#fff0cf;" href="https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/#metric-jvmsystemcpuutilization">jvm.cpu.utilization</a> , which shows the CPU utilization for the system as reported by the JVM.

With data now flowing into SolarWinds Observability SaaS, you can view, graph, and alert on key JVM signals.

Create Custom Dashboards

You can create custom dashboards to visualize multiple metrics side by side or overlaid on top of one another to get more value for your telemetry data. On the Dashboards page, click Create Dashboard. Select the Standard dashboard type and click Next.

An empty dashboard will be created. Click Save. Specify a dashboard name, and then click Save again.

Navigate back to the Metrics Explorer. Find metrics that are directly related to your tuning and performance goals. Click on the three-dots menu for the metric. Then, click Send to Dashboard.

Choose your recently created dashboard from the dropdown list. Specify a name for the widget with your metric. Then, click Save.

Add more metrics to your dashboard, creating an all-in-one visualization to help you understand your JVM performance.

For metrics directly related to GC performance, compare GC action duration with thread counts and look at memory usage.

Analyzing and Tuning Your JVM

With metrics in hand, you can begin making targeted improvements. Here's how to diagnose issues and apply effective JVM tuning strategies.

Diagnose Performance Issues and Apply Targeted JVM Settings

Start by identifying common problem areas:

GC issues: Frequent or long GC pauses may indicate poor garbage collector choice or memory pressure.
Memory usage: Consistently high heap usage might indicate a memory leak or an undersized heap.
Thread behavior: High or growing thread counts could signal contention or thread pool misconfiguration.

Once you've identified a potential issue, you can apply targeted changes to JVM configuration. For example:

Adjust heap size.
Choose a better GC algorithm. Depending on your pause time and throughput needs, try alternatives like G1, ZGC, or Shenandoah.
Reconfigure thread pools, ensuring they aren’t oversized or leaking threads.

After applying changes, compare metrics before and after to validate improvements.

Put Your Telemetry to Work

With just a few configuration steps, you can capture rich JVM internals using OpenTelemetry and view them in SolarWinds Observability SaaS. This visibility helps you troubleshoot, tune, and optimize with confidence.

If you haven’t already, start by instrumenting a test Java app and explore the metrics in your SolarWinds Observability SaaS account. Check out the SolarWinds OTel integration guide for more details and options.

Find more posts tagged with

Comments

donrobert5

Nice Blog