Skip to content

Memory leak: .debug() retains ~140KB of native memory per call #439

Description

@velo

Summary

Calling .debug() on a command retains roughly 140KB of native memory per invocation. The memory
is never released — not by GC, and not by the Go runtime's scavenger after minutes of idle time —
so a long-running JVM process that makes repeated helm calls grows without bound until the OS or a
container memory limit kills it.

Without .debug() the same loop is flat.

The growth is entirely outside the JVM heap, so -XX:+ExitOnOutOfMemoryError never fires and a
heap dump shows nothing. In a memory-limited container it surfaces only as an OOM kill.

Reproducer

Self-contained — it scaffolds its own chart via Helm.create(), so there is no external input.

import com.marcnuri.helm.Helm;
import java.nio.file.Files;
import java.nio.file.Path;

/** Args: iterations debug(true|false) idleSeconds */
public class DebugLeak {

  public static void main(String[] args) throws Exception {
    int iterations = Integer.parseInt(args[0]);
    boolean debug = Boolean.parseBoolean(args[1]);
    int idleSeconds = args.length > 2 ? Integer.parseInt(args[2]) : 0;

    Path dir = Files.createTempDirectory("helm-java-leak");
    Helm chart = Helm.create().withName("demo").withDir(dir).call();

    System.out.printf("iterations=%d debug=%s%n", iterations, debug);
    System.out.printf("start   rssKb=%d heapUsedKb=%d%n", rssKb(), heapUsedKb());

    for (int i = 1; i <= iterations; i++) {
      var command = chart.template();
      if (debug) {
        command = command.debug();
      }
      command.call();

      if (i % 500 == 0) {
        System.gc();
        System.out.printf("i=%-6d rssKb=%d heapUsedKb=%d%n", i, rssKb(), heapUsedKb());
      }
    }

    for (int elapsed = 30; elapsed <= idleSeconds; elapsed += 30) {
      Thread.sleep(30_000);
      System.gc();
      System.out.printf("idle %ds rssKb=%d heapUsedKb=%d%n", elapsed, rssKb(), heapUsedKb());
    }
  }

  private static long heapUsedKb() {
    Runtime r = Runtime.getRuntime();
    return (r.totalMemory() - r.freeMemory()) / 1024;
  }

  private static long rssKb() throws Exception {
    for (String line : Files.readAllLines(Path.of("/proc/self/status"))) {
      if (line.startsWith("VmRSS:")) {
        return Long.parseLong(line.replaceAll("\\D+", ""));
      }
    }
    return -1;
  }
}

Run with a small -Xmx so that any RSS growth is unambiguously outside the JVM heap:

java -Xmx256m -cp helm-java.jar:linux-amd64.jar:lib-api.jar:jna.jar:. DebugLeak 2000 true 120
java -Xmx256m -cp helm-java.jar:linux-amd64.jar:lib-api.jar:jna.jar:. DebugLeak 2000 false

Results

With .debug() — RSS climbs monotonically while the heap is flat:

iterations=2000 debug=true
start   rssKb=158292 heapUsedKb=32388
i=500    rssKb=259488 heapUsedKb=20854
i=1000   rssKb=299900 heapUsedKb=20283
i=1500   rssKb=366340 heapUsedKb=20251
i=2000   rssKb=430280 heapUsedKb=20236

~139KB per call, and the slope does not decay. Over a longer run against a larger chart I measured
the same behaviour out to 5000 calls (174MB to 877MB) with no plateau.

The memory is not simply waiting on the Go scavenger — two minutes of idle time return none of it:

idle 30s  rssKb=430308 heapUsedKb=19927
idle 60s  rssKb=427148 heapUsedKb=19909
idle 90s  rssKb=427164 heapUsedKb=19909
idle 120s rssKb=427168 heapUsedKb=19909

Without .debug() — bounded, and non-monotonic:

iterations=2000 debug=false
start   rssKb=158564 heapUsedKb=31876
i=500    rssKb=197164 heapUsedKb=20854
i=1000   rssKb=176392 heapUsedKb=20283
i=1500   rssKb=179236 heapUsedKb=20236
i=2000   rssKb=181168 heapUsedKb=20236

Notes

  • Reproduced on template. The same flag is available on install/upgrade/uninstall, and I
    would expect the cause to be shared, though I only measured template.
  • For template specifically, .debug() does not change the returned output at all, so the
    retained memory buys nothing on that path.
  • Workaround: invoke the helm binary as a subprocess, where the memory is reclaimed on exit.

Environment

  • helm-java 0.0.21, linux-amd64 native
  • OpenJDK 25.0.2 (Temurin)
  • Linux x86_64

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions