rs_malloc_tracker 1.0.0

Wraps LibC allocation calls to expose Prometheus memory statistics.
Documentation

rs-malloc-tracker

This tool is meant to solve a specific use-case: Troubleshooting unexplained memory usage in a C application (namely asterisk).

Ideally, you'd make every memory allocation go through a custom allocator that does proper tracking for your use-case. In my case though, I had a couple problems:

  1. Shared libraries obviously don't follow this pattern
  2. The memory that I was tracking through my custom allocator did not add up to the observed RSS of my application. Not even close: Screnshot of Asterisk RSS going up to around 1.25 GiB Screnshot of ast_malloc() instrumentation going up to around 100 MiB

From these observations, it was clear that a lower-level instrumentation was required to properly shine light on what was going on. Hence, this tool.

What this does

  • Use the LD_PRELOAD trick to:
    • Wrap malloc(3), free(3) and related functions to tabulate memory usage
    • Wrap mmap(3) and munmap(3) to tabulate VSS and RSS separately
  • Expose the results in prometheus format in a per-process UNIX socket

The end result looks like this:

Screenshot of rs-malloc-tracker showing malloc() and smaps instrumentation

This allowed me to understand what my problem was: https://github.com/facebook/jemalloc/issues/46