Expand description
Always-on stack telemetry for Rust, with nothing to instrument.
stackler answers the questions a program cannot normally answer about its
own stacks: how deep is this thread right now, how deep has it ever been,
how much of the stack space it reserves does it actually use, and how close is
it to running out. It answers them at runtime, in production, without a
debugger, without a rebuild, and — this is the point — without touching the
code being measured.
§why stackler?
Stack use is normally observed either not at all or all at once: a crash. The
tools in between are heavyweight — a debugger, a sampling profiler with
symbolication, RUST_MIN_STACK guesswork after the fact. stackler occupies
a lighter tier: a handful of numbers per thread, cheap enough to leave on.
- nothing to annotate: no macros on functions, no wrappers around recursion, no rebuild of the code under observation. Depth is read from the stack pointer, and peaks are read from the stack itself.
- free while running: with the
watermarkfeature the measured code pays nothing at all — the unused stack is filled with a marker and later inspected to see how far it was overwritten. All the cost is at the painting and the scan. - contention-free: every observation writes only to the recording thread’s own cache-aligned slot — depth, peak, counters and depth histogram alike. Nothing on that path is shared, so concurrent sampling scales with the cores doing it; the aggregates are computed when a snapshot is taken.
- overflow guard:
has_headroomlets input-driven recursion return an error instead of dying on a guard page, and counts how often it came close. - reserved vs. used:
Stats::utilizationandStats::wasteput a number on the address space a thread pool spends on stacks nobody descends into. no_stdfriendly: theStackprimitives — the same painting and scanning an RTOS uses to report a task’s high-water mark — need neitherstdnor an allocator.
§quickstart
// In `main`: configures the crate and registers this thread. Optional — any
// call into `stackler` attaches the calling thread on its own.
stackler::Stackler::new().install();
// ... do some deeply nested work ...
let stats = stackler::stats();
println!("deepest stack use so far: {} bytes", stats.peak_max);§measuring specific operations
With the watermark feature, the stack cost of an operation can be measured
exactly the way a heap profiler measures allocations — which makes it an
assertion, and an assertion makes it a regression test:
let (result, peak) = stackler::measure_peak(|| parse("[[[[nested]]]]"));
let peak = peak.expect("this thread's stack is known");
assert!(peak.bytes() < 64 * 1024, "regression: the parser got too deep!");§guarding recursion
Code whose depth depends on its input can ask whether it can afford to go deeper, and turn what would have been a fatal overflow into an ordinary error:
fn walk(node: &Node) -> Result<(), TooDeep> {
if !stackler::has_headroom(32 * 1024) {
return Err(TooDeep);
}
for child in &node.children {
walk(child)?;
}
Ok(())
}§how much is automatic
The features form a ladder, and each rung asks less of the program:
| build | what the program does | what it gets |
|---|---|---|
| default | calls sample wherever it has a natural tick | depth distribution, per-thread peaks, guard |
watermark | calls Stackler::install once | true high-water marks, for code that never calls in |
sampler | keeps a Sampler alive | those marks kept fresh for every thread, continuously |
hook_pthread | nothing | every thread in the process, including those it did not create |
§accuracy
Depth and headroom are exact on architectures whose stack pointer can be read directly, which is all of the mainstream ones; elsewhere they are off by at most one frame.
Watermarks are approximate, and always in the same direction — they can
understate depth, never overstate it. See Peak for the specifics: the
resolution is a few hundred bytes, a frame that reserves stack without writing
to all of it leaves part of the marker intact, and a peak deeper than the
painted region can only be reported as a lower bound.
§caveats
- Painting commits the memory it touches, so a painted thread’s resident set
grows by up to
Stackler::paint_depthbytes. That is the entire cost model of the technique, and the reason the depth is a knob. - Only the thread that owns a stack may paint it. Anyone may read a
watermark, which is what makes the background
samplerpossible, but such a read races with the owner by construction: the values it sees may be torn. It stays sound in practice because the only question asked of each word is whether the marker is still there, and a torn word is a written word. - On Linux, painting the main thread’s stack relies on the kernel growing the main stack mapping on demand. This is how every kernel since 4.13 behaves; on much older ones, prefer to leave the main thread unpainted.
- The registry tracks
MAX_THREADSthreads at a time. Threads beyond that are counted inStats::untrackedrather than silently dropped. - The first call on a thread is not reentrant: it asks the OS for the thread’s
bounds, and on glibc’s main thread that reads
/proc/self/mapsand allocates. Callingstacklerfrom inside a global allocator is therefore a bad idea unless the thread has beenattached beforehand.
Modules§
- sampler
- An optional background thread that keeps the aggregates fresh.
Structs§
- Histogram
- A histogram with power-of-two buckets containing the numbers of observations of different sizes, starting with 2^0 and ending with the [2^63, 2^64) bucket.
- Paint
- A painted region of a stack, produced by
Stack::paint. - Stack
- The address range of a single thread’s stack.
- Stackler
- Configures process-wide stack telemetry.
- Stats
- A summary of the stack use of a whole process.
- Thread
Stats - Everything known about a single tracked thread.
Enums§
- Peak
- A high-water mark measurement.
Constants§
- DEFAULT_
PAINT_ DEPTH - The number of bytes
crate::attachpaints below the stack pointer unless configured otherwise. - DEFAULT_
RESERVE - The headroom below which an observation is considered to be near the limit, unless configured otherwise.
- MAX_
THREADS - The number of threads that can be tracked at the same time.
Functions§
- attach
- Attaches the calling thread to the registry, returning
trueif it is tracked from now on. - current_
stack - Returns the calling thread’s stack, as recorded when it attached.
- depth
- Returns how many bytes of stack the calling thread is using right now, without recording anything.
- for_
each_ thread - Calls
fwith a snapshot of every tracked thread. - has_
headroom - Returns
trueif the calling thread can afford to usebytesmore stack. - headroom
- Returns how many bytes of stack the calling thread has left before it overflows.
- is_
attached - Returns
trueif the calling thread is already tracked. - measure
- Measures the stack telemetry of an operation, returning its result alongside
the
Statsdelta. - measure_
peak - Measures how much stack an operation uses, returning its result alongside the deepest point it reached.
- peak
- Returns the deepest point the calling thread has reached since it was painted, and folds the reading into its recorded peak.
- poll_
all - Re-reads the watermark of every tracked thread, folding what it finds into their recorded peaks.
- repaint
- Folds the current watermark into the recorded peak and paints the stack afresh, starting a new observation window.
- reset
- Clears the counters, and lowers every thread’s recorded peak to the last depth observed for it.
- sample
- Records the calling thread’s current depth and returns it.
- sp
- Returns the current stack pointer of the calling thread.
- stats
- Returns a summary of the stack use of the whole process.
- threads
- Returns a snapshot of every tracked thread.