stackler 0.1.0

A lightweight, always-on stack telemetry library that requires no instrumentation.
Documentation
# ๐Ÿ“ stackler

[![crates.io](https://img.shields.io/crates/v/stackler)](https://crates.io/crates/stackler)
[![docs.rs](https://docs.rs/stackler/badge.svg)](https://docs.rs/stackler)

**Always-on stack telemetry for Rust, with nothing to instrument.**

`stackler` answers the questions a program cannot normally answer about its own stacks: how deep is this thread right now, how deep has it *ever* been, how much of the stack space it reserves does it actually use, and how close is it to running out.

It answers them at runtime, in production, without a debugger, without a rebuild, and โ€” this is the point โ€” without touching the code being measured.

### Why Stackler?

Stack use is normally observed either not at all or all at once: a crash. The tools in between are heavyweight โ€” a debugger, a sampling profiler with symbolication, `RUST_MIN_STACK` guesswork after the fact. Stackler occupies a lighter tier: a handful of numbers per thread, cheap enough to leave on.

- **Nothing to annotate**: no macros on functions, no wrappers around recursion, no rebuild of the code under observation. Depth is read from the stack pointer, and peaks are read from the stack itself.
- **Free while running**: with the `watermark` feature the measured code pays *nothing at all* โ€” the unused stack is filled with a marker, and later inspected to see how far it was overwritten. All of the cost is at the painting and the scan.
- **Contention-free**: every observation writes only to the recording thread's own cache-aligned slot โ€” depth, peak, counters and depth histogram alike. Nothing on that path is shared, so concurrent sampling scales with the cores doing it.
- **Overflow guard**: `has_headroom` lets input-driven recursion return an error instead of dying on a guard page, and counts how often it came close.
- **Reserved vs. used**: `Stats::utilization` and `Stats::waste` put a number on the address space a thread pool spends on stacks nobody descends into.
- **`no_std` friendly**: the `Stack` primitives โ€” the same painting and scanning an RTOS uses to report a task's high-water mark โ€” need neither `std` nor an allocator.

### Quickstart

Add `stackler` to your `Cargo.toml`.

```toml
[dependencies]
stackler = { version = "0.X", features = ["watermark"] }
```

Then, in `main`:

```rust
fn main() {
    // Optional: any call into stackler attaches the calling thread on its own.
    // Doing it explicitly decides when the cost is paid, and with what settings.
    stackler::Stackler::new().install();

    // ... do some deeply nested work ...

    println!("{}", stackler::stats());
}
```

### Use Cases

**1. Regression tests on stack depth**

Stop guessing whether a refactor made a recursive descent parser twice as deep. `measure_peak` paints the stack, runs the operation, and reads the mark it left.

```rust
let (result, peak) = stackler::measure_peak(|| parse(input));
assert!(peak.unwrap().bytes() < 64 * 1024, "Regression: the parser got too deep!");
```

**2. Right-sizing thread stacks**

A pool of 200 threads at the 8 MiB default reserves 1.6 GiB of address space. `Stats::utilization` says how much of it was ever touched โ€” usually a fraction of a percent, which is a `stack_size` argument waiting to be written, or a case for more threads on the same budget.

**3. Turning stack overflows into errors**

A stack overflow is a `SIGSEGV`: no unwinding, no `catch_unwind`, no error message. Code whose depth depends on its input can ask first, and the times it came close show up in telemetry even when nothing goes wrong.

```rust
fn walk(node: &Node) -> Result<(), TooDeep> {
    if !stackler::has_headroom(32 * 1024) {
        return Err(TooDeep);
    }
    for child in &node.children {
        walk(child)?;
    }
    Ok(())
}
```

**4. Always-on production metrics**

`stats()` returns a plain struct that is straightforward to wire into a Prometheus endpoint, especially with `serde` enabled. Nothing in it is maintained on a hot path: the aggregates are computed when the snapshot is taken.

### How Much Is Automatic

The features form a ladder, and each rung asks less of the program:

| Build | What the program does | What it gets |
| :--- | :--- | :--- |
| **baseline** (default) | calls `sample()` wherever it has a natural tick | depth distribution, per-thread peaks, guard |
| `watermark` | calls `Stackler::install()` once | true high-water marks, including for code that never calls in |
| `sampler` | keeps a `Sampler` alive | those marks kept fresh for every thread, continuously |
| `hook_pthread` | nothing | every thread in the process, including the ones it did not create |

### Feature Guide

* **`baseline` (default features):** stack bounds discovery, `depth()`, `headroom()`, the `has_headroom()` guard, and `sample()`-driven peaks and averages. All of it is a thread-local lookup and a few relaxed atomics on the calling thread's own cache lines.
* **`watermark`:** paints the unused stack on attach, so peaks can be read back later without the measured code participating. This is what makes `peak()`, `measure_peak()` and `poll_all()` possible. Costs one `memset` per thread and the resident memory to back it.
* **`histograms`:** a power-of-two distribution of observed depths, kept per thread and summed when a snapshot is taken. Costs one more relaxed atomic per observation, on the recording thread's own cache line, and makes `stats()` roughly three times as expensive.
* **`sampler`:** a background thread that re-reads every tracked thread's watermark on a period. Implies `watermark`.
* **`hook_pthread`:** defines `pthread_create`, so that every thread in the process attaches itself before it runs. Unix only โ€” inert elsewhere, so enabling every feature still builds everywhere. Process-wide by nature: for binaries, not for libraries. See the caveats below.
* **`fmt` / `serde`:** formatting and serialization only. No effect on any measurement path.
* **`full`:** everything except `hook_pthread`, which is a decision rather than a convenience.

### What It Costs

Orders of magnitude from `cargo bench` on the author's machine (x86-64, release). The point is the shape, not the digits.

| Operation | Cost | What it does |
| :--- | :--- | :--- |
| `sp()` | ~0.2 ns | one instruction |
| `depth()`, `headroom()` | ~2 ns | thread-local lookup, two relaxed loads, no writes |
| `has_headroom()` | ~1.8 ns | one load and a comparison; a counter moves only on denial |
| `sample()` | ~12 ns | the above plus four relaxed atomics on this thread's own cache lines |
| `sample()` with `histograms` | ~16 ns | plus one more, on the same thread's own bucket |
| `stats()` | ~130 ns | walks all 256 slots, whatever the thread count |
| `stats()` with `histograms` | ~450 ns | plus 64 bucket loads per occupied slot |
| painting | ~1.1 ยตs per 256 KiB | one `memset`, once per thread |
| scanning | ~6 ยตs per 256 KiB | one pass over the untouched part of the region |
| `measure_peak()` | ~13 ยตs | a scan, a repaint, and a scan, at the default depth |
| `poll_all()` | one scan per tracked thread | what the background sampler does per tick |

Sampling scales: sixteen threads calling `sample()` in a tight loop retire ~14ร— the work one thread does, with or without histograms. That is the point of keeping the counters per thread โ€” an earlier version shared the histogram buckets, and sixteen threads then achieved exactly the throughput of one.

The interesting knob is `paint_depth`: it bounds what a watermark can report *and* what a scan costs *and* how much memory painting commits. The default of 256 KiB is deep enough for most threads; when a peak comes back as `Peak::AtLeast`, that is the crate saying the region was too small to answer.

### Accuracy

Depth and headroom are exact on architectures whose stack pointer can be read directly (x86, x86-64, ARM, AArch64, RISC-V). Elsewhere `sp()` takes the address of a local in a frame of its own, just below the caller's, which reads a few dozen bytes low โ€” so depth is very slightly over-reported and headroom very slightly under-reported, which is the safe direction for a guard.

Watermarks are approximate, and always in the same direction โ€” they can understate depth, never overstate it:

- resolution is a few hundred bytes, because a region starts slightly below the stack pointer of whoever painted it;
- a frame that reserves stack without writing to all of it leaves the marker intact in the gap, and the deepest *written* word is what gets reported;
- a peak deeper than the painted region can only be reported as a lower bound, which `Peak::AtLeast` says out loud.

### Caveats

- Painting commits the memory it touches, so a painted thread's resident set grows by up to `paint_depth` bytes.
- Only the thread that owns a stack may paint it. Anyone may *read* a watermark, which is what makes the background sampler possible, but such a read races with the owner by construction: the values it sees may be torn. It stays sound in practice because the only question asked of each word is whether the marker is still there, and a torn word is a written word.
- A watermark describes what happens *after* the paint. A local in the frame that painted has already been reserved, which is why `measure_peak` calls the closure through a barrier the optimizer may not inline.
- On Linux, painting the main thread's stack relies on the kernel growing the main stack mapping on demand. This is how every kernel since 4.13 behaves; on much older ones, prefer to leave the main thread unpainted.
- `hook_pthread` needs a dynamically linked libc, resolves the real `pthread_create` with `dlsym(RTLD_NEXT, ..)`, and belongs in a binary: a library that enables it makes the decision for everyone who depends on it. A fully static link fails at link time with a duplicate symbol, which is a better outcome than failing at runtime.
- The registry tracks 256 threads at a time. Threads beyond that are counted in `Stats::untracked` rather than silently dropped.
- On glibc, the initial thread's low end is derived from `RLIMIT_STACK` rather than from a mapping, because that stack grows on demand. With an unlimited `RLIMIT_STACK` what comes back spans most of the address space, so Stackler caps it โ€” at the rlimit where there is one, and at 8 MiB where there is not. Depth and peaks are measured from the base and are unaffected; headroom can only be understated.
- `reset()` writes many counters belonging to many threads and is synchronized with none of them. A snapshot taken while one is in flight can be internally incoherent โ€” most visibly, `sample_avg` can exceed `peak_max`. Use `measure()` or `measure_peak()` to scope a measurement instead.
- The first call on a thread is not reentrant: it asks the OS for the thread's bounds, and on glibc's main thread that reads `/proc/self/maps` and allocates. Calling Stackler from inside a global allocator is therefore a bad idea unless the thread was attached beforehand.

### Sample Output

From `cargo run --release --example pool --features full`: six workers recursing to six different depths, none of them reporting a peak about itself.

```plaintext
threads_live: 7
threads_seen: 7

reserved_sum: 19.99 MiB
reserved_max: 7.99 MiB

peak_sum: 208.99 KiB
peak_avg: 29.86 KiB
peak_max: 56.42 KiB
utilization: 1.02%
waste: 19.79 MiB

sample_count: 588
sample_avg: 21.35 KiB

depth_histogram:
[  4 KiB ..   8 KiB):  84  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ
[  8 KiB ..  16 KiB): 144  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ
[ 16 KiB ..  32 KiB): 240  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ
[ 32 KiB ..  64 KiB): 120  โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ

thread 74715 (slot 0): peak 6.87 KiB of 7.99 MiB (0.08%)
thread 74790 (slot 1): peak 10.95 KiB of 2 MiB (0.53%), last 10.95 KiB, 48 samples
thread 74791 (slot 2): peak 20.05 KiB of 2 MiB (0.98%), last 20.05 KiB, 100 samples
thread 74792 (slot 3): peak 29.14 KiB of 2 MiB (1.42%), last 29.14 KiB, 128 samples
thread 74793 (slot 4): peak 38.23 KiB of 2 MiB (1.87%), last 38.23 KiB, 132 samples
thread 74794 (slot 5): peak 47.33 KiB of 2 MiB (2.31%), last 47.33 KiB, 112 samples
thread 74795 (slot 6): peak 56.42 KiB of 2 MiB (2.75%), last 56.42 KiB, 68 samples

20.0 MiB of stack reserved for 209.0 KiB of use โ€” 98.98% of it is doing nothing
```

### `no_std`

Without default features the crate is `no_std` and dependency-free apart from `portable-atomic`. There is no registry โ€” that needs thread-local storage โ€” but the primitive it is built on is the whole of the technique:

```rust
// The bounds come from the linker script, the RTOS, or wherever else they live.
let stack = Stack::from_limit_and_size(STACK_LOW, STACK_SIZE).unwrap();

// SAFETY: our own stack, and nothing below the stack pointer is live.
let paint = unsafe { stack.paint(usize::MAX) }.unwrap();

// ... run the firmware ...

// SAFETY: our own paint, on our own stack.
match unsafe { stack.peak(&paint) } {
    Peak::Exact(bytes) => defmt::info!("high-water mark: {} bytes", bytes),
    Peak::AtLeast(bytes) => defmt::warn!("deeper than {} bytes", bytes),
}
```

### License

Dual-licensed under either of:

- Creative Commons Zero v1.0 Universal ([LICENSE-CC0]LICENSE-CC0)
- MIT License ([LICENSE-MIT]LICENSE-MIT)

at your option.