stackler 0.1.0

A lightweight, always-on stack telemetry library that requires no instrumentation.
Documentation
//! **Always-on stack telemetry for Rust, with nothing to instrument.**
//!
//! `stackler` answers the questions a program cannot normally answer about its
//! own stacks: how deep is this thread right now, how deep has it *ever* been,
//! how much of the stack space it reserves does it actually use, and how close is
//! it to running out. It answers them at runtime, in production, without a
//! debugger, without a rebuild, and — this is the point — without touching the
//! code being measured.
//!
//! ## why stackler?
//!
//! Stack use is normally observed either not at all or all at once: a crash. The
//! tools in between are heavyweight — a debugger, a sampling profiler with
//! symbolication, `RUST_MIN_STACK` guesswork after the fact. `stackler` occupies
//! a lighter tier: a handful of numbers per thread, cheap enough to leave on.
//!
//! - **nothing to annotate**: no macros on functions, no wrappers around
//!   recursion, no rebuild of the code under observation. Depth is read from the
//!   stack pointer, and peaks are read from the stack itself.
//! - **free while running**: with the `watermark` feature the measured code pays
//!   *nothing at all* — the unused stack is filled with a marker and later
//!   inspected to see how far it was overwritten. All the cost is at the
//!   painting and the scan.
//! - **contention-free**: every observation writes only to the recording
//!   thread's own cache-aligned slot — depth, peak, counters and depth histogram
//!   alike. Nothing on that path is shared, so concurrent sampling scales with the
//!   cores doing it; the aggregates are computed when a snapshot is taken.
//! - **overflow guard**: `has_headroom` lets input-driven recursion return an
//!   error instead of dying on a guard page, and counts how often it came close.
//! - **reserved vs. used**: `Stats::utilization` and `Stats::waste` put a
//!   number on the address space a thread pool spends on stacks nobody descends
//!   into.
//! - **`no_std` friendly**: the [`Stack`] primitives — the same painting and
//!   scanning an RTOS uses to report a task's high-water mark — need neither
//!   `std` nor an allocator.
//!
//! ## quickstart
//!
//! ```rust
//! # #[cfg(feature = "std")] {
//! // In `main`: configures the crate and registers this thread. Optional — any
//! // call into `stackler` attaches the calling thread on its own.
//! stackler::Stackler::new().install();
//!
//! // ... do some deeply nested work ...
//!
//! let stats = stackler::stats();
//! println!("deepest stack use so far: {} bytes", stats.peak_max);
//! # }
//! ```
//!
//! ## measuring specific operations
//!
//! With the `watermark` feature, the stack cost of an operation can be measured
//! exactly the way a heap profiler measures allocations — which makes it an
//! assertion, and an assertion makes it a regression test:
//!
//! ```rust
//! # fn parse(input: &str) -> usize { input.len() }
//! # #[cfg(all(feature = "std", feature = "watermark"))] {
//! let (result, peak) = stackler::measure_peak(|| parse("[[[[nested]]]]"));
//!
//! let peak = peak.expect("this thread's stack is known");
//! assert!(peak.bytes() < 64 * 1024, "regression: the parser got too deep!");
//! # }
//! ```
//!
//! ## guarding recursion
//!
//! Code whose depth depends on its input can ask whether it can afford to go
//! deeper, and turn what would have been a fatal overflow into an ordinary error:
//!
//! ```rust
//! # struct Node { children: Vec<Node> }
//! # struct TooDeep;
//! # #[cfg(feature = "std")]
//! fn walk(node: &Node) -> Result<(), TooDeep> {
//!     if !stackler::has_headroom(32 * 1024) {
//!         return Err(TooDeep);
//!     }
//!     for child in &node.children {
//!         walk(child)?;
//!     }
//!     Ok(())
//! }
//! ```
//!
//! ## how much is automatic
//!
//! The features form a ladder, and each rung asks less of the program:
//!
//! | build | what the program does | what it gets |
//! | :--- | :--- | :--- |
//! | default | calls `sample` wherever it has a natural tick | depth distribution, per-thread peaks, guard |
//! | `watermark` | calls `Stackler::install` once | true high-water marks, for code that never calls in |
//! | `sampler` | keeps a `Sampler` alive | those marks kept fresh for every thread, continuously |
//! | `hook_pthread` | nothing | every thread in the process, including those it did not create |
//!
//! ## accuracy
//!
//! Depth and headroom are exact on architectures whose stack pointer can be read
//! directly, which is all of the mainstream ones; elsewhere they are off by at
//! most one frame.
//!
//! Watermarks are approximate, and always in the same direction — they can
//! understate depth, never overstate it. See `Peak` for the specifics: the
//! resolution is a few hundred bytes, a frame that reserves stack without writing
//! to all of it leaves part of the marker intact, and a peak deeper than the
//! painted region can only be reported as a lower bound.
//!
//! ## caveats
//!
//! - Painting commits the memory it touches, so a painted thread's resident set
//!   grows by up to `Stackler::paint_depth` bytes. That is the entire cost
//!   model of the technique, and the reason the depth is a knob.
//! - Only the thread that owns a stack may paint it. Anyone may *read* a
//!   watermark, which is what makes the background `sampler` possible, but such
//!   a read races with the owner by construction: the values it sees may be torn.
//!   It stays sound in practice because the only question asked of each word is
//!   whether the marker is still there, and a torn word is a written word.
//! - On Linux, painting the main thread's stack relies on the kernel growing the
//!   main stack mapping on demand. This is how every kernel since 4.13 behaves;
//!   on much older ones, prefer to leave the main thread unpainted.
//! - The registry tracks `MAX_THREADS` threads at a time. Threads beyond that
//!   are counted in `Stats::untracked` rather than silently dropped.
//! - The first call on a thread is not reentrant: it asks the OS for the thread's
//!   bounds, and on glibc's main thread that reads `/proc/self/maps` and
//!   allocates. Calling `stackler` from inside a global allocator is therefore a
//!   bad idea unless the thread has been `attach`ed beforehand.

#![cfg_attr(not(feature = "std"), no_std)]
#![deny(missing_docs)]
#![deny(clippy::all)]
// A dangling link is a documentation bug, and it only shows up in the feature
// combinations that lack the item being linked to — which is why this is denied
// rather than left to a warning nobody builds often enough to see.
#![deny(rustdoc::broken_intra_doc_links)]

#[cfg(feature = "std")]
mod config;
#[cfg(feature = "fmt")]
mod fmt;
#[cfg(feature = "histograms")]
mod histogram;
// Interposing `pthread_create` needs a `pthread_create` to interpose; enabling the
// feature anywhere else is inert rather than a compile error, so that a workspace
// which turns on every feature still builds on every platform.
#[cfg(all(feature = "hook_pthread", unix))]
mod hook;
mod platform;
#[cfg(feature = "std")]
mod registry;
#[cfg(feature = "sampler")]
pub mod sampler;
mod stack;
#[cfg(feature = "std")]
mod stats;

#[cfg(feature = "std")]
pub use config::{DEFAULT_PAINT_DEPTH, DEFAULT_RESERVE};
#[cfg(feature = "histograms")]
pub use histogram::Histogram;
pub use platform::sp;
#[cfg(feature = "std")]
pub use registry::{
    MAX_THREADS, attach, current_stack, depth, for_each_thread, has_headroom, headroom,
    is_attached, reset, sample, threads,
};
#[cfg(all(feature = "std", feature = "watermark"))]
pub use registry::{peak, poll_all, repaint};
pub use stack::Stack;
#[cfg(feature = "watermark")]
pub use stack::{Paint, Peak};
#[cfg(feature = "std")]
pub use stats::{Stats, ThreadStats, stats};

/// Configures process-wide stack telemetry.
///
/// Everything here has a working default, so the only reason to build one of
/// these is to pick a different trade-off — and to decide *when* the calling
/// thread pays for attaching, which is what [`Stackler::install`] does on the
/// side.
///
/// ```rust
/// stackler::Stackler::new()
///     .paint_depth(1024 * 1024) // look a megabyte deep
///     .reserve(128 * 1024)      // treat the last 128 KiB as the danger zone
///     .install();
/// ```
#[cfg(feature = "std")]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Stackler {
    paint_depth: usize,
    reserve: usize,
}

#[cfg(feature = "std")]
impl Default for Stackler {
    fn default() -> Self {
        Self::new()
    }
}

#[cfg(feature = "std")]
impl Stackler {
    /// Creates a configuration with the default settings.
    pub const fn new() -> Self {
        Self {
            paint_depth: DEFAULT_PAINT_DEPTH,
            reserve: DEFAULT_RESERVE,
        }
    }

    /// Sets how deep below the stack pointer a thread's stack is painted when it
    /// attaches, in bytes.
    ///
    /// This bounds both what a watermark can report and what it costs: painting
    /// commits every page it touches, and a scan walks the untouched part of the
    /// region. Zero disables painting entirely, leaving sampling as the only
    /// source of peaks.
    ///
    /// Only meaningful with the `watermark` feature; without it, nothing paints.
    pub const fn paint_depth(mut self, bytes: usize) -> Self {
        self.paint_depth = bytes;
        self
    }

    /// Sets the headroom below which an observation counts as near the limit, in
    /// bytes.
    ///
    /// Recorded in [`Stats::near_limit_count`], which is the metric to alert on:
    /// it moves before anything overflows.
    pub const fn reserve(mut self, bytes: usize) -> Self {
        self.reserve = bytes;
        self
    }

    /// Applies the configuration and attaches the calling thread.
    ///
    /// Returns `true` if the calling thread is tracked from now on; `false` means
    /// its stack bounds could not be determined, or the registry was full. The
    /// configuration is applied either way, and governs every thread that attaches
    /// from here on.
    ///
    /// A thread that was already attached painted its stack under the previous
    /// settings, so this repaints it under the new ones — otherwise raising
    /// [`Stackler::paint_depth`] would apply to every thread except the one asking
    /// for it. Nothing is lost in the process: what the old paint knew is folded
    /// into the thread's recorded peak first.
    pub fn install(self) -> bool {
        #[cfg(feature = "watermark")]
        let attached_already = is_attached();

        config::set_paint_depth(self.paint_depth);
        config::set_reserve(self.reserve);

        let attached = attach();

        #[cfg(feature = "watermark")]
        if attached && attached_already {
            repaint();
        }

        attached
    }
}

/// Measures the stack telemetry of an operation, returning its result alongside
/// the [`Stats`] delta.
///
/// The counters describe what happened during the call; the gauges describe the
/// process as it was when the call returned. [`Stats`] says which is which.
///
/// This is the whole-process view, and it only sees what threads report about
/// themselves. To measure how deep *this* closure got, reach for
#[cfg_attr(feature = "watermark", doc = "[`measure_peak`] instead.")]
#[cfg_attr(
    not(feature = "watermark"),
    doc = "`measure_peak`, which the `watermark` feature provides."
)]
#[cfg(feature = "std")]
pub fn measure<R>(f: impl FnOnce() -> R) -> (R, Stats) {
    let before = stats();
    let r = f();
    let after = stats();
    (r, &after - &before)
}

/// Measures how much stack an operation uses, returning its result alongside the
/// deepest point it reached.
///
/// The stack is painted afresh before the call and read back afterwards, so the
/// answer covers everything the closure did — including code that has never heard
/// of `stackler` — and is expressed relative to the depth at the call site.
///
/// Returns `None` for the peak if the calling thread's stack could not be
/// painted: its bounds are unknown, the registry is full, or the configured
/// [paint depth](Stackler::paint_depth) is zero.
///
/// ```rust
/// let (_, peak) = stackler::measure_peak(|| {
///     let big = [0u8; 32 * 1024];
///     std::hint::black_box(&big);
/// });
///
/// assert!(peak.unwrap().bytes() >= 32 * 1024);
/// ```
///
/// # Accuracy
///
/// Painting describes the stack *below* the frame that painted it, so a closure
/// shallower than a kilobyte or so cannot be resolved and reads as
/// [`Peak::Exact(0)`](Peak::Exact). See [`Peak`] for the rest of the caveats.
///
/// Repainting discards the marks left by earlier excursions, so the thread's
/// watermark starts over from here. Its recorded peak does not: the reading is
/// folded into it, and [`Stats::peak_max`] stays monotonic.
#[cfg(all(feature = "std", feature = "watermark"))]
pub fn measure_peak<R>(f: impl FnOnce() -> R) -> (R, Option<Peak>) {
    /// A closure inlined into `measure_peak` would keep its locals in a frame
    /// that was already allocated by the time the painting happened, where no
    /// watermark can describe them. Calling it through a barrier the optimizer
    /// may not inline puts its frame below the paint, which is the whole point.
    #[inline(never)]
    fn barrier<R>(f: impl FnOnce() -> R) -> R {
        f()
    }

    // Everything is measured against this: the paint lies below it, and so does
    // every frame the closure creates.
    let sp = sp();

    let Some(paint) = repaint() else {
        return (barrier(f), None);
    };

    let r = barrier(f);

    // SAFETY: the region was painted by this thread, on this thread's stack, and
    // this thread is by definition still running.
    let peak = match unsafe { stack::deepest_touched(&paint) } {
        // Reading from `sp` rather than from the top of the painted region keeps
        // the unpaintable gap between them out of the answer.
        Some(addr) if addr == paint.lo() => Peak::AtLeast(sp.saturating_sub(addr)),
        Some(addr) => Peak::Exact(sp.saturating_sub(addr)),
        None => Peak::Exact(0),
    };

    // The reading is relative to the call site; the registry tracks absolute
    // depths, so translate before folding it in.
    if let Some(stack) = current_stack() {
        let here = stack.base().saturating_sub(sp);
        registry::record_peak(here + peak.bytes(), peak.is_saturated());
    }

    (r, Some(peak))
}