Skip to main content

Crate stackpulse

Crate stackpulse 

Source
Expand description

StackPulse is a Rust library for building Linux profilers with perf_event. Recorder captures CPU stack samples from running processes and writes them to a compact spool file. Snapshot loads that file back, and Symbolizer resolves the recorded addresses into frames with names and source locations, ready for whatever aggregation or output your profiler does with them.

§Quick example

use std::time::{Duration, Instant};
use stackpulse::{AttachMode, Pid, Recorder, RecorderOptions, SampleRate, Snapshot};
let pid = Pid::try_from(pid)?;
let mut recorder = Recorder::attach(
    pid,
    "profile.spool",
    AttachMode::StopWhileAttaching,
    RecorderOptions::new(SampleRate::hz(99)?).stack_size(60 * 1024),
)?;

let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline && recorder.process_is_active(pid)? {
    recorder.poll(Duration::from_millis(100))?;
}
recorder.finish()?;

let reader = Snapshot::open("profile.spool")?;
let mut symbolizer = reader.symbolizer().build()?;

for stack in reader.stacks() {
    for frame in symbolizer.resolve(stack)? {
        println!("{}", frame.display_name());
    }
}

§Core types

TypeRole
RecorderAttaches to one or more processes, drains perf_event_open ring buffers, writes a spool file.
SnapshotReads a completed spool into memory for random access.
ReplayValidates a spool, retains its definitions, and decodes samples sequentially to reduce memory use for large profiles.
TailDecodes bounded batches from a growing spool for live processing.
SymbolizerResolves raw frame addresses using ELF symbols, kernel symbols, Python perf maps, and address fallbacks. The native ELF backend is pluggable via symbolize::NativeSymbolizer.
symbolize::NativeSymbolizerTrait for swapping in your own native symbolizer (custom debuginfod, debug-dir, or source-info policy). Symbolizer still handles kernel and perf-map frames.
profile typesResolved frame data types: what an aggregator, UI, or exporter consumes.

The recorder writes a self-contained spool file. Symbolization reads it later and can run on another host if the same binaries and perf maps are available.

§Vocabulary

  • A sample is one timestamped observation of one thread.
  • A module is an executable memory range: a binary, shared object, anonymous JIT mapping, or kernel range.
  • A raw frame is an address recorded in the spool file.
  • A resolved frame is a displayable profile::ResolvedFrame produced by Symbolizer.
  • A spool file is the compact on-disk profile written by Recorder.

§Raw replay

Recording never depends on symbolization: the spool stores raw instruction pointers together with the module mappings observed at capture time. An application that already has its own symbol pipeline can skip Symbolizer and consume those raw frames directly:

let reader = stackpulse::Snapshot::open("profile.spool")?;

for stack in reader.stacks() {
    for context in stack.contexts() {
        let ip = context.frame.abs_ip;
        if let Some(module) = context.module {
            // Pass `ip`, `module.module`, and `module.file_relative_ip` to your symbolizer.
        }
    }
}

SampleStack::contexts does not symbolize anything. It binds each borrowed raw frame to the module mapping StackPulse recorded at capture time, which is what an external symbolizer needs to translate the address, even across remaps.

§Sequential replay

For large profiles, use Replay to avoid retaining every spool::SampleRecord in memory:

use stackpulse::Replay;
let reader = Replay::open("profile.spool")?;
let mut symbolizer = reader.symbolizer().build()?;

for stack in reader.stacks() {
    for frame in symbolizer.resolve(stack)? {
        println!("{}", frame.display_name());
    }
}

Opening still validates the complete spool and retains modules, frames, interned stacks, threads, and runtime markers; only the samples themselves are decoded on the fly as the iterator advances. The file must remain unchanged while the reader is alive. Use Snapshot instead when you need random access to samples().

§Live tailing

Use Tail when recording and processing run concurrently. Apply each batch to its symbolizer before resolving the batch’s stacks. With StackCache::External, discard prepared stacks selected by the returned symbolize::Invalidation before resolving more samples.

Prefer Recorder::tail when the recorder and tail live in one process. It shares handles still held by the recorder’s bounded image cache, so deleted or replaced files can still be symbolized exactly while their images remain in that cache. Tail::open has only the paths stored in the spool.

use stackpulse::{StackCache, Tail};
let mut tail = Tail::open("profile.spool")?;
let mut symbolizer = tail.symbolizer().stack_cache(StackCache::External).build()?;

// Call this function after each writer flush and once after the writer finishes.
fn process_visible(
    tail: &mut Tail,
    symbolizer: &mut stackpulse::Symbolizer,
) -> stackpulse::Result<()> {
  loop {
    let batch = tail.poll()?;
    let invalidation = symbolizer.update(&batch)?;
    invalidate_prepared_stacks(|pid| invalidation.affects_process(pid));

    for stack in batch.stacks() {
        for frame in symbolizer.resolve(stack)? {
            aggregate(frame);
        }
    }

    if !batch.has_more() {
        break;
    }
  }
  Ok(())
}

TailBatch borrows the tail’s reusable sample storage. Finish processing and drop the batch before polling again. has_more() means another complete batch may already be visible, not that the writer is still running. Poll again promptly when it is true. When it is false, wait for the writer to flush or finish before polling again. A final incomplete record is retried after the writer appends the remainder.

Symbolizer::update verifies that the batch and symbolizer came from the same tail. It installs new definitions, refreshes live perf maps and kernel symbols, and retires resources only after the last batch that can reference them has been processed. An invalidation can target individual processes or every prepared stack. Callers using StackCache::Internal do not maintain that external state, but must still call update before resolve.

External caches can use symbolize::ResolvedFrameId to reuse converted frames across different stacks. IDs are unique for one symbolizer and are never reused. Cache a whole resolved stack only when symbolize::ResolvedStack::is_cacheable is true; otherwise a temporary native-image failure would preserve an address-only result permanently.

§Plugging in an external native symbolizer

The default constructors install the bundled wholesym backend for native ELF symbol lookup. To replace it, implement symbolize::NativeSymbolizer and pass a factory to SymbolizerBuilder::native. StackPulse groups native lookups by process and passes them to the backend in batches. Each symbolize::NativeLookup contains the selected module and the absolute, relative, and image addresses:

use stackpulse::symbolize::{NativeLookup, NativeSymbolizer, NativeSymbols};
use stackpulse::Snapshot;

struct MySymbolizer { /* your wholesym / debuginfod / dwarf state */ }

impl NativeSymbolizer for MySymbolizer {
    type Error = std::convert::Infallible;

    fn symbolize(
        &mut self,
        requests: &[NativeLookup],
        output: &mut Vec<NativeSymbols>,
    ) -> Result<(), Self::Error> {
        for request in requests {
            let _ = (request.module().path(), request.image_address());
            output.push(NativeSymbols::unresolved());
        }
        Ok(())
    }
}

let reader = Snapshot::open("profile.spool")?;
let mut symbolizer = reader.symbolizer()
    .native(|_pid| MySymbolizer { /* ... */ })
    .build()?;

Kernel frames (/proc/kallsyms) and Python or JIT perf maps (/tmp/perf-<pid>.map) stay inside Symbolizer; the plug-in only sees native module addresses. Consumers that always supply a native symbolizer can disable StackPulse’s default features to omit wholesym and Tokio. In that configuration Symbolizer::has_native_backend is false until native(...) installs one; native frames otherwise resolve to address-only values.

Backend tests can construct requests directly with symbolize::NativeModule::new and symbolize::NativeLookup::new. Each synthetic module gets a distinct opaque image identity. Its image_path() is None; recorded modules expose that path only when StackPulse retained the exact validated file backing the Linux mapping.

During live tailing, symbolize::NativeSymbolizer::retire_module reports when a mapping is no longer active. Backends should use it to discard mapping-specific indexes. State shared by NativeModule::image_id() can stay cached until the last mapping for that image is retired.

Symbol results can be provisional when a retryable error prevents StackPulse from opening a validated native image. A later resolution attempt can replace that address-only fallback with more specific symbols. External stack caches should avoid permanently caching those temporary fallbacks.

§Runtime requirements

StackPulse runs on Linux and uses perf_event_open, /proc, ELF metadata, optional /proc/kallsyms, and optional Python perf maps under /tmp.

User-space recording works as the same user that owns the target. Kernel frames, containers, hardened systems, and aggressive sample rates may need extra capabilities (typically CAP_PERFMON) or a relaxed perf_event_paranoid setting. See the Permissions section in the explanation chapter for the full breakdown.

§Guide

The guide walks through process attachment, startup capture, configuration, symbolization, diagnostics, and the SPULSE file format.

Re-exports§

pub use error::Error;
pub use error::ErrorKind;
pub use error::Result;
pub use identity::Pid;
pub use identity::Tid;
pub use spool::Replay;
pub use spool::Snapshot;
pub use spool::Tail;
pub use symbolize::StackCache;
pub use symbolize::Symbolizer;
pub use symbolize::SymbolizerBuilder;

Modules§

children
Child-process discovery for recorded targets.
docs
Recording and integration guide.
error
Typed errors returned by recording, spool, and symbolization workflows.
identity
Validated Linux process and thread identifiers.
process
Launch a process suspended and attach recording before execve.
profile
Resolved frames and symbol metadata returned by Symbolizer.
record
Perf recording types and statistics.
spool
Spool readers and raw recorded profile types.
state
Process liveness checks, exit watching, and signal helpers.
symbolize
Resolves stack frames recorded in perf spool files into displayable profile frames.

Structs§

Recorder
Records stack samples for one or more Linux processes.
RecorderOptions
Options used when attaching a Recorder to a process.
RecordingSummary
Counters collected while recording.

Enums§

AttachMode
How recording should attach to a process.
SampleRate
Valid sampling frequency.