Expand description
StackPulse is a Rust library for building Linux profilers with perf_event.
Recorder captures CPU stack samples from running processes and writes
them to a compact spool file. Snapshot loads that file back, and
Symbolizer resolves the recorded addresses into frames with names and
source locations, ready for whatever aggregation or output your profiler does
with them.
§Quick example
use std::time::{Duration, Instant};
use stackpulse::{AttachMode, Pid, Recorder, RecorderOptions, SampleRate, Snapshot};
let pid = Pid::try_from(pid)?;
let mut recorder = Recorder::attach(
pid,
"profile.spool",
AttachMode::StopWhileAttaching,
RecorderOptions::new(SampleRate::hz(99)?).stack_size(60 * 1024),
)?;
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline && recorder.process_is_active(pid)? {
recorder.poll(Duration::from_millis(100))?;
}
recorder.finish()?;
let reader = Snapshot::open("profile.spool")?;
let mut symbolizer = reader.symbolizer().build()?;
for stack in reader.stacks() {
for frame in symbolizer.resolve(stack)? {
println!("{}", frame.display_name());
}
}§Core types
| Type | Role |
|---|---|
Recorder | Attaches to one or more processes, drains perf_event_open ring buffers, writes a spool file. |
Snapshot | Reads a completed spool into memory for random access. |
Replay | Validates a spool, retains its definitions, and decodes samples sequentially to reduce memory use for large profiles. |
Tail | Decodes bounded batches from a growing spool for live processing. |
Symbolizer | Resolves raw frame addresses using ELF symbols, kernel symbols, Python perf maps, and address fallbacks. The native ELF backend is pluggable via symbolize::NativeSymbolizer. |
symbolize::NativeSymbolizer | Trait for swapping in your own native symbolizer (custom debuginfod, debug-dir, or source-info policy). Symbolizer still handles kernel and perf-map frames. |
profile types | Resolved frame data types: what an aggregator, UI, or exporter consumes. |
The recorder writes a self-contained spool file. Symbolization reads it later and can run on another host if the same binaries and perf maps are available.
§Vocabulary
- A sample is one timestamped observation of one thread.
- A module is an executable memory range: a binary, shared object, anonymous JIT mapping, or kernel range.
- A raw frame is an address recorded in the spool file.
- A resolved frame is a displayable
profile::ResolvedFrameproduced bySymbolizer. - A spool file is the compact on-disk profile written by
Recorder.
§Raw replay
Recording never depends on symbolization: the spool stores raw instruction
pointers together with the module mappings observed at capture time. An
application that already has its own symbol pipeline can skip
Symbolizer and consume those raw frames directly:
let reader = stackpulse::Snapshot::open("profile.spool")?;
for stack in reader.stacks() {
for context in stack.contexts() {
let ip = context.frame.abs_ip;
if let Some(module) = context.module {
// Pass `ip`, `module.module`, and `module.file_relative_ip` to your symbolizer.
}
}
}SampleStack::contexts does not symbolize anything. It binds each borrowed raw
frame to the module mapping StackPulse recorded at capture time, which is what
an external symbolizer needs to translate the address, even across remaps.
§Sequential replay
For large profiles, use Replay to avoid retaining every
spool::SampleRecord in memory:
use stackpulse::Replay;
let reader = Replay::open("profile.spool")?;
let mut symbolizer = reader.symbolizer().build()?;
for stack in reader.stacks() {
for frame in symbolizer.resolve(stack)? {
println!("{}", frame.display_name());
}
}Opening still validates the complete spool and retains modules, frames,
interned stacks, threads, and runtime markers; only the samples themselves are
decoded on the fly as the iterator advances. The file must remain unchanged
while the reader is alive. Use Snapshot instead when you need
random access to samples().
§Live tailing
Use Tail when recording and processing run concurrently. Apply each batch
to its symbolizer before resolving the batch’s stacks. With
StackCache::External, discard prepared stacks selected by the returned
symbolize::Invalidation before resolving more samples.
Prefer Recorder::tail when the recorder and tail live in one process. It
shares handles still held by the recorder’s bounded image cache, so deleted or
replaced files can still be symbolized exactly while their images remain in
that cache. Tail::open has only the paths stored in the spool.
use stackpulse::{StackCache, Tail};
let mut tail = Tail::open("profile.spool")?;
let mut symbolizer = tail.symbolizer().stack_cache(StackCache::External).build()?;
// Call this function after each writer flush and once after the writer finishes.
fn process_visible(
tail: &mut Tail,
symbolizer: &mut stackpulse::Symbolizer,
) -> stackpulse::Result<()> {
loop {
let batch = tail.poll()?;
let invalidation = symbolizer.update(&batch)?;
invalidate_prepared_stacks(|pid| invalidation.affects_process(pid));
for stack in batch.stacks() {
for frame in symbolizer.resolve(stack)? {
aggregate(frame);
}
}
if !batch.has_more() {
break;
}
}
Ok(())
}TailBatch borrows the tail’s reusable sample storage. Finish processing and
drop the batch before polling again. has_more() means another complete batch
may already be visible, not that the writer is still running. Poll again
promptly when it is true. When it is false, wait for the writer to flush or
finish before polling again. A final incomplete record is retried after the
writer appends the remainder.
Symbolizer::update verifies that the batch and symbolizer came from the same
tail. It installs new definitions, refreshes live perf maps and kernel symbols,
and retires resources only after the last batch that can reference them has
been processed. An invalidation can target individual processes or every
prepared stack. Callers using StackCache::Internal do not maintain that
external state, but must still call update before resolve.
External caches can use symbolize::ResolvedFrameId to reuse converted
frames across different stacks. IDs are unique for one symbolizer and are
never reused. Cache a whole resolved stack only when
symbolize::ResolvedStack::is_cacheable is true; otherwise a temporary
native-image failure would preserve an address-only result permanently.
§Plugging in an external native symbolizer
The default constructors install the bundled wholesym backend for native
ELF symbol lookup. To replace it, implement symbolize::NativeSymbolizer and pass a
factory to SymbolizerBuilder::native. StackPulse groups native lookups by
process and passes them to the backend in batches. Each symbolize::NativeLookup
contains the selected module and the absolute, relative, and image addresses:
use stackpulse::symbolize::{NativeLookup, NativeSymbolizer, NativeSymbols};
use stackpulse::Snapshot;
struct MySymbolizer { /* your wholesym / debuginfod / dwarf state */ }
impl NativeSymbolizer for MySymbolizer {
type Error = std::convert::Infallible;
fn symbolize(
&mut self,
requests: &[NativeLookup],
output: &mut Vec<NativeSymbols>,
) -> Result<(), Self::Error> {
for request in requests {
let _ = (request.module().path(), request.image_address());
output.push(NativeSymbols::unresolved());
}
Ok(())
}
}
let reader = Snapshot::open("profile.spool")?;
let mut symbolizer = reader.symbolizer()
.native(|_pid| MySymbolizer { /* ... */ })
.build()?;Kernel frames (/proc/kallsyms) and Python or JIT perf maps
(/tmp/perf-<pid>.map) stay inside Symbolizer; the plug-in only sees
native module addresses. Consumers that always supply a native symbolizer can
disable StackPulse’s default features to omit wholesym and Tokio. In that
configuration Symbolizer::has_native_backend is false until native(...)
installs one; native frames otherwise resolve to address-only values.
Backend tests can construct requests directly with
symbolize::NativeModule::new and symbolize::NativeLookup::new. Each
synthetic module gets a distinct opaque image identity. Its image_path() is
None; recorded modules expose that path only when StackPulse retained the
exact validated file backing the Linux mapping.
During live tailing, symbolize::NativeSymbolizer::retire_module reports
when a mapping is no longer active. Backends should use it to discard
mapping-specific indexes. State shared by NativeModule::image_id() can stay
cached until the last mapping for that image is retired.
Symbol results can be provisional when a retryable error prevents StackPulse from opening a validated native image. A later resolution attempt can replace that address-only fallback with more specific symbols. External stack caches should avoid permanently caching those temporary fallbacks.
§Runtime requirements
StackPulse runs on Linux and uses perf_event_open, /proc, ELF metadata,
optional /proc/kallsyms, and optional Python perf maps under /tmp.
User-space recording works as the same user that owns the target. Kernel
frames, containers, hardened systems, and aggressive sample rates may need
extra capabilities (typically CAP_PERFMON) or a relaxed
perf_event_paranoid setting. See the Permissions section in the
explanation chapter for the full breakdown.
§Guide
The guide walks through process attachment, startup capture, configuration, symbolization, diagnostics, and the SPULSE file format.
Re-exports§
pub use error::Error;pub use error::ErrorKind;pub use error::Result;pub use identity::Pid;pub use identity::Tid;pub use spool::Replay;pub use spool::Snapshot;pub use spool::Tail;pub use symbolize::StackCache;pub use symbolize::Symbolizer;pub use symbolize::SymbolizerBuilder;
Modules§
- children
- Child-process discovery for recorded targets.
- docs
- Recording and integration guide.
- error
- Typed errors returned by recording, spool, and symbolization workflows.
- identity
- Validated Linux process and thread identifiers.
- process
- Launch a process suspended and attach recording before
execve. - profile
- Resolved frames and symbol metadata returned by
Symbolizer. - record
- Perf recording types and statistics.
- spool
- Spool readers and raw recorded profile types.
- state
- Process liveness checks, exit watching, and signal helpers.
- symbolize
- Resolves stack frames recorded in perf spool files into displayable profile frames.
Structs§
- Recorder
- Records stack samples for one or more Linux processes.
- Recorder
Options - Options used when attaching a
Recorderto a process. - Recording
Summary - Counters collected while recording.
Enums§
- Attach
Mode - How recording should attach to a process.
- Sample
Rate - Valid sampling frequency.