Crate processkit

Expand description

processkit — async child-process management for Rust + tokio: whole-tree kill-on-drop (no orphaned subprocesses), run-and-capture, streaming, shell-free pipelines, timeouts & cancellation, and supervision.

Two layers:

ProcessGroup — a kill-on-drop container for a process tree. Every child spawned into the group, and everything those children spawn, dies with the group, so an exiting or panicking owner doesn’t leak subprocesses. Containment is a Windows Job Object, a Linux cgroup v2 (with a POSIX process-group fallback), a FreeBSD procctl(2) process reaper, or a POSIX process group on macOS/the other BSDs — observable via Mechanism. A spawn-free host_containment reports which Mechanism (and the reach of soft stop / abrupt-owner-death cleanup) a group would get on this host, before any group exists. Two caveats the ProcessGroup / Mechanism docs spell out: the guarantee rides on Drop running (a panic = "abort" process, or a SIGKILL/power-loss of the owner, skips it — Windows still reaps through Job Object handle close, while Linux’s opt-in parent-death signal reaches only the direct child and macOS/BSD have no equivalent), and on the process-group mechanism a child that calls setsid escapes containment. The one deliberate way out is Command::spawn_detached — an explicit, loudly-named opt-in that hands back a DetachedChild for which the crate exposes no public lifetime, kill, wait, timeout, capture, or control operations (Unix still reaps its exit status through a private background owner), and never contains (it inverts this guarantee on purpose; see its docs). The whole tree can be signalled (ProcessGroup::signal, see Signal), paused/resumed (ProcessGroup::suspend / ProcessGroup::resume), and inspected (ProcessGroup::members); wait_any races several running processes and reports the first to exit.
runner — async run-and-capture built on the group. Describe a run with Command, then drive it to completion (Command::output_string, Command::run, …) or start it for streaming and interactive I/O. The ProcessRunner trait runs commands to completion and is the mock seam (see ScriptedRunner). A Supervisor keeps a command alive — restarting it per policy with backoff — where Command::retry merely replays one run to success. Readiness probes (RunningProcess::wait_for_line / RunningProcess::wait_for_stderr_line / wait_for_port / wait_for) wait until a started child is actually ready instead of sleeping. A Pipeline (Command::pipe) chains commands stdout→stdin without a shell — each stage spawns into its own kill-on-drop ProcessGroup sub-group, with chain-wide teardown fanning the kill across every sub-group, pipefail outcome. Command::cancel_on ties a run to a CancellationToken: cancelling it kills the tree and every consuming path resolves to ErrorReason::Cancelled. Spawn-time sandboxing knobs: Command::inherit_env (env allow-list), Command::uid / Command::gid (Unix privilege drop), Command::setsid, Command::create_no_window, Command::priority (CPU-scheduling priority, both platforms), Command::cpu_affinity (Linux/Windows CPU placement), Command::io_priority (Linux I/O scheduling), Command::umask (Unix file-creation mask).

Async throughout (tokio). Errors are the structured Error; a non-zero exit is reported in ProcessResult, not raised, until you call ProcessResult::ensure_success.

Stability. Since 1.0, processkit follows Semantic Versioning: the public API is stable, and any breaking change lands only in a new major version, so 2.x upgrades are backward-compatible (the last breaking release was 2.1.0). (The lone exception is the mock feature’s mockall-generated expect_* surface — see below.)

Stable machine identifiers. The reporting and configuration enums — Mechanism, Outcome, ParentDeathCleanup, StopReason, StdioMode, LineTerminator, OverflowMode, Priority, RestartPolicy, plus the feature-gated LimitKind / LimitReason / LimitVerdict (limits) and Signal / SoftStopScope / SoftSignal (process-control), given as bare names here since this crate-root doc also builds with those features off — each expose a name() that returns a short, lowercase snake_case identifier for machine-readable output (a CLI’s JSONL schema, a cross-language binding, a structured log field), so a consumer publishing a contract over these types has one canonical spelling per variant instead of a hand-maintained table. These identifiers are a diagnostic surface — a stable vocabulary rather than a frozen wire schema (see the report-serde feature below, which puts exactly these identifiers on the wire) — and they carry the same stability promise as the rest of the public API: a new variant gets a new identifier, and an existing identifier is never renamed without a major release. Every enum whose value can arrive from outside (config, CLI, another language) also has a from_name(&str) inverse that returns None — an honest miss, never a silent default — on an unrecognized name. See the Errors guide’s “Stable machine identifiers” section for the whole set.

Beyond this page, the repository ships a narrative guide set — a task-oriented cookbook (“I want to …” → snippet), a deep guide per capability, and every per-platform caveat collected in one place.

Run vocabulary — one verb, one meaning, at every layer (Command, ProcessRunner/ProcessRunnerExt, CliClient):

run — require an accepted exit (0 by default, widened by Command::ok_codes) and return stdout as a String, trailing whitespace trimmed (trim_end: the final newline is noise, but leading whitespace can be significant). run_unit — the same, discarding the output.
output_string / output_bytes — return the full ProcessResult (stdout as text / raw bytes); a non-zero exit is not an error here. (output_string, not a bare output, since std::process::Command::output yields bytes — the explicit name avoids that footgun and is spelled the same on every layer.)
exit_code — the exit code, with a missing code surfaced as an error. (On a ProcessResult, code is the plain Option<i32> accessor — None for a timeout/signal kill, never a -1 sentinel.)
probe — run a predicate and read its exit code as a bool: 0 → true, 1 → false, anything else is an error (git diff --quiet, …).
parse / try_parse — run to a clean success and feed the captured stdout to a closure: parse for an infallible closure, try_parse for one returning Result (the JSON-deserialization shape). Send-contract exception: Command::parse / CliClient::parse require F: Send (and T: Send), so the returned future is Send and movable into tokio::spawn; Pipeline::parse deliberately does not require Send — its closure runs inline on the awaiting task rather than across a tokio::spawn boundary, so it accepts strictly more closures, but the resulting future is Send only when F/T happen to be. If you need to move a Pipeline::parse / Pipeline::try_parse call into tokio::spawn, make sure your closure and its output are Send yourself; the compiler won’t require it for you the way it does for Command/CliClient.
output_json (feature json) — the success-checking typed JSON form of try_parse; RunningProcess::stdout_json_lines provides strict line-wise NDJSON without buffering the complete stdout.

§Features

Every flag is additive and gates visibility only — the kill-on-drop tree guarantee is unconditional in every configuration.

stats — resource measurement: ProcessGroupStats, ProcessGroup::stats (plus the sample_stats time-series sampler and its owning 'static twin OwnedStatsSampler), the per-process RunningProcess::cpu_time/peak_memory_bytes diagnostics, and the RunningProcess::profile run summary. Opt-in for its specialized purpose (on Windows it calls the system ProcessStatus/PSAPI API — a link to an OS library, not an added crate dependency); enable with features = ["stats"], or limits, which implies it. (The features that do pull an extra crate are mock → mockall, tracing → tracing, and record / json → serde/serde_json.)
process-control (default) — tree control beyond contain+kill: Signal and ProcessGroup::{signal, suspend, resume, members, members_info, adopt, adopt_external}, the enriched MemberInfo member snapshot, and the free-standing process_info / process_is_alive queries for a pid held outside any group (reuse-safe liveness by the (pid, start time) pair).
limits — whole-tree resource caps: ResourceLimits, the max_memory/max_processes/cpu_quota builders on ProcessGroupOptions, ErrorReason::ResourceLimit (why a requested cap could not be applied), and the post-run ProcessGroup::limit_evidence report — LimitEvidence / LimitVerdict — saying whether a cap the group carried then actually fired. Implies stats.
mock — the mockall-generated testing::MockRunner for consumers’ tests. Its expect_* surface is generated by mockall and is exempt from this crate’s semver guarantees — it tracks the mockall version (an implementation detail) rather than a frozen API. The first-class doubles (ScriptedRunner / RecordingRunner) are the stable, recommended seam; reach for mock only if you specifically want expectation-style mocking.
tracing — tracing events on the processkit target: spawn and exit (program/pid/mechanism), timeout and cancellation firing, group terminate/shutdown, retry attempts, supervisor restarts and storm pauses, and teardown anomalies (stdin-writer failures, pump overruns). Never logs argv or environment values.
metrics — metrics counters and histograms over data the crate already computes: run/spawn counters, run-duration histograms, an exit-code/timeout/cancel/signal tally, and retry / supervisor restart / storm-pause events. A thin façade — the crate emits into whatever global recorder (a Prometheus/OTel exporter) the consumer installs. Labels carry only program name / mechanism / outcome / exit code — never argv or environment values, and no unbounded-cardinality key like a pid. See the docs/observability.md guide.
record — record/replay cassettes over the ProcessRunner seam: RecordReplayRunner records real Invocation → ProcessResult pairs to a JSON fixture once, then replays them hermetically — no subprocess in CI. Pulls in serde + serde_json.
json — typed JSON capture through Command::output_json, ProcessRunnerExt::output_json, and CliClient::output_json, plus line-wise NDJSON through RunningProcess::stdout_json_lines. Parse failures carry bounded raw fragments and exact decoded-output locations. Pulls in serde + serde_json.
report-serde — serde::Serialize for the crate’s report types, so a finished run, a graceful teardown, a stats tick or a supervision event can be emitted as one JSONL line (or any other self-describing serde format) without hand-copying fields and hand-calling name() per enum: ProcessResult, RunProfile, ProcessGroupStats, ShutdownReport, MemberInfo, LimitEvidence, SupervisionEvent / SupervisionOutcome / SupervisionStatus, and the enums those carry. Reuses the optional serde dependency record / json already pull, and pulls no codec of its own — pick serde_json, serde_yaml, ciborium, … yourself. The schema targets self-describing formats; rule 4 says why a non-self-describing binary codec is out of scope. Four rules define the shape:
1. Every enum travels as its stable name() identifier, never a serde-derived variant tag: {"kind": "exited", "code": 0, "signal_number": null}, not {"Exited": 0}. An enum carrying a payload is an object tagged under "kind"; one without is the bare identifier string ("restarts_exhausted"). The wire vocabulary is therefore exactly the dictionary described under Stable machine identifiers above — the same one spec/identifiers.json publishes — never a second spelling. Signal is the one named exception, with a third form: Signal::Other(i32) is a raw OS number that deliberately has no curated identifier (Signal::name() answers None there rather than minting a spelling no dictionary defines and no from_name parses back), so a signal travels as its identifier string when curated ("term") and as that bare number when not (37). The union stops there, because a key names one domain: signal is always a Signal (a ShutdownReport’s soft tier), while the raw OS number an Outcome carries is a different fact under its own key, signal_number — always a number or null. Two keys, so a consumer folding both into one JSONL table never reconciles a string with an integer under one column.
2. Serialize only — deliberately no Deserialize. These types are reported by the crate and never supplied back to it: the same asymmetry that leaves Outcome / ErrorKind / SupervisionEvent without a from_name inverse. Enums a caller genuinely does supply (Signal, RestartPolicy, Priority, …) keep their from_name, so the missing direction leaves no gap.
3. Reports about processes — never what a process produced. Nothing here serializes captured stdout/stderr content, argv, or environment values, the same secret hygiene the tracing and metrics seams keep (a child’s output routinely carries tokens, and a capture can be multi-megabyte). ProcessResult reports the run — program name, outcome, timings, truncation totals — and leaves the streams to the caller, who already holds them. For the same reason Error / ErrorReason (captured streams, the searched PATH), ProcessEvent and Finished (captured output) deliberately have no impl: report ErrorKind and attach whatever bounded, redacted detail your own contract calls for.
4. The set of fields is not frozen; the spelling of each field is. Every one of these types is #[non_exhaustive], keeps its fields private, or both — they are grown, not frozen, and no downstream struct literal pins today’s set — and that stays true on the wire: a future minor may add a key, so a consumer must ignore unknown ones (the same discipline any JSONL reader already needs). That promise is also why the schema targets self-describing formats: “ignore the keys you don’t know”, and rule 1’s identifier-or-number Signal, both need an encoding that carries names and types, which bincode / postcard deliberately do not — they will happily encode these values, but nothing here pins their layout across a minor release. What is held stable, like the rest of the public API, is everything already there — an identifier or a key is never renamed or repurposed without a major release. Time is always a number of seconds (duration_secs, elapsed_secs, delay_secs, …), the unit the metrics histograms record; a measurement a platform cannot report is null, never a fabricated 0.

§Other languages

Not on Rust? processkit-py is a Python wrapper (PyO3 bindings) over this crate’s core, with an asyncio-facing API. This crate remains the single source of truth for the containment/runner logic underneath.

Modules§

prelude: Re-exports of small vocabulary types from the crate’s 0.x dependencies, kept out of the crate root so use processkit::* doesn’t pull them in (and so a future 0.x major bump of either dependency stays contained to this module rather than the whole crate surface).
testing: Test doubles for the ProcessRunner seam: a ScriptedRunner that serves canned replies, a RecordingRunner that asserts on invocations, the Invocation it captures, a DryRunRunner that renders and echoes commands without spawning them, and (behind features) record/replay cassettes and a mockall mock.

Macros§

cli_client: Scaffold a typed CLI-wrapper struct around a CliClient.

Structs§

CancellationToken: Re-exported so callers can use processkit::CancellationToken; without a direct tokio-util dependency. See Command::cancel_on. A token which can be used to signal a cancellation request to one or more tasks.
CliClient: Owns a CLI tool’s program name, ProcessRunner, and default timeout, and builds + runs Commands against them.
Command: A description of a child process to launch: program, arguments, working directory, environment, stdin source, and an optional timeout.
DetachedChild: A minimal handle to a child spawned outside this crate’s kill-on-drop containment via Command::spawn_detached.
Error: The crate’s error type: a pointer-sized handle to a structured ErrorReason.
Finished: The outcome of a run driven via stdout_lines or events: how the run ended plus the captured standard error. Returned by RunningProcess::finish.
HostContainment: A spawn-free, side-effect-free report of how process containment behaves on this host — the answer to a consumer’s preflight “what will I get here?” without paying to create a real ProcessGroup.
JobRunner: The default runner: every run gets a fresh, private ProcessGroup owned by the run, so its tree is torn down when the run finishes (or its handle drops).
JsonLinesjson: A typed NDJSON stream returned by RunningProcess::stdout_json_lines.
LimitEvidencelimits: Post-run evidence about a group’s resource caps: one LimitVerdict per LimitKind axis, read from the container the crate itself owns.
MemberInfoprocess-control: An enriched snapshot of one member of a ProcessGroup — its pid plus best-effort metadata (parent pid, image name, start time).
OutputBufferPolicy: Caps how many captured/streamed output lines are retained in memory.
OutputLine: One decoded line carried by a ProcessEvent.
OutputOverflow: The overflow counters carried by an OutputTooLarge failure, read through Error::output_overflow / ErrorReason::output_overflow without destructuring the #[non_exhaustive] variant.
OwnedStatsSamplerstats: A periodic ProcessGroupStats series that does not borrow the group by lifetime — the owning, 'static twin of StatsSampler, for a group held behind a shared Arc.
Pipeline: A chain of Commands connected stdout→stdin — built with Command::pipe, extended with pipe, driven with the same verb vocabulary as a single Command: output_string / output_bytes for capture, run / run_unit / checked for success-checked runs, exit_code / probe for the code, and parse / try_parse for typed output — each operating on the pipefail outcome. Bound the whole chain with timeout / cancel_on.
PipelineSession: A live streaming session over a running Pipeline — the multi-stage analogue of a RunningProcess, returned by Pipeline::start. It streams the last stage’s stdout as it arrives while every inner stage drains in the background, then folds the same pipefail outcome as the buffering verbs at finish.
ProcessEvents: A Stream of a running child’s ProcessEvents (see RunningProcess::events).
ProcessGroup: A container that ties the lifetime of a child-process tree to its own.
ProcessGroupOptions: Tuning for a ProcessGroup — graceful-shutdown timing and (with the limits feature) resource limits.
ProcessGroupStatsstats: A snapshot of a process group’s resource usage.
ProcessResult: The captured result of running a process to completion.
ProcessStdin: An interactive writer to a child’s standard input.
ResourceLimitslimits: Resource limits enforced on a process group as a whole.
RetryPolicy: How a retryable run is re-attempted: a bounded number of retries with exponential backoff, a per-delay cap, and optional full jitter.
RunProfilestats: Resource summary of one finished run — produced by RunningProcess::profile.
RunningProcess: A handle to a process spawned by a runner.
ShutdownReportprocess-control: The observed facts of one graceful group teardown, returned by ProcessGroup::stop.
StatsSamplerstats: A periodic ProcessGroupStats series — created by ProcessGroup::sample_stats.
Stdin: What to feed a child process on standard input.
StdoutLines: A Stream of the child’s standard-output lines (see RunningProcess::stdout_lines).
SupervisionEvents: A bounded stream of SupervisionEvent values from one live session.
SupervisionOutcome: What a finished supervision reports — the last run plus the keeper’s telemetry.
SupervisionSession: A live handle to a running supervision, returned by Supervisor::start. Unlike run — which only reports its SupervisionOutcome at the very end — a session lets a caller watch supervision while it runs (status), ask it to stop gracefully (stop), and await its eventual outcome (wait). This is the primitive for building daemons / process managers on top of the runner layer.
SupervisionStatus: A consistent, point-in-time snapshot of a live SupervisionSession’s state — read atomically under the same lock the supervision loop publishes each change under, so every field agrees with the others (no torn read). Only non-secret facts appear here (activity, counts, the current child’s pid / start time); argv and environment values never do.
Supervisor: Keeps a Command alive: runs it, classifies every exit against the RestartPolicy and the stop_when predicate, and restarts it after an exponential-backoff delay until supervision ends.

Enums§

ErrorKind: The kind of an Error — a total, compact classification of the failure into one bucket per operational disposition, reached through Error::kind / ErrorReason::kind.
ErrorReason: The structured failure mode behind an Error — the enum you reach through Error::reason.
GiveUpAttempt: What the give_up_when classifier inspects: a crashed run that produced a ProcessResult, or a spawn/IO failure that prevented the child from ever starting (e.g. ENOENT for a mistyped program name) and so never produced one.
IoPriority: An I/O-scheduling priority for a child process (see Command::io_priority).
LimitKindlimits: Which ResourceLimits field an ErrorReason::ResourceLimit failure is about — Memory for max_memory, Processes for max_processes, Cpu for cpu_quota.
LimitReasonlimits: Why a requested resource limit could not be applied — the classification an ErrorReason::ResourceLimit failure carries so a caller (e.g. the processkit-py binding) can branch on the kind of failure without parsing the English detail text.
LimitVerdictlimits: The post-run verdict for one limit axis — did a cap this group carried actually engage while the tree ran?
LineTerminator: How the output pump decides where one captured/streamed line ends.
Mechanism: The containment mechanism actually in effect for a process group.
Outcome: How a run ended — the explicit form of the code()/timed_out() pair.
OutputStream: Which of a child’s two captured streams a decoded line came from.
OverflowMode: What to drop when a bounded output buffer is full.
ParentDeathCleanup: The reach of the parent-death hardening (Command::kill_on_parent_death) when the owning process dies abruptly — a SIGKILL of the owner or a crash, where Drop never runs to tear the containment group down.
Priority: A portable CPU-scheduling priority for a child process (see Command::priority), mapped onto the native primitive at spawn time: Unix setpriority/nice (applied through the same pre_exec seam as uid/gid), Windows priority class (OR’d into creation_flags, the same seam as create_no_window).
ProcessEvent: A lifecycle event produced by a running child process, yielded by RunningProcess::events, which merges the process’s lifecycle transitions and its two output streams into a single ordered sequence:
RestartPolicy: When the supervisor restarts an exited child. See each variant; in every case stop_when and max_restarts can end supervision first.
RlimitResource: A Unix per-process resource controlled by Command::rlimit.
Signalprocess-control: A signal to broadcast to every process in a ProcessGroup via signal.
SoftSignalprocess-control: The fate of a graceful teardown’s best-effort soft-signal tier — what the kernel actually observed of the polite “please exit” request ProcessGroup::stop issues before the grace window, as opposed to what it tried to do.
SoftStopScopeprocess-control: How far a soft stop on the group axis reaches on this group, right now — the honest answer to “if I ask this group to stop gracefully (signal(Signal::Term) / Signal::Int), which of its members will actually receive that request?”
StdioMode: How a child process’s standard output or error stream is connected.
StopReason: Why supervision ended.
SupervisionEvent: A typed transition emitted by a live SupervisionSession.

Traits§

CapturePolicy: A consumer-supplied, typed seam that shapes each decoded line just before it enters the capture backlog — the redaction-at-capture extension point.
IntoCommand: What a CliClient verb accepts: either an argument list — built into a Command for the client’s program with its defaults (timeout, env, cancellation) applied — or a ready-made Command, run as-is.
ProcessRunner: Runs a Command — to a captured result (output_string / output_bytes) or a live handle (start).
ProcessRunnerExt: Convenience methods available on every ProcessRunner (including &dyn ProcessRunner), layered over output_string.

Functions§

host_containment: Report how process containment behaves on this host without creating a container or spawning anything — a spawn-free preflight (a doctor / host-check command that must have no side effects) that answers what a ProcessGroup would otherwise only reveal after it exists: which Mechanism a group created here and now would use, how far a soft stop reaches, what the OS guarantees on abrupt owner death, and this crate’s version.
output_all: Run every command in commands, keeping at most concurrency of them live at once, and collect all their results in input order.
output_all_bytes: The raw-bytes companion to output_all: captures each command’s stdout as Vec<u8> instead of decoded text. All other semantics are identical — see output_all. The streaming counterpart is output_stream_bytes.
output_stream: Run every command in commands with at most concurrency live at once, yielding each result — an (input index, Result<ProcessResult<String>>) pair — the moment that command finishes. This is the streaming sibling of output_all: the same bounded fan-out and the same per-command error semantics (an Err is a spawn/I/O failure; a non-zero exit is an Ok(ProcessResult); the fan-out never short-circuits), but presented as a Stream over completions instead of a single Vec at the end.
output_stream_bytes: The raw-bytes companion to output_stream: each yielded ProcessResult captures stdout as Vec<u8> instead of decoded text (for binary artifacts — git cat-file, tar -c, an image transcoder). Scheduling, completion ordering, input indexing, no-short-circuit, and cancellation/teardown are identical to output_stream; the buffering counterpart is output_all_bytes.
output_string: Run program with args inside a private job and capture the result without erroring on a non-zero exit — for commands whose exit code is meaningful.
process_infoprocess-control: Look up the identity and best-effort metadata of an arbitrary process by pid — the standalone companion to ProcessGroup::members_info, for a pid the caller holds outside any group (a pid saved to disk across runs, a launch registry, an e2e probe watching a process from outside its container).
process_is_aliveprocess-control: Reuse-safe liveness: is the process at pid still the same instance you saw earlier — the one whose start_time you saved?
run: Run program with args inside a private job and return trimmed stdout, or an Error on a non-zero exit / spawn failure / timeout. A thin shim over Command; use the builder for a working directory, env, stdin, timeout, or the full verb vocabulary.
wait_all: Wait for all of several running processes to exit, returning their Outcomes in the same order as processes. The processes are only borrowed and stay usable afterwards (the exit status tokio caches remains re-readable).
wait_any: Wait for whichever of several running processes exits first, returning its index in processes and its Outcome (matching RunningProcess::wait).
which: Resolve program to a concrete executable path without launching it — a spawn-free preflight for a doctor / early-diagnosis check (“is git installed?”) that must have no side effects. A thin shim over Command::new(program).resolve_program(); use the builder form when you need prefer_local directories or a relocated PATH honored (the client-level CliClient::resolve_program does the same for a wrapped tool).

Type Aliases§

Result: Crate result alias.