Crate processkit

Expand description

processkit — async child-process management for Rust + tokio: whole-tree kill-on-drop (no orphaned subprocesses), run-and-capture, streaming, shell-free pipelines, timeouts & cancellation, and supervision.

Two layers:

ProcessGroup — a kill-on-drop container for a process tree. Every child spawned into the group, and everything those children spawn, dies with the group, so an exiting or panicking owner doesn’t leak subprocesses. Containment is a Windows Job Object, a Linux cgroup v2 (with a POSIX process-group fallback), or a POSIX process group on macOS/BSD — observable via Mechanism. Two caveats the ProcessGroup / Mechanism docs spell out: the guarantee rides on Drop running (a panic = "abort" process, or a SIGKILL/power-loss of the owner, skips it — the OS-owned Job Object / cgroup still reaps on handle close, the POSIX process-group fallback does not), and on the process-group mechanism a child that calls setsid escapes containment. The whole tree can be signalled (ProcessGroup::signal, see Signal), paused/resumed (ProcessGroup::suspend / ProcessGroup::resume), and inspected (ProcessGroup::members); wait_any races several running processes and reports the first to exit.
runner — async run-and-capture built on the group. Describe a run with Command, then drive it to completion (Command::output_string, Command::run, …) or start it for streaming and interactive I/O. The ProcessRunner trait runs commands to completion and is the mock seam (see ScriptedRunner). A Supervisor keeps a command alive — restarting it per policy with backoff — where Command::retry merely replays one run to success. Readiness probes (RunningProcess::wait_for_line / wait_for_port / wait_for) wait until a started child is actually ready instead of sleeping. A Pipeline (Command::pipe) chains commands stdout→stdin without a shell — one shared group, pipefail outcome. Command::cancel_on ties a run to a CancellationToken: cancelling it kills the tree and every consuming path resolves to Error::Cancelled. Spawn-time sandboxing knobs: Command::inherit_env (env allow-list), Command::uid / Command::gid (Unix privilege drop), Command::setsid, Command::create_no_window.

Async throughout (tokio). Errors are the structured Error; a non-zero exit is reported in ProcessResult, not raised, until you call ProcessResult::ensure_success.

Stability. Since 1.0, processkit follows Semantic Versioning: the public API is stable, and any breaking change lands only in a new major version, so 1.x upgrades are backward-compatible. (The lone exception is the mock feature’s mockall-generated expect_* surface — see below.)

Beyond this page, the repository ships a narrative guide set — a task-oriented cookbook (“I want to …” → snippet), a deep guide per capability, and every per-platform caveat collected in one place.

Run vocabulary — one verb, one meaning, at every layer (Command, ProcessRunner/ProcessRunnerExt, CliClient):

run — require an accepted exit (0 by default, widened by Command::ok_codes) and return stdout as a String, trailing whitespace trimmed (trim_end: the final newline is noise, but leading whitespace can be significant). run_unit — the same, discarding the output.
output_string / output_bytes — return the full ProcessResult (stdout as text / raw bytes); a non-zero exit is not an error here. (output_string, not a bare output, since std::process::Command::output yields bytes — the explicit name avoids that footgun and is spelled the same on every layer.)
exit_code — the exit code, with a missing code surfaced as an error. (On a ProcessResult, code is the plain Option<i32> accessor — None for a timeout/signal kill, never a -1 sentinel.)
probe — run a predicate and read its exit code as a bool: 0 → true, 1 → false, anything else is an error (git diff --quiet, …).

§Features

Every flag is additive and gates visibility only — the kill-on-drop tree guarantee is unconditional in every configuration.

stats — resource measurement: ProcessGroupStats, ProcessGroup::stats (plus the sample_stats time-series sampler), the per-process RunningProcess::cpu_time/peak_memory_bytes diagnostics, and the RunningProcess::profile run summary. Opt-in for its specialized purpose (on Windows it calls the system ProcessStatus/PSAPI API — a link to an OS library, not an added crate dependency); enable with features = ["stats"], or limits, which implies it. (The features that do pull an extra crate are mock → mockall, tracing → tracing, and record → serde/serde_json.)
process-control (default) — tree control beyond contain+kill: Signal and ProcessGroup::{signal, suspend, resume, members, adopt}.
limits — whole-tree resource caps: ResourceLimits, the memory_max/max_processes/cpu_quota builders on ProcessGroupOptions, and Error::ResourceLimit. Implies stats.
mock — the mockall-generated testing::MockRunner for consumers’ tests. Its expect_* surface is generated by mockall and is exempt from this crate’s semver guarantees — it tracks the mockall version (an implementation detail) rather than a frozen API. The first-class doubles (ScriptedRunner / RecordingRunner) are the stable, recommended seam; reach for mock only if you specifically want expectation-style mocking.
tracing — tracing events on the processkit target: spawn and exit (program/pid/mechanism), timeout and cancellation firing, group terminate/shutdown, retry attempts, supervisor restarts and storm pauses, and teardown anomalies (stdin-writer failures, pump overruns). Never logs argv or environment values.
record — record/replay cassettes over the ProcessRunner seam: RecordReplayRunner records real Invocation → ProcessResult pairs to a JSON fixture once, then replays them hermetically — no subprocess in CI. Pulls in serde + serde_json.

§Other languages

Not on Rust? processkit-py is a Python wrapper (PyO3 bindings) over this crate’s core, with an asyncio-facing API. This crate remains the single source of truth for the containment/runner logic underneath.

Modules§

testing: Test doubles for the ProcessRunner seam: a ScriptedRunner that serves canned replies, a RecordingRunner that asserts on invocations, the Invocation it captures, and (behind features) record/replay cassettes and a mockall mock.

Macros§

cli_client: Scaffold a typed CLI-wrapper struct around a CliClient.

Structs§

CancellationToken: Re-exported so callers can use processkit::CancellationToken; without a direct tokio-util dependency. See Command::cancel_on. A token which can be used to signal a cancellation request to one or more tasks.
CliClient: Owns a CLI tool’s program name, ProcessRunner, and default timeout, and builds + runs Commands against them.
Command: A description of a child process to launch: program, arguments, working directory, environment, stdin source, and an optional timeout.
Encoding: Re-exported from encoding_rs so a caller can name the type passed to Command::stdout_encoding/stderr_encoding without a direct dependency. Semver note: this is a flat re-export of a 0.x dependency’s type, so a major bump of encoding_rs is a breaking change here, and use processkit::* pulls Encoding into scope (a glob-collision risk). A future major version may move it behind a dedicated module. An encoding as defined in the Encoding Standard.
Finished: The outcome of a run driven via stdout_lines or output_events: how the run ended plus the captured standard error. Returned by RunningProcess::finish.
JobRunner: The default runner: every run gets a fresh, private ProcessGroup owned by the run, so its tree is torn down when the run finishes (or its handle drops).
OutputBufferPolicy: Caps how many captured/streamed output lines are retained in memory.
OutputEvents: A merged Stream of both stdout and stderr lines (see RunningProcess::output_events).
OutputLine: One decoded line carried by an OutputEvent.
Pipeline: A chain of Commands connected stdout→stdin — built with Command::pipe, extended with pipe, driven with the same verb vocabulary as a single Command: output_string / output_bytes for capture, run / run_unit / checked for success-checked runs, exit_code / probe for the code, and parse / try_parse for typed output — each operating on the pipefail outcome. Bound the whole chain with timeout / cancel_on.
ProcessGroup: A container that ties the lifetime of a child-process tree to its own.
ProcessGroupOptions: Tuning for a ProcessGroup — graceful-shutdown timing and (with the limits feature) resource limits.
ProcessGroupStatsstats: A snapshot of a process group’s resource usage.
ProcessResult: The captured result of running a process to completion.
ProcessStdin: An interactive writer to a child’s standard input.
ResourceLimitslimits: Resource limits enforced on a process group as a whole.
RetryPolicy: How a retryable run is re-attempted: a bounded number of retries with exponential backoff, a per-delay cap, and optional full jitter.
RunProfilestats: Resource summary of one finished run — produced by RunningProcess::profile.
RunningProcess: A handle to a process spawned by a runner.
StatsSamplerstats: A periodic ProcessGroupStats series — created by ProcessGroup::sample_stats.
Stdin: What to feed a child process on standard input.
StdoutLines: A Stream of the child’s standard-output lines (see RunningProcess::stdout_lines).
SupervisionOutcome: What a finished supervision reports — the last run plus the keeper’s telemetry.
Supervisor: Keeps a Command alive: runs it, classifies every exit against the RestartPolicy and the stop_when predicate, and restarts it after an exponential-backoff delay until supervision ends.

Enums§

Error: Errors produced when launching or running a child process.
Mechanism: The containment mechanism actually in effect for a process group.
Outcome: How a run ended — the explicit form of the code()/timed_out() pair.
OutputEvent: An event produced by a child process: a decoded line from stdout or stderr.
OverflowMode: What to drop when a bounded output buffer is full.
RestartPolicy: When the supervisor restarts an exited child. See each variant; in every case stop_when and max_restarts can end supervision first.
Signalprocess-control: A signal to broadcast to every process in a ProcessGroup via signal.
StdioMode: How a child process’s standard output or error stream is connected.
StopReason: Why supervision ended.

Traits§

IntoCommand: What a CliClient verb accepts: either an argument list — built into a Command for the client’s program with its defaults (timeout, env, cancellation) applied — or a ready-made Command, run as-is.
ProcessRunner: Runs a Command — to a captured result (output_string / output_bytes) or a live handle (start).
ProcessRunnerExt: Convenience methods available on every ProcessRunner (including &dyn ProcessRunner), layered over output_string.
StreamExt: Re-exported from tokio_stream so callers consume the stdout/event streams (e.g. StdoutLines, OutputEvents) without a direct tokio-stream dependency. Semver note: a flat re-export of a 0.x dependency’s trait — a major bump of tokio-stream is breaking here, and use processkit::* pulls StreamExt into scope (it collides with futures::StreamExt / tokio_stream::StreamExt under a glob). Prefer importing it by path, or the upstream trait directly. A future major version may move it behind a module. An extension trait for the Stream trait that provides a variety of convenient combinator functions.

Functions§

output_all: Run every command in commands, keeping at most concurrency of them live at once, and collect all their results in input order.
output_all_bytes: The raw-bytes companion to output_all: captures each command’s stdout as Vec<u8> instead of decoded text. All other semantics are identical — see output_all.
output_string: Run program with args inside a private job and capture the result without erroring on a non-zero exit — for commands whose exit code is meaningful.
run: Run program with args inside a private job and return trimmed stdout, or an Error on a non-zero exit / spawn failure / timeout. A thin shim over Command; use the builder for a working directory, env, stdin, timeout, or the full verb vocabulary.
wait_all: Wait for all of several running processes to exit, returning their Outcomes in the same order as processes. The processes are only borrowed and stay usable afterwards (the exit status tokio caches remains re-readable).
wait_any: Wait for whichever of several running processes exits first, returning its index in processes and its Outcome (matching RunningProcess::wait).

Type Aliases§

Result: Crate result alias.