Crate processkit

Expand description

processkit — async child-process management for Rust + tokio: whole-tree kill-on-drop (no orphaned subprocesses), run-and-capture, streaming, shell-free pipelines, timeouts & cancellation, and supervision.

Two layers:

ProcessGroup — a kill-on-drop container for a process tree. Every child spawned into the group, and everything those children spawn, dies with the group, so an exiting or panicking owner doesn’t leak subprocesses. Containment is a Windows Job Object, a Linux cgroup v2 (with a POSIX process-group fallback), or a POSIX process group on macOS/BSD — observable via Mechanism. Two caveats the ProcessGroup / Mechanism docs spell out: the guarantee rides on Drop running (a panic = "abort" process, or a SIGKILL/power-loss of the owner, skips it — the OS-owned Job Object / cgroup still reaps on handle close, the POSIX process-group fallback does not), and on the process-group mechanism a child that calls setsid escapes containment. The whole tree can be signalled (ProcessGroup::signal, see Signal), paused/resumed (ProcessGroup::suspend / ProcessGroup::resume), and inspected (ProcessGroup::members); wait_any races several running processes and reports the first to exit.
runner — async run-and-capture built on the group. Describe a run with Command, then drive it to completion (Command::output_string, Command::run, …) or start it for streaming and interactive I/O. The ProcessRunner trait runs commands to completion and is the mock seam (see ScriptedRunner). A Supervisor keeps a command alive — restarting it per policy with backoff — where Command::retry merely replays one run to success. Readiness probes (RunningProcess::wait_for_line / wait_for_port / wait_for) wait until a started child is actually ready instead of sleeping. A Pipeline (Command::pipe) chains commands stdout→stdin without a shell — each stage spawns into its own kill-on-drop ProcessGroup sub-group, with chain-wide teardown fanning the kill across every sub-group, pipefail outcome. Command::cancel_on ties a run to a CancellationToken: cancelling it kills the tree and every consuming path resolves to Error::Cancelled. Spawn-time sandboxing knobs: Command::inherit_env (env allow-list), Command::uid / Command::gid (Unix privilege drop), Command::setsid, Command::create_no_window, Command::priority (CPU-scheduling priority, both platforms), Command::umask (Unix file-creation mask).

Async throughout (tokio). Errors are the structured Error; a non-zero exit is reported in ProcessResult, not raised, until you call ProcessResult::ensure_success.

Stability. Since 1.0, processkit follows Semantic Versioning: the public API is stable, and any breaking change lands only in a new major version, so 2.x upgrades are backward-compatible (the last breaking release was 2.1.0). (The lone exception is the mock feature’s mockall-generated expect_* surface — see below.)

Beyond this page, the repository ships a narrative guide set — a task-oriented cookbook (“I want to …” → snippet), a deep guide per capability, and every per-platform caveat collected in one place.

Run vocabulary — one verb, one meaning, at every layer (Command, ProcessRunner/ProcessRunnerExt, CliClient):

run — require an accepted exit (0 by default, widened by Command::ok_codes) and return stdout as a String, trailing whitespace trimmed (trim_end: the final newline is noise, but leading whitespace can be significant). run_unit — the same, discarding the output.
output_string / output_bytes — return the full ProcessResult (stdout as text / raw bytes); a non-zero exit is not an error here. (output_string, not a bare output, since std::process::Command::output yields bytes — the explicit name avoids that footgun and is spelled the same on every layer.)
exit_code — the exit code, with a missing code surfaced as an error. (On a ProcessResult, code is the plain Option<i32> accessor — None for a timeout/signal kill, never a -1 sentinel.)
probe — run a predicate and read its exit code as a bool: 0 → true, 1 → false, anything else is an error (git diff --quiet, …).
parse / try_parse — run to a clean success and feed the captured stdout to a closure: parse for an infallible closure, try_parse for one returning Result (the JSON-deserialization shape). Send-contract exception: Command::parse / CliClient::parse require F: Send (and T: Send), so the returned future is Send and movable into tokio::spawn; Pipeline::parse deliberately does not require Send — its closure runs inline on the awaiting task rather than across a tokio::spawn boundary, so it accepts strictly more closures, but the resulting future is Send only when F/T happen to be. If you need to move a Pipeline::parse / Pipeline::try_parse call into tokio::spawn, make sure your closure and its output are Send yourself; the compiler won’t require it for you the way it does for Command/CliClient.

§Features

Every flag is additive and gates visibility only — the kill-on-drop tree guarantee is unconditional in every configuration.

stats — resource measurement: ProcessGroupStats, ProcessGroup::stats (plus the sample_stats time-series sampler), the per-process RunningProcess::cpu_time/peak_memory_bytes diagnostics, and the RunningProcess::profile run summary. Opt-in for its specialized purpose (on Windows it calls the system ProcessStatus/PSAPI API — a link to an OS library, not an added crate dependency); enable with features = ["stats"], or limits, which implies it. (The features that do pull an extra crate are mock → mockall, tracing → tracing, and record → serde/serde_json.)
process-control (default) — tree control beyond contain+kill: Signal and ProcessGroup::{signal, suspend, resume, members, members_info, adopt}, plus the enriched MemberInfo member snapshot.
limits — whole-tree resource caps: ResourceLimits, the max_memory/max_processes/cpu_quota builders on ProcessGroupOptions, and Error::ResourceLimit. Implies stats.
mock — the mockall-generated testing::MockRunner for consumers’ tests. Its expect_* surface is generated by mockall and is exempt from this crate’s semver guarantees — it tracks the mockall version (an implementation detail) rather than a frozen API. The first-class doubles (ScriptedRunner / RecordingRunner) are the stable, recommended seam; reach for mock only if you specifically want expectation-style mocking.
tracing — tracing events on the processkit target: spawn and exit (program/pid/mechanism), timeout and cancellation firing, group terminate/shutdown, retry attempts, supervisor restarts and storm pauses, and teardown anomalies (stdin-writer failures, pump overruns). Never logs argv or environment values.
record — record/replay cassettes over the ProcessRunner seam: RecordReplayRunner records real Invocation → ProcessResult pairs to a JSON fixture once, then replays them hermetically — no subprocess in CI. Pulls in serde + serde_json.

§Other languages

Not on Rust? processkit-py is a Python wrapper (PyO3 bindings) over this crate’s core, with an asyncio-facing API. This crate remains the single source of truth for the containment/runner logic underneath.

Modules§

prelude: Re-exports of small vocabulary types from the crate’s 0.x dependencies, kept out of the crate root so use processkit::* doesn’t pull them in (and so a future 0.x major bump of either dependency stays contained to this module rather than the whole crate surface).
testing: Test doubles for the ProcessRunner seam: a ScriptedRunner that serves canned replies, a RecordingRunner that asserts on invocations, the Invocation it captures, a DryRunRunner that renders and echoes commands without spawning them, and (behind features) record/replay cassettes and a mockall mock.

Macros§

cli_client: Scaffold a typed CLI-wrapper struct around a CliClient.

Structs§

CancellationToken: Re-exported so callers can use processkit::CancellationToken; without a direct tokio-util dependency. See Command::cancel_on. A token which can be used to signal a cancellation request to one or more tasks.
CliClient: Owns a CLI tool’s program name, ProcessRunner, and default timeout, and builds + runs Commands against them.
Command: A description of a child process to launch: program, arguments, working directory, environment, stdin source, and an optional timeout.
Finished: The outcome of a run driven via stdout_lines or output_events: how the run ended plus the captured standard error. Returned by RunningProcess::finish.
JobRunner: The default runner: every run gets a fresh, private ProcessGroup owned by the run, so its tree is torn down when the run finishes (or its handle drops).
MemberInfoprocess-control: An enriched snapshot of one member of a ProcessGroup — its pid plus best-effort metadata (parent pid, image name, start time).
OutputBufferPolicy: Caps how many captured/streamed output lines are retained in memory.
OutputEvents: A merged Stream of both stdout and stderr lines (see RunningProcess::output_events).
OutputLine: One decoded line carried by an OutputEvent.
Pipeline: A chain of Commands connected stdout→stdin — built with Command::pipe, extended with pipe, driven with the same verb vocabulary as a single Command: output_string / output_bytes for capture, run / run_unit / checked for success-checked runs, exit_code / probe for the code, and parse / try_parse for typed output — each operating on the pipefail outcome. Bound the whole chain with timeout / cancel_on.
ProcessGroup: A container that ties the lifetime of a child-process tree to its own.
ProcessGroupOptions: Tuning for a ProcessGroup — graceful-shutdown timing and (with the limits feature) resource limits.
ProcessGroupStatsstats: A snapshot of a process group’s resource usage.
ProcessResult: The captured result of running a process to completion.
ProcessStdin: An interactive writer to a child’s standard input.
ResourceLimitslimits: Resource limits enforced on a process group as a whole.
RetryPolicy: How a retryable run is re-attempted: a bounded number of retries with exponential backoff, a per-delay cap, and optional full jitter.
RunProfilestats: Resource summary of one finished run — produced by RunningProcess::profile.
RunningProcess: A handle to a process spawned by a runner.
StatsSamplerstats: A periodic ProcessGroupStats series — created by ProcessGroup::sample_stats.
Stdin: What to feed a child process on standard input.
StdoutLines: A Stream of the child’s standard-output lines (see RunningProcess::stdout_lines).
SupervisionOutcome: What a finished supervision reports — the last run plus the keeper’s telemetry.
Supervisor: Keeps a Command alive: runs it, classifies every exit against the RestartPolicy and the stop_when predicate, and restarts it after an exponential-backoff delay until supervision ends.

Enums§

Error: Errors produced when launching or running a child process.
GiveUpAttempt: What the give_up_when classifier inspects: a crashed run that produced a ProcessResult, or a spawn/IO failure that prevented the child from ever starting (e.g. ENOENT for a mistyped program name) and so never produced one.
LimitKindlimits: Which ResourceLimits field an Error::ResourceLimit failure is about — Memory for max_memory, Processes for max_processes, Cpu for cpu_quota.
LimitReasonlimits: Why a requested resource limit could not be applied — the classification an Error::ResourceLimit failure carries so a caller (e.g. the processkit-py binding) can branch on the kind of failure without parsing the English detail text.
LineTerminator: How the output pump decides where one captured/streamed line ends.
Mechanism: The containment mechanism actually in effect for a process group.
Outcome: How a run ended — the explicit form of the code()/timed_out() pair.
OutputEvent: An event produced by a child process: a decoded line from stdout or stderr.
OverflowMode: What to drop when a bounded output buffer is full.
ParentDeathCleanup: The reach of the parent-death hardening (Command::kill_on_parent_death) when the owning process dies abruptly — a SIGKILL of the owner or a crash, where Drop never runs to tear the containment group down.
Priority: A portable CPU-scheduling priority for a child process (see Command::priority), mapped onto the native primitive at spawn time: Unix setpriority/nice (applied through the same pre_exec seam as uid/gid), Windows priority class (OR’d into creation_flags, the same seam as create_no_window).
RestartPolicy: When the supervisor restarts an exited child. See each variant; in every case stop_when and max_restarts can end supervision first.
Signalprocess-control: A signal to broadcast to every process in a ProcessGroup via signal.
StdioMode: How a child process’s standard output or error stream is connected.
StopReason: Why supervision ended.

Traits§

IntoCommand: What a CliClient verb accepts: either an argument list — built into a Command for the client’s program with its defaults (timeout, env, cancellation) applied — or a ready-made Command, run as-is.
ProcessRunner: Runs a Command — to a captured result (output_string / output_bytes) or a live handle (start).
ProcessRunnerExt: Convenience methods available on every ProcessRunner (including &dyn ProcessRunner), layered over output_string.

Functions§

output_all: Run every command in commands, keeping at most concurrency of them live at once, and collect all their results in input order.
output_all_bytes: The raw-bytes companion to output_all: captures each command’s stdout as Vec<u8> instead of decoded text. All other semantics are identical — see output_all.
output_string: Run program with args inside a private job and capture the result without erroring on a non-zero exit — for commands whose exit code is meaningful.
run: Run program with args inside a private job and return trimmed stdout, or an Error on a non-zero exit / spawn failure / timeout. A thin shim over Command; use the builder for a working directory, env, stdin, timeout, or the full verb vocabulary.
wait_all: Wait for all of several running processes to exit, returning their Outcomes in the same order as processes. The processes are only borrowed and stay usable afterwards (the exit status tokio caches remains re-readable).
wait_any: Wait for whichever of several running processes exits first, returning its index in processes and its Outcome (matching RunningProcess::wait).
which: Resolve program to a concrete executable path without launching it — a spawn-free preflight for a doctor / early-diagnosis check (“is git installed?”) that must have no side effects. A thin shim over Command::new(program).resolve_program(); use the builder form when you need prefer_local directories or a relocated PATH honored (the client-level CliClient::resolve_program does the same for a wrapped tool).

Type Aliases§

Result: Crate result alias.