# agentplane
**A durable, replayable, policy-governed runtime for AI agents β in Rust.** π¦
[](https://crates.io/crates/agentplane)
[](https://docs.rs/agentplane)
[](#-license)
[](#-status)
[](#-status)
**[Documentation](https://hupe1980.github.io/agentplane/docs/) Β·
[Getting started](https://hupe1980.github.io/agentplane/docs/getting-started/) Β·
[API reference](https://docs.rs/agentplane) Β·
[What will move](https://hupe1980.github.io/agentplane/docs/status/)**
Not a prompt framework. Not an agent library. The layer *beneath* those β the
thing that makes an agent's actions survivable, auditable, and governable when it
is calling real systems that move real money.
```rust
// Performs its effects once, and journals everything.
let outcome = runtime.run("reconcile", Tainted::trusted(input)).await?;
// Replay re-executes the logic and reads every effect back from the journal.
// No tool is called again. No clock is read again. No invoice is issued twice.
runtime.replay(outcome.run_id, Mode::Strict).await?;
```
---
## π₯ The problem
Production agents fail in ways a better model does not fix:
- A 40-minute run dies at minute 38, and the retry re-issues every invoice.
- *"Why did the agent refund β¬4,200?"* has no answer, because the reasoning was
prose in a log line.
- Untrusted tool output steers the next tool call.
- A prompt change ships with no way to know what it broke.
These are **runtime** problems. agentplane is a runtime.
## π‘ The idea
> **The journal is the plan of record.** Orchestration is deterministic and
> replayable. Everything non-deterministic β model inference, tool calls, the
> clock, randomness β is an *effect*: performed at most once, written to an
> append-only hash-chained log, and read back on replay.
Get that right and six things fall out of **one** mechanism: crash recovery,
audit, cost accounting, regression testing, tamper evidence, and regulatory
record-keeping. They stop being six subsystems that can each rot independently.
And critically: **the audit trail is also the recovery mechanism**, so it cannot
quietly stop working β the system would stop working with it. Logging that exists
only to satisfy an auditor always rots.
## π Try it
```sh
cargo run --example hello_skill # one skill, one run, one replay β start here
cargo run --example durable_pipeline # crash, resume, divergence
cargo run --example clearing_case # correlation, obligations, human tasks
cargo run --example plan_graph # multi-step plans, contract, provenance
cargo run --example governed_transfer --features manifest
# field provenance, protected arguments
cargo run --example saga_checkout # reverse compensation, replay-safe unwind
cargo run --example effect_group # calls that take together, or not at all
cargo run --example memory_run # private/team memory, provenance, recall
cargo run --example batch_run # one act, many items: a resume that
# re-settles nothing, and partial failure
# as a terminal state
cargo run --example budget_pause # a ceiling pauses the run; a raise
# resumes it, on the record, nothing repeated
cargo run --example answered_doubt # a call nobody can account for: a person
# supplies the fact, the runtime keeps the
# verdict, and giving up leaves a finding
cargo run --example operator_stop # cancel a run and it unwinds; halt a
# tenant, an agent or one revision and
# nothing new starts; withdraw a
# credential and the runs acting for it
# pause, work intact, until you lift it
cargo run --example observability # the last mile: latency without replays,
# gauges from the census, one alert
# predicate β and the OTLP wiring
cargo run --example recovered_run # an instance dies mid-run; the survivor's
# sweep finds it and finishes it
cargo run --example bedrock_live --features bedrock
# env-gated Amazon Bedrock Converse call
cargo run --example openai_live --features providers
# env-gated OpenAI Responses call
cargo run --example tool_loop --features redb,fake-model,manifest
# a model choosing tools, and four refusals
cargo run --example approved_call --features redb,fake-model,manifest
# a person approves the exact call β
# suspend, worklist, approve or refuse
cargo run --example planned_run --features redb,fake-model,manifest
# plan once, execute without the model β
# a prompt injection with no reader, and
# an invented recipient refused
cargo run --example camel_live --features redb,providers,manifest
# the same, against two real models: a
# privileged planner and a quarantined
# extractor (env-gated)
cargo run --example sealed_run --features redb,testkit,keyring
# erase a case: every copy unreadable,
# and the chain still verifies
# Calls a model and replays without calling it again β no API key, no network.
cargo run --example model_run --features redb,fake-model
# Digest-only multimodal dispatch and zero-I/O replay β also fully offline.
cargo run --example media_run --features redb,fake-model,media
# An agent whose prompt, model, result shape and ceilings come from a file.
cargo run --example manifest_run --features redb,fake-model,manifest
# A real MCP server in this process beside a typed Rust tool β one agent
# reaching both, and a strict replay that calls neither.
cargo run --example mcp_tools --features redb,fake-model,manifest,mcp
# Four agents, one plane: a coded editor that dictates the sequence, and a
# YAML desk that consults the same specialists as tool://agent/... grants.
cargo run --example blog_room --features redb,fake-model,manifest
# This plane served as an A2A 1.0 agent, called the way a peer would call it:
# a public card, authenticated methods, and a message that arrives untrusted.
cargo run --example a2a_peer --features redb,a2a-server,manifest
# Two planes in one process: a served reviewer and a desk that consults it
# through `cx.call_peer` β the peer sees the run's chain plus one link, and a
# strict replay of the desk's run never reaches the reviewer.
cargo run --example peer_call --features redb,testkit,manifest,a2a,a2a-server
# Live tokens for a human, one journaled completion for the machine β and a
# replay that performs neither.
cargo run --example streaming_run --features redb,fake-model
# One customer's approved β¬500, spent across two separate runs, then revoked β
# with the terms still readable afterwards.
cargo run --example standing_authority --features redb,fake-model
```
Or skip Rust entirely β a file and a key are the whole agent, and a file may
hold a whole **room**: several manifests separated by `---`, the Kubernetes
packaging convention. Each document keeps its own digest β the file is
packaging, not identity β and a run starts at the room's declared orchestrator:
```sh
cargo install agentplane --features cli
agentplane run examples/summariser.yaml --input '{"ticket": "printer on fire"}'
agentplane run examples/room.yaml --input '{"topic": "durable execution"}'
```
Or without a Rust toolchain at all β needing one to run a YAML file rather
defeats the point of the file:
```sh
docker run --rm -v "$PWD/examples:/work:ro" ghcr.io/hupe1980/agentplane \
run /work/summariser.yaml --input '{"ticket": "printer on fire"}'
```
Distroless, nonroot, no shell. It runs `--read-only --network none` because the
default journal is genuinely in memory and the example's provider is the
deterministic fake β so the first run needs neither a disk nor the internet.
`:slim` (the default, and `:latest`) carries every model provider β Anthropic,
OpenAI, Gemini, Bedrock, any OpenAI-compatible server; `:full` adds MCP, the A2A
peer server, the operator HTTP API, Cedar, key rings, governed media and
Postgres. Both are multi-arch, cosign-signed keylessly, and carry SLSA build
provenance and an SBOM attached to the digest:
```sh
cosign verify ghcr.io/hupe1980/agentplane:slim \
--certificate-identity-regexp 'https://github.com/hupe1980/agentplane/.*' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
gh attestation verify oci://ghcr.io/hupe1980/agentplane:slim -R hupe1980/agentplane
```
`durable_pipeline` prints the whole claim in four steps: a live run, a strict
replay that touches nothing, a crash that resumes without repeating work, and a
changed build that is **quarantined instead of quietly rewriting history**.
A first program is one import and about forty lines:
```rust
use agentplane::prelude::*;
```
And when it goes wrong, the plane answers with what it *does* have rather than
with a variant name β `fn main()` reports through `Debug`, so on the errors you
hold, `Debug` is the message:
```text
Error: no skill provides capability 'demo.greeet' β this plane provides:
demo.greet. `run` takes a capability, not a skill name; a skill declares its own
with `SkillDescriptor::new(..).provides(..)`
```
A declarative **tool loop** runs from a file too β the manifest grants
`tool://tickets/read`, and which transport reaches `tickets` is deployment
wiring rather than part of the reviewed declaration:
```sh
agentplane run examples/tool-calling.yaml --input '{"ticket": "T-1"}' \
--mcp "tickets=python3 examples/mcp-server.py"
```
A grant naming a server nobody wired is refused at **build**, not on every run.
Needs `--features cli,mcp-stdio`, or the `:full` image.
And an agent can be **hosted** from the same file β the A2A 1.0 server that
passes the protocol project's own conformance kit, started without writing Rust:
```sh
agentplane serve examples/served.yaml \
--url http://localhost:8080 \
--policy examples/serve-policy.cedar \
--tokens examples/serve-tokens.yaml \
--store ./served.redb
```
`--store` takes a redb file or a `postgres://` connection string, and `--tenant`
says whose plane β one flag apart, because on the shared store several instances
coexist and so do an operator's verbs and a serving process.
Add `--operator-addr 127.0.0.1:9090` and the operator surface is served too, on
its **own** listener, off unless asked for, and separated from the peer surface
by *policy* (`peer` reaches `a2a:*`, `operator` reaches `api:*`) rather than by
the port. It is an HTTP API rather than a console: **anything holding a
delegation that carries the verbs can drive it**, and who is acting comes from
the authenticated identity, never from the request body β the decision type has
no actor field to spoof. It serves the worklist and task
decisions, plus the backlogs an on-call person asks for by question rather than
by id: what is quarantined, what is
escalated, which obligations were missed, which messages reached nobody, which
webhook receivers stopped accepting, **what is executing right now** and **what
is stopped**
([the full table](https://hupe1980.github.io/agentplane/docs/operations/#what-the-endpoints-are-for)).
`GET /runs/live` carries the agent, the revision and the delegation subject
beside each id, because an incident is rarely *cancel this run* but a bad deploy
or a credential somebody has just withdrawn.
Every backlog that is *work* has a verb that empties it, including
[the hard one](https://hupe1980.github.io/agentplane/docs/operations/#answering-a-quarantine)
β a run stopped on an effect nobody can account for. The one listing with no verb
is dead letters, because it is a *diagnosis* rather than a queue: the fix is a
correlation key in somebody's emitter, so it is ordered newest-first. A served
plane also sweeps deadlines, task expiry, dead letters, due timers **and
abandoned runs** β a lease that expired while still naming an owner is an
instance that died holding the run β so a run that sleeps, waits or loses its
instance actually finishes.
And it **drains** on `SIGTERM`: stop accepting, answer what is in hand, close
admission, and give the runs still executing `--drain-secs` to reach a journaled
resting point. A process killed mid-call leaves an effect nothing can decide the
outcome of, and a rolling deploy should not be the ordinary way a plane produces
those ([stopping an
instance](https://hupe1980.github.io/agentplane/docs/operations/#stopping-an-instance)).
Both `--policy` and `--tokens` are required and have no defaults. That is the
design rather than an inconvenience: a permissive engine and no engine are the
same behaviour, and a server that authenticates nobody has no actor to record a
decision against. A token may carry its caller's own `scope` and `not_after`;
every run that caller starts is then admitted under a chain rooted at the
caller β checked against the plan, refused once expired β and the journal
names the caller, never the plane, as who the run acted for. Needs
`--features cli,a2a-server,cedar`, or the `:full` image.
New here? β **[docs/getting-started.md](https://hupe1980.github.io/agentplane/docs/getting-started/)**
## π¦ What you get
| π§Ύ | **A journal you can audit** β append-only, hash-chained, per-record signatures naming the workload that wrote them, and a per-plane Merkle log so deleting a whole run is detectable |
| β±οΈ | **Durable execution** β crash mid-run and resume from the last completed effect. Recovery is *initiated*, not merely possible: a sweep finds every run whose owner died holding it and takes it over, and a scheduled stop drains rather than becoming a crash |
| ποΈ | **Cases, not long-lived workflows** β runs stay minutes, business processes span months, so a deploy never migrates an in-flight workflow. Admission claims an idempotency key in the transaction that writes the first record, so a redelivery is answered with the original run |
| π‘οΈ | **Policy before live dispatch** β a total, I/O-free gate; denials are journaled, strict replay never re-judges history, and plan authority is checked before step 1 |
| π·οΈ | **Field-level information flow** β outbound arguments carry hierarchical provenance, so an authority-bearing field can require a trusted or named source while ordinary content stays untrusted. Volume is the axis a label lacks, so the size crossing each sink is journaled and `max_egress_bytes` bounds it |
| πΈ | **Budgets and tenant quotas that bind** β a failed model call is billed for what it burned, because the provider bills for it too, and a replayed run reaches the same tally at the same point β [budgets](https://hupe1980.github.io/agentplane/docs/plans-cases/#budgets) |
| 𧬠| **Effects that take together, or not at all** β each reversible member records the concrete call that undoes it, built from what that call *actually returned*; an irreversible send is **deferred** to commit, so an aborted group never sends it β [effects](https://hupe1980.github.io/agentplane/docs/effects/) |
| π€ | **Human oversight on the *call*, not a summary of it** β a task carries the exact tool and arguments about to be dispatched, and a read-only `preview` puts *four thousand records* on the reviewer's screen instead of `older_than: "2024-01-01"` β [worklists](https://hupe1980.github.io/agentplane/docs/plans-cases/#human-tasks) |
| π | **Erasure that reaches the backups** β payload bytes are sealed under a per-case key the crate never holds, so erasing a case destroys the key and last hour's backup with it. The chain commits to the **ciphertext**, so an auditor with no keys still verifies the run β and a legal hold refuses the sweep for a matter you must preserve β [erasure and keys](https://hupe1980.github.io/agentplane/docs/erasure/) |
| π | **An agent that is only a file** β `agentplane run agent.yaml`. No Rust, no `main`, no skill. The digest covers the agent *in its entirety*, and the run is journaled and deterministically replayable |
Ten rows, not the inventory. The full surface β the export/audit/restore
toolchain, a durable manifest registry with an enumerable inventory, typed
release, standing authorities, effect groups that commit with the journal,
batch runs over 10β΅ items with per-item journals and an item-granular resume,
the scoped emergency stop, the audited sweeper, a scheduled recovery drill, a
retention pass that says what it could not reach and the legal holds that stop
one, model drivers and streaming, MCP and A2A on both sides, signed Agent Cards,
governed media and memory, multi-tenancy, quotas, witnessing, break-glass, and
why there is no `AllowAll` anywhere β is documented mechanism by mechanism on
the site:
**[what you get, in full](https://hupe1980.github.io/agentplane/docs/)**.
What is deliberately **not** built, and what will move β
**[docs/status.md](https://hupe1980.github.io/agentplane/docs/status/)**
## π Documentation
| π | [Getting started](https://hupe1980.github.io/agentplane/docs/getting-started/) β first run, first skill, first replay |
| π£ | [Your first agent](https://hupe1980.github.io/agentplane/docs/first-agent/) β a step-by-step tutorial: one agent, from an empty file to a durable, tool-using, pinnable declaration, no Rust required |
| π§ | [Concepts](https://hupe1980.github.io/agentplane/docs/concepts/) β the ideas the rest is built from |
| ποΈ | [Architecture](https://hupe1980.github.io/agentplane/docs/architecture/) β the determinism boundary, the module layout, and where each mechanism lives |
| βοΈ | [The effect protocol](https://hupe1980.github.io/agentplane/docs/effects/) β at-most-once outward calls, unknown outcomes, sagas, transactional groups, stopping a run |
| π§Ύ | [The journal](https://hupe1980.github.io/agentplane/docs/journal/) β the hash chain, the signatures and the Merkle log, and the claims they refuse to make |
| πΊοΈ | [Plans, cases and time](https://hupe1980.github.io/agentplane/docs/plans-cases/) β frozen authorization graphs, month-long cases, waits, timers, budgets, worklists |
| π | [Models, agents and peers](https://hupe1980.github.io/agentplane/docs/interop/) β everything this runtime calls that it does not own |
| π¦ | [Publishing and pinning agents](https://hupe1980.github.io/agentplane/docs/registry/) β the manifest as an artifact, and a registry that will not rewrite a version |
| π³ | [Cookbook](https://hupe1980.github.io/agentplane/docs/cookbook/) β task-shaped recipes, including wiring an MCP server beside typed tools |
| π | [Manifest reference](https://hupe1980.github.io/agentplane/docs/manifest/) β every field, what enforces it, and what an absent value means; the [published JSON Schema](https://hupe1980.github.io/agentplane/agent.schema.json) gives editors autocomplete and inline errors via one modeline |
| π§ͺ | [Testing agents](https://hupe1980.github.io/agentplane/docs/testing/) β the fake provider, fault injection, and proving a replay actually replayed |
| π¬ | [How this is proven](https://hupe1980.github.io/agentplane/docs/assurance/) β model-checked specifications, mutation-tested specs, and every guarantee broken on purpose |
| π | [Record format](https://hupe1980.github.io/agentplane/docs/format/) β the normative wire specification: canonical JSON, the chain, the Merkle log, the export file. Enough to verify a history without this crate |
| π | [Security model](https://hupe1980.github.io/agentplane/docs/security/) β the trust boundary, and what it does not cover |
| ποΈ | [Erasure and keys](https://hupe1980.github.io/agentplane/docs/erasure/) β erasure that reaches backups, key rotation and revocation, and how tenants are kept apart |
| βοΈ | [Operations](https://hupe1980.github.io/agentplane/docs/operations/) β deploying, HA, retention, observability |
| βοΈ | [Regulation](https://hupe1980.github.io/agentplane/docs/regulation/) β EU AI Act obligation by obligation, and what is missing |
| π | [Status](https://hupe1980.github.io/agentplane/docs/status/) β what is pre-alpha, what to pin, what is deliberately absent |
| β¬οΈ | [Upgrading](https://hupe1980.github.io/agentplane/docs/upgrading/) β what breaks between pre-alpha releases, and the shortest correct fix |
| π | [Changelog](CHANGELOG.md) β what changed and when, including every mechanism's reasoning as it landed |
| π€ | [Contributing](CONTRIBUTING.md) β the assurance ladder, and how to run it |
## π§ͺ Assurance
Each layer answers a question the others structurally cannot.
```sh
just # list every check
just ci # lint Β· every feature alone Β· tests Β· examples Β· docs Β· packaging
just ci-full # the above, plus TLA+ specs and the full mutation sweep
python3 tools/mutants.py <name> --verify # break one guarantee, run its test
```
Two are unusual enough to name:
**π¬ Formal specs.** TLA+ specifications are model-checked on every push β the
effect protocol, effect groups, retry safety, sagas, fencing, authorization,
delegation. And because a spec whose invariants cannot be violated proves
nothing, each is re-checked against deliberately broken copies of itself; every
mutant must be caught by the *specific* invariant written for it.
**π A second reader of the record format.** The
[format specification](https://hupe1980.github.io/agentplane/docs/format/) is
normative prose, and `tools/verify_export.py` is written from it and reads none
of this crate's Rust β enforced by a guard, because a verifier that consulted
`src/` would agree with the implementation by construction. `just verify-golden`
runs it: it **re-derives** all 27 record vectors from their parsed values with
its own canonicalizer and chain digest, verifies the sealed export end to end,
and then damages that export six ways and asserts each is reported. Vectors a
project generates and then checks are that project agreeing with itself; this
is the part that is not.
**π§― A recovery drill, not a backup.** The restore path is exercised against a
real `PostgreSQL` server β restoring one tenant's history into another tenant of
a database somebody else is already using, which is the shape a disaster
actually puts an operator in. It asserts equal roots at equal size, records
hash-for-hash, the matter with its obligation and its artifact, isolation in
both directions, and a first lease past the journal's highest epoch. Then it
asks the restored plane to take **new work**, and checks that the seal extends
the log it restored: a store that reads correctly and cannot be written to is a
backup. The RPO/RTO tables, and the list of what an operator re-establishes by
hand, are on the
[operations page](https://hupe1980.github.io/agentplane/docs/operations/#disaster-recovery).
**π An anchor from a party this plane does not control.** The hash chain, the
signatures and the Merkle log all draw both halves of their comparison from
the store, so an operator who removes a run and recomputes the tree satisfies
every one of them β and `agentplane audit` says so rather than reporting a
clean history. `RuntimeBuilder::witnesses(..)` submits each checkpoint to
witnesses over C2SP `tlog-witness` on the periodic sweep, and `agentplane
audit --witness <prefix> --witness-key <name>=<key>` reads back what they
hold. That second direction is the one that matters: the anchor reaches a
reader who did not get it from the operator, and two witnesses holding one
tree size with two different roots is a split view no single anchor exhibits.
**π§Ύ Conformance by the protocol's own kit.** `just test-a2a-tck` runs the
official [a2a-tck](https://github.com/a2aproject/a2a-tck) against this crate's
A2A server on a live socket. Every other A2A test drives this server with this
crate's own client, which proves symmetry, not conformance β a client and
server written from the same misreading agree everywhere. The kit's first run
found five defects no in-repo test could reach.
**π Tests against a real provider.** `just test-live` runs the OpenAI, Gemini
and `OpenAI`-compatible drivers, plus the embedding wire, against the actual
APIs. They are gated twice β an explicit `AGENTPLANE_LIVE=1`
*and* a key β because a credential being available is not a decision to spend
money with it, and they are never part of `ci`. They exist because a stubbed
provider is structurally unable to have the defects a real one finds: it never
rejects a malformed request and never returns a shape the driver mis-reads.
What they catch had passed every offline test β including a plan format no
provider with constrained decoding accepts, which left the dual-model execution
kind unable to run for real at all. The Gemini
battery is the sharpest case: a **thought signature** is minted and validated by
Google, so a canned server accepts whatever a fixture tells it to and says
nothing about whether Gemini takes the signature back β the one check that
distinguishes a driver carrying the model's turn verbatim from one rebuilding
it, which is where the rest of the ecosystem has been losing this.
**𧬠Mutation testing over the code.** Every load-bearing guarantee is broken on
purpose, and the test *named for each one* must fail. A mutation caught by some other test is
reported **weak**, not passing β that usually means the guarantee has no test of
its own and is being held up by one that could be rewritten without anyone
noticing what it protected.
This is not decoration. The project shipped an unfalsifiable guarantee once: the
refusal to replan on untrusted data was implemented, tested, and green β and
deleting it would have failed no test, because the fixtures laundered the taint
before it reached the check. It was found by accident. The sweep is so the next
one is not.
It runs on **every push**, sharded ten ways. `MUTANTS_SHARD=k/n` takes a
contiguous slice of a list grouped by the feature set each mutation builds
under, cut on **measured seconds rather than count** β a mutation checked by a
library unit test costs six times one checked in an integration binary, and a
matrix finishes when its slowest job does. Each shard needs its own checkout:
the sweep rewrites source in place.
`just anchors` is the cheap half, and it checks text rather than types: a
mutation still *matching* the code it names does not prove its replacement still
compiles. That is what `--verify` is for.
## π« Non-goals
| Ship a prompt library or IDE | Your prompts; agentplane pins the manifest that governs them by digest |
| Route or proxy model traffic | LiteLLM, Bifrost. **The drivers themselves ship** β what is out of scope is *choosing between them at runtime* |
| Implement a vector database | LanceDB / pgvector behind the `SemanticRetriever` seam; embedding is a journaled effect so the query vector is history rather than a recomputation |
| Ship a built-in tool catalogue | Write a typed `Tool`, or wire an MCP server. The tools other frameworks ship are mostly **provider-hosted** β they run during generation, so the call is never announced, authorized, metered or replayable |
| Replace a deterministic protocol engine | Keep it; agentplane sits *beside* it, never inside it |
| Require Kubernetes | One static binary |
| Train, fine-tune, or serve models | Permanently out of scope |
| Grade output quality | It emits replayable traces; grade them elsewhere |
| Interpret payload contents | Payloads are opaque, and labeled |
| Claim regulatory compliance | It provides technical means; compliance is the deployer's |
**Who should not use this:** a team running three agents against low-stakes data.
The complexity is justified when agents touch money, meters, or regulated
records.
## π Status
**Pre-alpha, pre-release, no API stability.** Breaking changes land without
deprecation. The journal record format and the storage schema will change.
Rust **1.94.1+**. `#![forbid(unsafe_code)]`. One crate, feature-gated: an embedded
[redb](https://github.com/cberner/redb) store by default β pure Rust, two crates
deep, no C toolchain β with everything else opt-in.
Honest framing on regulation: agentplane is not "compliant" and cannot be.
Compliance attaches to a system in a context, assessed by its provider or
deployer. What this gives you is the **technical means** to discharge EU AI Act
Articles 12 and 14 β means that are already load-bearing for recovery and
testing, and therefore cannot quietly rot. [Regulation](https://hupe1980.github.io/agentplane/docs/regulation/) maps
obligation to mechanism, names what is *not* built, and notes that the Digital
Omnibus moved the high-risk dates to December 2027 without amending the
articles.
## π License
MIT OR Apache-2.0, at your option.