Reef
Reef is an open, provider-neutral set of Rust agent primitives.
An agent, here, is a loop: it sends a model everything the conversation has said so far, records what comes back, runs any tools the model asked for, records those too, and goes round again until something ends it. Reef owns that loop and the two things it is made of — the transcript it reads and extends, and the events it publishes while doing so — and it owns the seams that let you change what happens at every boundary.
What reef owns, and what it does not
Reef owns the run lifecycle (Agent / Session / Run), the transcript and
its compare-and-set authority, the outward event plane, the tool registry and
execution engine, typed per-run and per-session state, and the hook
mechanism the seams hang off.
Reef deliberately does not own a provider, an HTTP client, a wire format, an
approval policy, a sandbox, a persistence format, a compaction strategy, or a
configuration file. Those are yours. They are written against exactly the public
API that cargo doc renders — no consumer of it here has private access — and
that is the point rather than a gap: a primitives layer whose composition
layer is the user's is one you can put your own product policy into without
forking it.
If you want a batteries-included framework, reef is not it. If you want to own the policy and not the loop, it is.
Status
0.4.0 is the release represented by this source tree. Its public surface is
frozen in api/reef-core.txt, the snapshot the verification gate compares
against, and the complete gate is ./scripts/check-all.sh. What a 0.x release
may and may not change is stated in the packaged docs/versioning.md.
This release is breaking against 0.3.0 without changing a single public
signature: a mailbox batch whose admission fails is now restored to the mailbox
instead of being destroyed, so a composition that re-delivered such input itself
must drain first, and before_input_admit must be idempotent. See the
migration guide before upgrading.
The gate has been exercised with the pinned toolchains on both
aarch64-apple-darwin and x86_64 Linux. There is no hosted CI: it runs from a
clone on a machine you control, and a green tree means somebody ran it there.
Install
The package is reef-core (the crates.io name reef belongs to an unrelated
crate); the library target is reef_core.
The release line represented by this source is 0.4. Once crates.io lists
0.4.0, select it explicitly:
[]
= "0.4"
= { = "1", = ["macros", "rt"] }
A 0.3 requirement will not resolve to 0.4.0, so upgrading remains an explicit
requirement change. Registry availability is determined by crates.io; this source
tree and its immutable package documentation state what the 0.4.0 artifact
contains rather than claiming that the upload has already happened.
Requires Rust 1.85.0 or newer (edition 2024). The default build uses
futures-core, tokio, and tokio-util with the feature sets declared in
Cargo.toml, and #![forbid(unsafe_code)]. The manifest declares an empty
default feature set; the release gate verifies that configuration separately.
The optional serde feature is off by default and adds serde_core.
Getting started
Everything before Agent::start is assembly, and it happens in order: tools,
hooks, configuration, the agent, a session, then the run. The program below
targets 0.4.0 and follows that sequence. It registers no tools and no hooks, so
the only part you have to supply is the one
reef never supplies — a model::Client, the adapter that sits between the loop
and whatever answers its requests. This one answers offline, so the program needs
no network and no credentials.
use ;
use Pin;
use Arc;
use ;
use cancel;
use Stream;
use ;
use ;
/// The provider. Reef ships none, because a provider is an HTTP client with a
/// wire format, and both of those are yours.
;
/// `futures-core` carries no combinators, so a stream is a `poll_next`.
;
async
Two things in there are obligations rather than style. You must drain the stream
returned by start_observed(): a strong seam blocks the run until the consumer
decides it, and a successful channel send is never mistaken for acceptance. And
Agent::start_observed must be called from inside a Tokio runtime context, because
the run gets its own task. The compatibility Agent::start entry point instead
leaves the unique stream on the Run for one later take_events() call; observable
hosts should prefer the atomic form so a fast Run cannot finish first and discard
an untaken receiver.
That program is compiled and executed by this repository's test suite, so it cannot rot.
Approval, sandboxing, persistence and resume live in the composition layer — another repository, assembled from the public API. This crate ships no sample agent, because a sample agent is one product choice and reef is the bricks.
Ideas that explain most of the API
Synchronous admission. Agent::start and Agent::start_observed finish
attachment, the session lease, the revision/input freeze, the lease's admitted
barrier, the prepare commit and admitted observers before they return. There is
no "admitted later" window, so a consumer at T=0 sees consistent state;
start_observed also transfers the event receiver before the Run task is spawned.
Strong delivery. The three decision points —
event::DecisionPoint::{ToolCall, ToolResult, AfterToolCommit} — are
rendezvous, not notifications. The run does not advance until the
consumer explicitly accepts or rejects. Kind has sixteen variants and no
AfterToolCommit; the rendezvous is the decision point, not the category.
Transcript CAS. The transcript is versioned and mutated through reserve → compare-and-set → publish, with no gap where a reservation is committed but unpublished. A tool batch executes concurrently but commits atomically in source order: all of it or none.
Cancellation and terminals. Parent cancellation, request_stop and dropping
the event stream share one first-nonfatal cause; a fatal can still upgrade it.
When a terminal becomes observable, cleanup has already happened — owners
joined, lease released, events closed.
Composition over configuration. Approval policy, sandboxing, persistence,
provider integration, and dependency-continuity policy are yours. A launcher
that requires the same dependency identities across successor Runs records and
checks them in its own per-Session coordination before Agent::start; a prepare
hook may stage the accepted identity in Session state. That guard is not atomic
with reef's private Session control lock, so the launcher must serialize every
participating start path itself rather than claiming it shares the lease lock.
Core provides the lifecycle and assembly seams.
Lifecycle guarantees
docs/contracts.md states thirty contracts normatively. The guarantees below
shape everything a consumer can observe.
- Admission is a barrier, not a hope (C01–C05).
Agent::startcompletes attachment, the session lease, the frozen revision/input/resources, the no-argument lease admission, the prepare transcript commit and the staged session-state commit, and admitted-observer fan-out, before it spawns the sole run task and returns. - Work commits atomically (C06–C09, C14, C17). Transcript versions are monotonic and a compare-and-set never overwrites a conflict; concurrent tools merge in source order; assistant output and the complete set of results commit together or not at all.
- A terminal is published behind its own cleanup (C23–C25). The sole run task performs the run-end fan-out while the lease is still held, freezes the terminal state, cancels late work, releases the lease, detaches the agent and closes the event stream — and only then makes the terminal observable. A latched fatal outranks a later cancellation, and a task-owned guard performs the same cleanup on a core panic. Every strong seam also confirms authority on the caller side after the consumer decides (C12/C13, C18), so an acceptance that raced a stop or a fatal cannot authorize later execution.
- Close is linearizable and holds the only teardown authority (C29).
Configconsumes the hook builder, the full registry and teardown authority stay private to the agent, revision and run handles expose only close-free data, and one lock orders start, retryableBusy, once-only teardown, concurrent wait and cached close results.
Verification
./scripts/check-core.sh is the single authoritative core verdict: fmt, clippy,
all targets, doctests, rustdoc, MSRV, Loom on the stable and MSRV tracks, Miri,
per-file coverage, direct-dependency/feature policy, cargo-deny, cargo-shear,
the public-API comparison and package contents. ./scripts/check-bench-harness.sh
is the independent composition-harness verdict and owns the closed harness
Criterion policy. Both scopes reject manifest and applicable repository Cargo
source overrides, including indirect includes. The harness verdict also rejects
ambient Cargo target runners and injects a repository-controlled transparent
runner only for its test command, overriding user-global Cargo runner config. It
retains a separate output-manifest proof that its integration tests and every
Criterion ID in the authoritative manifest actually executed.
./scripts/check-all.sh separately reports its aggregate infrastructure, core,
and harness statuses; the truth-table test covers the complete status matrix, and
neither an infrastructure nor harness failure becomes a core failure. All run
from a clone rather than the published crate.
Coverage is enforced per production file at 100% line and branch — there is no aggregate to hide behind. The gate also passes identically under a deliberately hostile ambient environment, because it sets its own authoritative values at each execution point rather than trusting what it inherits.
Current evidence and the commands that produce it are recorded in
docs/verification.md. Volatile totals are printed by the gate rather than
copied into prose. This paragraph used to restate such totals, and some had
already drifted — exactly the failure mode docs/traceability.md avoids.
Where to read next
https://docs.rs/reef-core renders the crate-level documentation, which is the
recommended starting point once the example above makes sense: it walks the same
assembly sequence with links into every type it names. Locally the same page is
cargo doc --all-features --open.
In the repository, docs/README.md indexes the rest by who it is for. The short
version: docs/contracts.md is the behavioural specification, docs/adr/ holds
the decisions with docs/adr/README.md as its index, and docs/verification.md
is the gate's own notes. This repository-only material is not packaged; it lives
beside the source in the repository named by Cargo.toml's repository key,
which is private and requires access to read.
Some material stays in the repository and is not in the published crate: the
verification harness and its evidence (scripts/**, api/**), the scaling
bench harness (benchmarks/**), this repository's issue and dependency
configuration, toolchain pin and lint/format settings (.github/**,
rust-toolchain.toml, clippy.toml, rustfmt.toml, deny.toml), the documents
addressed to whoever works on reef rather than to whoever depends on it
(CONTRIBUTING.md, CODE_OF_CONDUCT.md), and docs/**, which is reasoning about
how this tree reached its shape and is rendered by neither cargo doc nor
docs.rs. Cargo.toml explains each case at its exclude key.
SECURITY.md, AGENTS.md, and docs/versioning.md are packaged deliberately.
They state the security boundary, the criteria that decide the API's shape, and
the compatibility promise respectively. docs/versioning.md is negated back out
of the broader documentation exclusion because a promise readable only
somewhere other than where the version came from is not a promise.
Contributing
CONTRIBUTING.md is worth reading before the first change rather than after it,
because the gate is stricter than most: Loom models on the stable and MSRV
toolchains, Miri on a pinned nightly, 100% line and branch coverage enforced per
production file, and a cargo public-api snapshot that a change to the public
surface has to update deliberately. None of that is discoverable from one
failing run, and no hosted CI will discover it for you. It is not packaged; it
lives beside the source.
AGENTS.md is the other half of the answer, and it is the one that
decides whether a change is accepted at all: it is this project's design
constitution, and it fixes what may enter this crate and what must stay in the
layer above it. It ships inside the published package on purpose, because it is
why the API has the shape it has.
License
Licensed under either of
- Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0; the
text also ships in the package as
LICENSE-APACHE) - MIT license (http://opensource.org/licenses/MIT; the text also ships in the
package as
LICENSE-MIT)
at your option.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.