Rusty Server
The network face of rusty-agent-runtime — serve your agent graphs over HTTP + SSE from a single static binary. No interpreter, no Postgres, no Redis. Dual-licensed under MIT OR Apache-2.0.
Status: v0.5, under active development. The crate ships as a library: you call
rusty_agent_server::serve()from your ownmain.rs. The endpoint set, streaming semantics, and config surface follow the architecture document indocs/rusty-server-design.md. The corerusty-agent-runtimecrate is untouched — it has no HTTP, no axum, no server dependencies, and never learns that a server exists.
New in v0.5 — the Flight Recorder surface. Every run is journaled (core R0.5 kernel): the server attaches a journal to the executor at run start, persists its snapshot at every checkpoint boundary and at run completion (
{store_path}/journals/{run_id}.json, or the auto-migratedserver_journalstable on Postgres), and serves it read-only viaGET /runs/{run_id}/events— fetchable by run id even after the run's in-memory record is evicted or the process restarts. On top of that:GET /runs/{run_id}/fixture(portable CI replay bundle),POST /runs/replay(server-side exact replay with evidence verification), andGET /runs/diff(branch diff of two runs' journals). Fully additive — no breaking changes in this version.
Why one binary instead of three containers
A self-hosted LangGraph Platform standalone deployment needs three moving parts: the API container, Postgres (threads / runs / checkpoints / task queue), and Redis (pub/sub fan-out for background-run streaming) — plus a queue-worker topology for exactly-once background runs. rusty-agent-server collapses that into a single static binary, because the primitives LangGraph rents from infrastructure fall out of rusty-agent-runtime's execution model for free:
| Concern | LangGraph Platform | rusty-agent-server |
|---|---|---|
| User-code loading | langgraph.json + pip install at image build |
Cargo.toml + main.rs, static link |
| Deployment unit | API image + Postgres + Redis (compose) | one static binary |
| Checkpoint store | Postgres | embedded JsonFileCheckpointer (wired from ServerConfig::store_path), or core's PostgresCheckpointer via ServerConfig::with_postgres (feature postgres) |
| Stream fan-out | Redis pub/sub | in-process tokio::sync::broadcast per run |
| Background-run queue | Postgres task queue + workers | in-process per-thread run queue |
| Stream resume | stream_resumable contract |
replay from the per-run in-memory event log, deduped by Last-Event-ID |
| Multi-process scale-out | supported | Phase B gRPC worker protocol (see roadmap) |
The trade is explicit: this is a single-process server. That covers the overwhelming majority of self-hosted agent deployments, and the Node trait keeps the multi-process door open: remote gRPC workers and WASM nodes are planned implementations of the same trait, not architectural changes.
Setup: Cargo.toml is the new langgraph.json
LangGraph's langgraph.json exists because Python can import user modules at runtime. Rust cannot, so in Rust, the declaration of "which graphs this server hosts" is your main.rs, and the dependency list is Cargo.toml. The server is a crate you call, not a binary you load graphs into.
[]
= "0.4"
= "0.5"
= { = "1", = ["full"] }
= "1"
= "0.3"
A realistic main.rs — register a graph under a name, hand the registry to serve:
use Arc;
use *;
use ;
// your code: build_support_graph(), etc.
async
A GraphRegistry entry is a name plus the two things the executor needs — a Graph and its StateSpec — so Executor::run(&graph, &spec, state, config) can be driven for any registered name over HTTP. Registration is compile-checked: a graph whose nodes write channels absent from its spec fails in your CI, not in production.
Dev loop. No langgraph dev equivalent is needed. cargo watch -x run (or bacon run) recompiles and restarts on save; incremental rebuilds of a single-graph binary take seconds. During development, point ServerConfig::new's store_path at a scratch directory you can delete between runs.
Embedding. serve(registry, config) binds and blocks; if you want the routes inside a larger axum application (or want to drive the API in tests via tower::ServiceExt::oneshot), call rusty_agent_server::router(registry, config) instead and merge the returned Router yourself.
Graceful shutdown (R0.6 wave 2c). serve drains on SIGINT/SIGTERM: axum stops accepting connections and waits for in-flight requests; a shared token cooperatively cancels in-flight runs at their next super-step boundary — where a checkpoint was just persisted, so re-running the thread resumes the work — and ends them terminal-cancelled; new run submissions answer 503 shutting_down; the outbox relay finishes its current pass and stops (pending rows publish on the next process's first pass); the cron scheduler stops firing. The whole drain is bounded by ServerConfig::with_shutdown_grace (default 25 s, under Kubernetes' 30 s pod-termination grace); past it the server stops anyway, which is the crash case the checkpoint log and lease expiry already cover. Embedders get the same pieces: serve_with_shutdown(registry, config, future) takes any shutdown future, shutdown_signal() is the SIGINT/SIGTERM default, and router_with_shutdown(registry, config, token) wires the cooperative drain into a self-hosted Router.
HTTP API
An Agent-Protocol-compatible subset — wire-compatible with the core run/thread shapes LangGraph Platform uses, without the commercial surface. This table is the v0.5 endpoint inventory; everything listed here is implemented and covered by integration tests.
| Endpoint | Description |
|---|---|
GET /ok |
Liveness probe → {"ok": true} |
GET /info |
Service version, checkpointer kind, store path, registered graphs + their channels |
POST /threads |
Create a thread bound to a registered graph: {graph, metadata?, thread_id?} → 201 |
POST /threads/{id}/fork |
Time travel: copy the thread's checkpoint history into a new thread: {new_thread_id?, checkpoint_id?} → 201 {thread_id, checkpoints_copied} |
GET /threads/{id}/state |
Latest checkpoint: {values, next, checkpoint} |
POST /threads/{id}/state |
Write a new checkpoint (the update_state analog; optional as_node, next_nodes) → 201 |
POST /threads/{id}/history |
List checkpoints, newest first, with limit / before (400 for an unknown before cursor) |
POST /threads/{id}/runs |
Start a background run → 202 + {run_id, thread_id, status} |
POST /threads/{id}/runs/wait |
Run to completion; returns the terminal JSON ({status, output || interrupt, …}); server-side wait ceiling of 3600 s → 504 on timeout (the run keeps executing — poll GET /runs/{id}) |
POST /threads/{id}/runs/stream |
Run with SSE streaming; a fresh run starts a new frame sequence, so Last-Event-ID is ignored here |
GET /runs/{id}/stream |
Attach to an existing run's SSE stream: replay the event log — honoring Last-Event-ID — then follow live frames until end; 404 for unknown or cross-tenant runs |
GET /runs/{id}/events |
Flight Recorder: the run's journaled RunEvents → {run_id, events, complete}; 404 for unknown or cross-tenant runs; stays fetchable by run id after run eviction/restart via the persisted journal |
GET /runs/{id}/fixture |
Flight Recorder: download the run as a portable ReplayFixture bundle (journal + graph topology hash + final checkpoint) for CI replay; 409 before the first persisted snapshot |
POST /runs/replay |
Flight Recorder: re-drive a journaled run against its registered graph (zero outbound) and verify the replayed evidence → {run_id, verified, expected_events, actual_events, first_divergence}; 404 unknown/cross-tenant, 409 no journal or still executing, 422 graph not registered in this process / effect-carrying or resumed journal |
GET /runs/diff?base=&branch= |
Flight Recorder: structural diff of two runs' journals (core's BranchDiff shape); 404 unknown/cross-tenant either side, 409 when either run has no persisted journal |
DELETE /threads/{id}/runs/{run_id} |
Rollback: delete a finished run's checkpoints, re-anchoring the thread to the pre-run checkpoint (409 while the run is active, while the thread is busy, when the run's checkpoints are no longer the history tail, or on the Postgres backend) |
GET /runs/{run_id} |
Poll a run: {run_id, thread_id, graph, attempt, status}; once terminal the body also carries the run's output / error / interrupt fields (up to 1024 terminal runs retained per process, oldest evicted beyond that) |
POST /assistants |
Create a named graph alias: {name, graph, config?, metadata?, assistant_id?} → 201 (persisted under {store_path}/assistants/) |
GET /assistants / GET /assistants/{id} |
List / fetch assistants |
POST /crons |
Schedule recurring runs: {graph, interval_secs ‖ cron_expr, input?, metadata?, on_run_completed?} → 201 (persisted under {store_path}/crons/) |
GET /crons / DELETE /crons/{id} |
List crons (with runs_fired, last_run_at) / delete a cron (404 when unknown) |
PUT /store/{ns}/{key} |
Upsert a JSON value in a namespace → 201 on create, 200 on replace (created_at preserved) |
GET /store/{ns}/{key} / DELETE /store/{ns}/{key} |
Fetch / delete one item (404 when absent) |
GET /store/{ns} |
List a namespace's items, sorted by key (empty array for an unwritten namespace) |
Not in v0.5 (roadmap, see below): thread listing/deletion endpoints, /metrics, /graphs, the replay-POST Flight Recorder endpoint, and the gRPC worker protocol. Thread records are durable: each thread persists as one JSON file under {store_path}/threads/ (or in the server_threads table with Postgres persistence) and reloads on startup — persistence is what makes the checkpoint durability story reachable through the API, since a restart that forgot the thread records would 404 every pre-restart thread while its checkpoints sat orphaned on disk. Assistants, crons, and store items are likewise durable (JSON files under store_path, or the server_* tables) and reload on startup.
Run-create payload (subset of LangGraph's shape):
assistant_idruns through a named assistant: the assistant must be bound to the same graph as the thread (400on mismatch,404when unknown), and itsconfig.recursion_limitapplies as a default when the payload doesn't set one.command.resumeis the human-in-the-loop channel: it maps directly toRunConfig::with_resume(value). The executor restores the thread's latest checkpoint, re-runs the interrupted node withNodeContext::resume_value()returning the payload, and the run continues. An interrupted run is reported as{"status": "interrupted", "interrupt": <value>, "checkpoint_id": …, "state": …}.checkpoint.checkpoint_idis the time-travel channel: it maps toRunConfig::with_checkpoint_id(id)— the run replays from that checkpoint of the thread (its state and next-node set) instead of the latest.404when the thread has no checkpoint with that id. Prefer replaying on a fork rather than the original thread, so the new history grows on a branch instead of appending to the live timeline. Combines withcommand.resume:checkpoint_idselects where the run restarts,resumesupplies the resume value for the first super-step.config.recursion_limitmaps toRunConfig::with_max_steps(n).stream_modeselects which frame families the SSE endpoint emits; default["values", "updates"].metadata,error, andendframes are always emitted. Add"messages"for LLM token deltas.multitask_strategy— one active run per thread:enqueue(default) queues onto the per-thread run queue (depth-capped byServerConfig::max_concurrent_runs_per_thread),rejectreturns409 Conflict. LangGraph'srollbackstrategy is instead an explicit operation:DELETE /threads/{id}/runs/{run_id}on a finished run.
Client-chosen ids. thread_id / new_thread_id / assistant_id / cron_id (including assistant_id in run bodies) must be non-empty, ≤ 256 chars, free of path separators (/, \), not all dots, and not one of the reserved layout names — assistants, crons, journals, store, threads, latest — which already name directories at the store root (or the latest pointer file inside each checkpoint dir); claiming one would write checkpoints into platform directories. Violations answer 400. Tenant ids in with_tenant_key follow the same reserved-name rule with a 64-char cap, enforced at startup.
Auth. A single static API key checked against the X-Api-Key header (the LangSmith managed-deployment convention), set via ServerConfig::with_api_key("…"). With no key configured (the default), the server runs in dev mode with auth disabled.
CORS. router() layers tower_http::cors::CorsLayer::permissive() as the outermost middleware: every response carries access-control-allow-origin: *, and OPTIONS preflights are answered before the auth middleware runs. That makes browser clients — like the zero-build Studio — work out of the box from any origin, including file://. Production deployments should restrict this: call router() and layer your own restrictive CorsLayer policy on top in your binary (allowed origins, methods, and headers narrowed to your frontend), or terminate CORS at a reverse proxy.
Flight Recorder: run journals
Every run is journaled by the core Flight Recorder (R0.5): super-step boundaries, node inputs/outputs (with declared effect classes and measured latencies), model/tool/remote/WASM calls recorded by node code, interrupts, resumes, routing decisions, and checkpoint writes land in an append-only, hash-chained Journal — one per run, keyed by the server-minted run id. The server attaches that journal to the executor at run start and persists its JournalSnapshot:
- at every checkpoint boundary (flushed from the
CheckpointSavedevent path), so stored evidence trails the live journal by at most one super-step, and - at run completion — success, interrupt, or error; evidence of a failed run is still evidence.
Snapshots persist as one JSON file per run under {store_path}/journals/ (or the server_journals table with Postgres persistence) and are served read-only:
# {
# "run_id": "7c1e…",
# "events": [
# {"id": "7c1e…:0", "run_id": "7c1e…", "thread_id": "3f2b…",
# "node_id": null, "seq": 0, "kind": "super_step_start",
# "effect": "pure", "input": {"kind": "inline", "value": …},
# "output": null, "latency_ms": null, "tokens": null,
# "cost_usd": null, "status": "ok", "parent": null,
# "recorded_at": "2026-08-07T…Z"},
# …
# ],
# "complete": true
# }
The events are core's RunEvents in seq order, in the exact golden-pinned wire shape (rusty-core/tests/golden/run_event.json); event ids are deterministic ({run_id}:{seq}) and parent forms the causal chain. complete is true once the run is terminal — the served snapshot is the final journal. Guardrails: the stored snapshot's chained head hash is re-verified on every read (tampered evidence answers 500, not a plausible-looking lie), and 404/tenant-isolation semantics are identical to GET /runs/{id} — a cross-tenant run is invisible here too. Reachability: journal reads resolve the run through the live run manager first and fall back to the persisted journal — so /events, /fixture, /replay, and /diff keep answering by run id after the run's in-memory record is evicted (past the 1024-run retention cap) or the process restarts, for as long as the store holds the journal; store-fallback reads are served as complete (no live writer remains). Both SDKs expose this endpoint as run_events(run_id) (Python) / runEvents(runId) (TypeScript).
Fixture download — GET /runs/{run_id}/fixture bundles the run for portable replay: the integrity-verified journal snapshot, the graph's topology hash, the run's final checkpoint, and provenance metadata, in core's ReplayFixture envelope (format_version: 1). Feed the JSON to ReplayFixture::import to re-drive the run in CI. A run with no persisted journal yet (queued, or before its first checkpoint) answers 409; 404 and tenant-isolation semantics match GET /runs/{id}, and the served checkpoint's thread_id is always the external one. Server runs record under the system clock and OS entropy, so fixtures carry no logical-clock/RNG-seed parameters — byte-identical CI replay is for runs recorded with determinism seams. SDKs: get_fixture(run_id) / getFixture(runId).
Server-side replay — POST /runs/replay with body {"run_id": "…"} re-drives the journaled run against the graph code registered in this process (zero outbound calls: node code re-executes, but any journaled model/tool/remote/WASM effects make the endpoint refuse — see below) over a throwaway in-memory checkpointer, and verifies the replayed evidence against the recorded journal:
# {"run_id": "7c1e…", "verified": true, "expected_events": 12,
# "actual_events": 12, "first_divergence": null}
verified compares the two journals on the evidence axes — kinds, nodes, sequences, effect classes, statuses, and resolved payloads — excluding per-run minted checkpoint ids and wall-clock measurements (server runs record under the system clock and OS entropy, so byte-identity is the CI-fixture story, not this one). first_divergence is the journal seq of the first disagreeing event (or of the first recorded event the replay never produced). Statuses: 404 unknown or cross-tenant run; 409 no persisted journal yet, or the run is still executing (replay verifies a final journal); 422 when the run's graph is not registered in this process, when the journal carries recorded model/tool/remote/WASM calls (server-side replay cannot serve them — download the fixture and replay it in CI), or when the run resumed from a checkpoint. SDKs: replay_run(run_id) / replayRun(runId).
Branch diff — GET /runs/diff?base=<run_id>&branch=<run_id> returns the structural diff of two runs' journals in core's BranchDiff serde shape as-is: first_divergent_seq, the events added (branch) and removed (base) at and after the divergence point, per-super-step state-channel step_diffs, and token/cost base_totals / branch_totals. Events compare logically (identity and timing fields excluded), so two runs forked from one point show their shared prefix as equal; a run diffed against itself reports first_divergent_seq: null. 404 for an unknown or cross-tenant run on either side, 409 when either run has no persisted journal yet. SDKs: diff_runs(base, branch) / diffRuns(base, branch).
The Studio's compare/replay UI consumes both endpoints with exactly these response shapes; the Python and TypeScript SDKs expose all four Flight Recorder endpoints (run_events/get_fixture/replay_run/diff_runs, camelCase in TypeScript).
Time travel: fork & checkpoint replay
Every super-step boundary is a checkpoint, and every checkpoint is a handle. Two endpoints turn that into LangGraph-style time travel:
Fork a thread's history — POST /threads/{id}/fork copies the source thread's checkpoints (preserving their ids, steps, states, and next-node sets; only the thread_id changes) into a new thread bound to the same graph, via core's Checkpointer::fork_thread:
# Full-history fork
# -> 201 {"thread_id": "9c1e…", "checkpoints_copied": 2}
# Mid-history fork: copy only up to (and including) a checkpoint,
# so the fork branches off at that point in the timeline
# -> 201 {"thread_id": "branch-a", "checkpoints_copied": 1}
Errors: 404 when the source thread (or the checkpoint_id) is unknown, 400 when the source thread has no checkpoints to copy, 409 when new_thread_id is already taken.
Replay a run from a checkpoint — all three run endpoints accept "checkpoint": {"checkpoint_id": "…"}, which maps to RunConfig::with_checkpoint_id(id): the run restores that checkpoint's state and next-node set instead of the latest and continues from there (404 when the checkpoint is unknown):
# Re-run the tail of the graph from an earlier boundary, on the fork
The safe pattern is fork first, replay on the fork: replaying on the original thread appends new history on top of the old timeline (supported, but rarely what you want), while a fork gives the alternate path its own thread id and its own history. Checkpoint ids come from POST /threads/{id}/history, GET /threads/{id}/state, or an interrupted run's checkpoint_id field.
Assistants, crons, and the KV store
The v0.2 platform surface, all durable as JSON files under store_path:
Assistants bind a name plus free-form config / metadata to a registered graph, so clients can create runs by assistant_id instead of repeating a graph name and config. Files live at {store_path}/assistants/{assistant_id}.json and reload on startup.
Crons fire runs on a schedule. POST /crons takes exactly one schedule kind: interval_secs (fixed interval, 1..=31_536_000 s — one year; out-of-range values answer 400) or cron_expr (5-field min hour day-of-month month day-of-week, UTC, minute resolution — parsed with the cron crate). A background scheduler (200 ms tick) fires each due cron by creating a fresh thread bound to the cron's graph and scheduling a background run with the cron's input. Records carry runs_fired / last_run_at bookkeeping and persist at {store_path}/crons/{cron_id}.json. on_run_completed: "delete" turns a cron into a one-shot: it removes itself once its first fired run reaches a terminal state (an in-process tombstone keeps it from re-firing while that first run drains).
Store is a cross-thread key-value memory: PUT /store/{namespace}/{key} writes any JSON value, namespaced items persist at {store_path}/store/{namespace}/{key}.json, and listing a namespace returns its items sorted by key. Namespace and key segments are restricted to [A-Za-z0-9._-] (1–128 chars) to keep the path mapping unambiguous.
Streaming (SSE)
The executor emits one typed event stream — GraphEvent::{SuperStep, NodeStart, NodeEnd, StateUpdate, CheckpointSaved, Token} — and LangGraph's stream modes are filters over that single stream, implemented as such:
stream_mode |
SSE frame | Source |
|---|---|---|
updates |
event: updates — {"step": n, "updates": {channel → post-reducer value}} per step (the full appended list for an Append channel, read back from the merged state) |
GraphEvent::StateUpdate |
values |
event: values — full state per step |
the Checkpoint.state persisted at that step's boundary, read back from the checkpoint log |
messages |
event: messages — {"node": …, "delta": …} per LLM token |
GraphEvent::Token (requires the node to stream via ChatModel::chat_stream) |
metadata |
first frame: {run_id, thread_id, graph, attempt, metadata} |
synthesized by the server |
error |
{error, message} |
Err(RustyError) from the executor |
end |
{status: success||interrupted||error} (plus interrupt when interrupted) |
the run's ExecutionOutcome |
Fan-out is in-process: each run owns a tokio::sync::broadcast channel fed from the executor's event sink, and every attached SSE client subscribes. No Redis.
Last-Event-ID resume
Every SSE frame carries id: {checkpoint_id}:{step}:{seq}, where seq is a per-run monotonically increasing sequence number (1-based; frames emitted before the first checkpoint use - as the checkpoint component). The two streaming endpoints treat Last-Event-ID differently, on purpose:
POST /threads/{id}/runs/streamstarts a fresh run with a fresh frame sequence, so the header is ignored there — applying a stale seq from a previous run would silently drop the new run's first frames.GET /runs/{id}/streamattaches to an existing run and honors the header: the server replays the run's event-log frames after the last-seen sequence number, then follows live frames until theendframe. This is the reconnect path: start a background run (POST /threads/{id}/runs), attach, and re-attach withLast-Event-IDafter a disconnect — you skip every frame you have already seen.
The event log is a per-run, in-memory ring buffer (ServerConfig::event_log_capacity, default 1000 frames), so replay covers client reconnects within the server's lifetime; durable cross-restart stream resume is roadmap (checkpoints are the stream history, so the data to rebuild it is already on disk). Terminal runs (and their event logs) are retained for attach/polling up to 1024 runs per process; the oldest terminal runs are evicted beyond that, after which attach answers 404.
Proxy guidance. Behind nginx or another reverse proxy, disable response buffering for SSE routes (X-Accel-Buffering: no) and flush per event, or your clients will see nothing until the buffer fills.
Postgres persistence (feature postgres)
The default deployment needs no infrastructure — checkpoints, assistants, crons, and KV items are JSON files under store_path. When you outgrow a single process's file system (several replicas, shared state, operational tooling), build with the postgres feature and point the server at a database:
[]
= { = "0.5", = ["postgres"] }
let config = new
.with_postgres;
// or twelve-factor style: .with_postgres(std::env::var("DATABASE_URL")?)
with_postgres(url) switches both persistence layers in one call:
| Layer | Default | With with_postgres |
|---|---|---|
| Run checkpoints | JsonFileCheckpointer under store_path |
core's PostgresCheckpointer → table rusty_checkpoints |
| Threads | {store_path}/threads/{thread_id}.json |
table server_threads (record as JSONB payload) |
| Assistants | {store_path}/assistants/*.json |
table server_assistants (record as JSONB payload) |
| Crons | {store_path}/crons/*.json |
table server_crons (record as JSONB payload) |
| KV store | {store_path}/store/{ns}/{key}.json |
table server_kv (namespace + "key" primary key, JSONB value, created_at/updated_at) |
| Run journals | {store_path}/journals/{run_id}.json |
table server_journals (run_id primary key, JSONB snapshot, created_at/updated_at) |
All six schemas (rusty_checkpoints plus the five server_* tables) are auto-migrated (CREATE TABLE IF NOT EXISTS …) on connect; connections are established lazily on first use, so router() stays synchronous and the server starts even if the database is briefly unreachable (first-touch failures surface as 500s until Postgres is back). The HTTP surface is identical either way — GET /info reports "checkpointer": "postgres" when enabled — with one deliberate exception: rollback (DELETE /threads/{id}/runs/{run_id}) answers 409 on the Postgres backend rather than silently deleting nothing (the Checkpointer trait has no delete operation, so removal goes through the JSON-file layout directly). Everything else — fork, replay, crons, thread durability across restarts, the KV store, and journal persistence — runs the same code paths against the ServerStore / Checkpointer traits.
The live-Postgres integration tests are gated and skipped by default; run them against a scratch database with:
DATABASE_URL=postgres://user:pass@localhost/rusty_test \
Multi-tenancy: API keys → tenants, with full isolation
Single-key auth (with_api_key) covers the one-organization deployment. For a shared deployment serving several organizations, map API keys to tenants:
let config = new
.with_tenant_key // (tenant, key) pairs
.with_tenant_key
.with_api_key; // optional: legacy key = `default` tenant
Key → tenant model. Every configured key maps to exactly one tenant. A request's X-Api-Key header is resolved to a tenant by the auth middleware (401 for missing/unknown keys) and everything the request touches is scoped to that tenant. With no keys configured the server is in open (dev) mode — no header required, all requests run as the default tenant, and behavior is byte-identical to pre-multi-tenancy versions.
Isolation semantics. Every tenant-scoped resource is namespaced per tenant: threads (including their checkpoint history and run bookkeeping), assistants, crons, and KV namespaces. Concretely, internal ids and namespaces carry a {tenant}/ prefix at the handler layer, so the storage backends separate naturally with no schema changes:
| Resource | JSON-file layout | Postgres |
|---|---|---|
| Threads + checkpoints | records: {store_path}/threads/{tenant}/{id}.json; checkpoints: {store_path}/{tenant}/{thread_id}/… |
server_threads.thread_id resp. rusty_checkpoints.thread_id = "{tenant}/{id}" |
| Assistants | {store_path}/assistants/{tenant}/{id}.json |
server_assistants.assistant_id = "{tenant}/{id}" |
| Crons | {store_path}/crons/{tenant}/{id}.json |
server_crons.cron_id = "{tenant}/{id}" |
| KV store | {store_path}/store/{tenant}/{ns}/{key}.json |
server_kv.namespace = "{tenant}/{ns}" |
The default tenant (open mode and the legacy with_api_key) is unprefixed, so existing deployments keep their flat layout and data — upgrading is a non-event. Tenant ids must match [A-Za-z0-9._-] (1–64 chars).
What this buys you:
- Same external ids can coexist across tenants.
acmeandglobexcan both have a threadt1, an assistantbot, and a KV namespacememorieswithout collisions — each resolves inside its own namespace. - Cross-tenant access is 404, never 403. Fetching another tenant's thread, run, assistant, cron, or KV item returns
not_found, exactly as if the resource didn't exist. A 403 would confirm the resource exists (and that the id is worth attacking); a 404 leaks nothing. - The wire never shows internal prefixes. Responses always carry the external ids the client supplied; the
{tenant}/prefix is an internal storage detail. - The cron scheduler is tenancy-aware. It lists crons across all tenants but fires each one inside its owning tenant's namespace — cron-spawned threads and runs land in the right tenant.
GET /infostays tenant-neutral: it exposes only the service metadata and registered graphs — no tenants, keys, or resource counts.
Configuration
Configuration is code, via ServerConfig (constructed with ServerConfig::new(bind_addr, store_path) plus builder methods, or ServerConfig::default()). If you want twelve-factor env-based config in your binary, read the environment in your own main.rs and build the ServerConfig from it — the crate deliberately does not read process env itself.
| Field / builder | Default | Purpose |
|---|---|---|
bind_addr |
0.0.0.0:8080 |
Listen address (used by serve) |
store_path |
./data/checkpoints |
JsonFileCheckpointer root ({store_path}/{thread_id}/{checkpoint_id}.json) and JSON-file platform persistence |
with_postgres(…) |
None = JSON files |
Postgres URL for both checkpoints and the platform store (feature postgres; see Postgres persistence) |
with_api_key(…) |
None = dev mode, no auth |
Static key required via the X-Api-Key header; maps to the default tenant |
with_tenant_key(…) |
— | Map an API key to a tenant for multi-tenant deployments (see Multi-tenancy) |
with_max_concurrent_runs_per_thread(…) |
1 |
Per-thread enqueue queue depth cap (there is always at most one active run per thread) |
with_event_log_capacity(…) |
1000 |
Per-run SSE replay buffer (frames) |
Built-in limits (constants, not config knobs):
| Limit | Value | Behavior |
|---|---|---|
| Terminal-run retention | 1024 runs | Run status/stream history is in-memory by design; the oldest terminal runs are evicted beyond the cap (active and queued runs are never evicted), after which GET /runs/{id} and GET /runs/{id}/stream answer 404 |
| Blocking-wait ceiling | 3600 s | POST /threads/{id}/runs/wait answers 504 Gateway Timeout when the run hasn't terminated within the ceiling; the run itself keeps executing — poll GET /runs/{id} |
| Cron interval ceiling | 31 536 000 s (1 year) | interval_secs above the ceiling answers 400 (unbounded values would overflow the scheduler's timestamp math) |
Deployment
Build one static binary:
# -> target/release/my-agent (statically linked; the bundled server_demo
# measures ~5 MB as a macOS release build — feature sets shift the size)
Ship it in a scratch image — no interpreter, no pip layer, no system Python:
FROM rust:1-bookworm AS build
WORKDIR /app
COPY . .
RUN cargo build --release
FROM scratch # or gcr.io/distroless/static
COPY --from=build /app/target/release/my-agent /my-agent
ENTRYPOINT ["/my-agent"]
curl quickstart
With the server running locally in dev mode (no API key configured):
# Liveness + what's registered
# {"ok":true}
# {"service":"rusty-server","version":"0.5.0","checkpointer":"json_file",
# "store_path":"./data/checkpoints",
# "graphs":[{"name":"react_agent","channels":["messages"]}]}
# Create a thread bound to a registered graph
# -> 201 {"thread_id": "3f2b9c…", "graph": "react_agent",
# "metadata": null, "created_at": "2026-08-05T…Z"}
# Blocking run
# Streaming run (SSE) — note -N to disable curl buffering
# Resume an interrupted run (human-in-the-loop)
# Thread state + checkpoint history
# Roll back a finished run's checkpoints
# Time travel: fork the thread at an earlier checkpoint, replay the fork
CP_ID=
# -> 201 {"thread_id": "9c1e…", "checkpoints_copied": 1}
# Poll a background run's status (terminal runs carry output/error)
# Flight Recorder: the run's journaled evidence (RunEvents, seq order)
# …or download the run as a portable replay fixture for CI
# …or re-drive it server-side and verify the replayed evidence
# …or diff two runs' journals (e.g. a run vs its fork)
# Attach to a background run's SSE stream; reconnect with Last-Event-ID
# to skip frames you have already seen
# Create an assistant and run by assistant_id
# A cron that fires a run every 60 seconds on a fresh thread
# Cross-thread KV store
With auth configured, add -H "X-Api-Key: $KEY" to every call. For a full walkthrough — project scaffolding, a two-node graph, streaming, and a complete interrupt/resume round trip — see docs/server-quickstart.md.
Roadmap
- Phase A — the server crate (v0.1).
GraphRegistry, the thread/run/SSE endpoint set, per-thread run queue withmultitask_strategy(enqueue/reject) plus explicit rollback viaDELETE /threads/{id}/runs/{run_id}, SSE with mode filters (updates/values/messages) + per-run event log +Last-Event-IDdedup, static API-key middleware,JsonFileCheckpointerwiring fromServerConfig::store_path. Implemented. - Phase C (partial) — platform surface (v0.2).
GET /runs/{run_id}status polling, assistants (named graph + config aliases, JSON-persisted,assistant_idon run-create), crons (interval or 5-field cron schedules, durable records, background tokio scheduler firing runs on fresh threads,on_run_completed: keep|delete), and the cross-thread KV store (/store/{namespace}/{key}, JSON-file-backed). Implemented. - Phase C (continued) — time travel + Postgres persistence (v0.3).
POST /threads/{id}/fork(full- or mid-history forks via core'sCheckpointer::fork_thread), checkpoint replay on all run endpoints ("checkpoint": {"checkpoint_id": …}→RunConfig::with_checkpoint_id), and thepostgresfeature:ServerConfig::with_postgres(url)switches the run checkpointer toPostgresCheckpointerand the assistants/crons/KV surface to auto-migratedserver_assistants/server_crons/server_kvtables behind aServerStoretrait. Implemented. - Phase C (continued) — permissive CORS (v0.3).
router()layerstower_http::cors::CorsLayer::permissive(), so browser clients (the Studio) call the API cross-origin; preflights are answered before auth. Restrict for production — see CORS. Implemented. - Phase C (continued) — multi-tenancy (v0.4). API keys map to tenants (
with_tenant_key(tenant, key); legacywith_api_key= thedefaulttenant); threads + checkpoints, runs, assistants, crons, and KV namespaces are fully isolated via internal{tenant}/id prefixing, with cross-tenant access answering 404 (never 403). Open mode and the default tenant keep the legacy flat storage layout. See Multi-tenancy. Implemented. - Phase C (continued) — hardening (v0.4). Durable thread records (
threads/JSON files /server_threadstable — pre-restart checkpoints stay reachable through the API), the SSE attach endpoint (GET /runs/{id}/streamwithLast-Event-IDreplay), rollback guards (409on the Postgres backend, on busy threads, and on mid-history suffix violations), the croninterval_secsclamp (≤ 1 year) + one-shot tombstones, reserved layout names rejected as client-chosen ids, the 1024-run retention cap, the 3600 s blocking-wait ceiling (504), and400for unknown historybeforecursors. Implemented. - R0.5 — Flight Recorder read surface (v0.5). Every run is journaled by the executor (core R0.5 kernel); the server persists the journal snapshot at every checkpoint boundary and at run completion (
{store_path}/journals/{run_id}.json, or the auto-migratedserver_journalstable on Postgres) and serves it viaGET /runs/{run_id}/events→{run_id, events, complete}in the golden-pinnedRunEventwire shape, with head-hash re-verification on read and 404/tenant-isolation semantics identical toGET /runs/{id}.GET /runs/{run_id}/fixturedownloads the run as a portableReplayFixturefor CI replay. SDK parity:run_events(run_id)(Python) /runEvents(runId)(TypeScript). Implemented — the replay-POST endpoint lands in a later R0.5 wave. - Phase B — gRPC worker protocol (
rusty-proto).RemoteNode: a gRPC client behind the sameNodetrait, delegating node execution to stateless out-of-process workers that long-poll named node-queues. The server keeps checkpoints, super-step scheduling, interrupts, and stream fan-out. Agent nodes are dominated by LLM latency (hundreds of ms to minutes), so a 1–5 ms gRPC hop is <1% overhead — and sinceStateis already a JSON map, the wire boundary is lossless. Crash isolation, polyglot workers (a Python worker can host the LangChain ecosystem while Rust owns orchestration), and independent scaling of tool-heavy nodes follow. - Phase C (remainder). Thread listing/deletion endpoints,
/metrics, and/graphs. (WasmNodeis implemented in corerusty-agent-runtimev0.4 behind thewasmfeature — register a Wasm-backed graph and this crate serves it unchanged.)
Deliberately skipped: A2A/MCP server endpoints and WebSocket "protocol v2" (SSE + HTTP sidecar is sufficient), and feedback_keys (LangSmith-tracing coupling we don't have).
License
Dual-licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE)
- MIT License (LICENSE-MIT)
at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work by you, as defined in the Apache-2.0 license, shall be dual-licensed as above, without any additional terms or conditions.