salvor-engine 0.10.0

The Salvor graph engine: drives a validated graph document through its nodes over the public RunCtx durability substrate
Documentation

The Salvor graph engine: drives a frozen graph document through its nodes, recording the walk into one durable run log.

Where this crate sits, and why it is its own crate

The engine is deliberately not part of salvor-runtime (that would drag the graph document format into the built-in agent loop) and not part of salvor-graph (that crate is a pure, IO-free leaf). It sits above both and composes only their public surfaces: the graph document from salvor-graph, and the durability substrate (RunCtx, drive_loop) from salvor-runtime. It reaches into nothing private. That is a deliberate proof of the runtime's API guardrail: everything the engine needs, an outside crate could also do.

What it drives

[run_graph] opens a run's log with GraphRunStarted, walks the nodes in deterministic topological order (see [walk]), and drives each one:

  • an agent node runs the built-in agent loop (drive_loop) inside the same log, framed by NodeEntered / NodeExited. A node that declares an output_schema runs the structured form of that loop (drive_loop_structured) instead, so the node's output is the validated object the schema describes rather than the reply text, and downstream expressions can read its fields. The resolved agent may declare a schema of its own, in which case the node's declaration wins and the agent's is the fallback (see Agent::output_schema);
  • a tool node records one tool call through the same write-ahead intent/completion machinery the built-in loop uses, honoring the tool's effect class. A tool that asks to park the run parks it: a suspension through Suspended / Resumed, a sleep through SleepStarted / SleepCompleted. Either way the node stays entered with no NodeExited, so a later drive re-enters it and continues from the recorded park. The sleep request rides inside the call's own completion, so the call settles (and releases any idempotency claim) before the timer starts;
  • a gate node parks the run through the exact Suspended / Resumed machinery the built-in loop uses for a tool suspension: entering it records NodeEntered, then suspend records the gate's approval_schema as the suspension schema and the drive returns [GraphOutcome::Parked]. A later drive over the log (carrying the resume input the existing resume machinery appended) passes that input through the gate as its output and continues. A gate needs no event kind of its own. A resume input is ENFORCED against the gate's approval_schema at the accept edge, between the suspend and the await_resume that would record it, so a non-conforming approval is a typed refusal that appends nothing and leaves the run parked; a recorded Resumed is never re-judged on replay (see [approval]);
  • a branch node routes on its input: an expression branch evaluates its cases in author order and the first true case wins; a model-decision branch drives the node's agent and maps the reply to a case name. Either way the chosen case is recorded as BranchTaken, the walk follows the like-named edge, and every node reachable only through a non-taken case is recorded NodeSkipped;
  • a map node fans out over a list. Its over reference resolves against the routed value to a JSON array (a non-array is a typed [EngineError::MapOverNotAList] refused before NodeEntered); the engine records NodeEntered, then MapFannedOut with the resolved item list, then walks the list IN INDEX ORDER, and for each element records MapIterationStarted (carrying the derived child-run id), runs the body's work inline, and records MapIterationJoined. The joined output is the per-element outputs as a list in index order. Iterations run inline and sequentially in the parent's own log: the concurrency cap is accepted (the validator requires it be at least 1) but not honored: a deliberate v0.4 choice that costs only wall-clock and changes no event shape, so the whole fan-out is proven by the same single-log replay machinery already proven for linear and branching graphs. Concurrent child runs are not yet supported. A subgraph body, or a body node that is not an agent or tool, is a typed [EngineError::UnsupportedMapBody] refused before NodeEntered;
  • a fold node runs its body up to max_iterations times, inline and sequentially in the same log. Pass 0's input is the fold's routed value; every later pass's input is the previous pass's output, which IS the accumulated value: there is no separate accumulator state and no merge rule, because the document has no vocabulary for one. A value that is an MCP result envelope (an object with a content ARRAY and a structuredContent key) contributes that PAYLOAD as the accumulated value, not the object around it: an MCP tool answers with a {content, structuredContent} envelope, and the payload is the value the loop is folding, so the next pass's input, the stop_when predicate, a best_by reference, and the join's own output all read the bare payload and a fold expression never reaches through a transport detail. This holds for the value ENTERING the fold as much as for one a pass produced, so a fold fed by a tool node over an edge folds the same shape at pass 0 that it folds at pass 3. Any other value (an agent body's structured object, a native tool's flat struct, an object that merely carries a field called structuredContent as data, a string, a list) is carried verbatim. The unwrap is DERIVATION, not recording: ToolCallCompleted still holds the whole envelope, and the payload is a pure function of it (see [unwrap_pass_output]). Nothing outside a fold is touched: an ordinary edge routes a node's recorded output verbatim, so a branch expression reading structuredContent. still reads what it always read. The engine records NodeEntered, then per pass FoldIterationStarted, the body's work inline, and FoldIterationJoined, and stops when the stop_when predicate holds over the pass just joined or when the bound is reached (there is no third cause: nothing stops a fold for "failing to improve"). Reaching the bound means what on_bound says it means: absent or join joins the passes anyway, and fail is a typed [EngineError::FoldBoundExceeded] returned from exactly where FoldConverged would have been recorded, so the passes and their joins stay in the log and no convergence and no NodeExited land. Otherwise the join rule picks the value the node produces: last takes the final pass, all takes every pass's value as a list in pass order, and best_by is an argmax over ALL passes of the reference's value, ordered by the expression language's own comparison ([salvor_graph::expr::compare]) so the argmax and the predicate beside it can never order values differently, keeping the earliest pass on a tie. A best_by with no comparable candidate in any pass is a typed [EngineError::FoldNoComparableCandidate] refused before FoldConverged. The chosen winner and the stop reason are recorded on FoldConverged, then NodeExited. A subgraph body, or a body node that is not an agent or tool, is a typed [EngineError::UnsupportedFoldBody] refused before NodeEntered;
  • a delay node parks the run on a durable timer, the timer counterpart of the gate: NodeEntered, then sleep_for (a recorded clock reading followed by SleepStarted { wake_at }), then a wait. Before the deadline the drive returns [GraphOutcome::Parked] with ParkReason::Sleeping and no NodeExited, so a drive that arrives early records nothing and a later one re-enters the same node and continues from the recorded sleep. At or past the deadline SleepCompleted lands, NodeExited closes the node, and the walk continues with the node's input passed through UNCHANGED: a delay moves a run in time, never in value, so its output is its input verbatim. It needs no event kind of its own; the sleep pair is the whole vocabulary. The wait is a DURATION in the document and the instant is derived from the recorded reading, which is what keeps the same document runnable more than once (see [salvor_graph::DelayNode]).

A node that is a map's or a fold's body is executed ONLY as that owner's per-item or per-pass worker; it is never walked independently, so its own events (a tool call, an agent loop) are recorded inline between the owner's iteration markers and its node id is never framed with a NodeEntered of its own. That keeps node ids unambiguous in the one log and is why forking INTO a map iteration or a fold pass is refused: neither is a node boundary (see [plan_fork]).

After the last node the engine records the single terminal RunCompleted. It records no terminal for a refusal: refuse-before-record is what keeps the log free of events past one. Whether a refused run is DEAD or merely stuck is the driver's call, and [EngineError::is_permanent] is how the engine tells it apart; [record_permanent_refusal] is the append the drivers make when the answer is dead. There is no ambient clock or randomness in any decision: everything the engine feeds forward (the walk order, each node's input, the branch route, a map's resolved item list and its per-iteration child ids, a fold's stop decision, its winner and its recorded reason, an idempotent tool's idempotency key) is a pure function of the document or of values the RunCtx recorded, so a second drive over the recorded log replays with no live calls and produces a byte-identical log. A map iteration's child-run id is sha256: over the parent run id, the node id, and the index (see [map_child_run_id]): pure recorded data, so replay reconstructs the identical id without storing anything extra. The idempotency key is derived from the call's position in the graph (graph hash, node id, call index) rather than from drawn randomness, which is what lets a FORK of a run re-walk a segment and present the same key its origin recorded. See [fork_safe_idempotency_key] and the salvor-server fork endpoint.

Data flow

Each node's output flows to its successors along the edges, and a node's input is the recorded output of the live inbound edge that reaches it (the graph input for an entry node with no inbound edge). A branch passes its routed value through unchanged to the taken case's edge; the decision only selects the route, never the data. A tool node's input references are still not resolved yet; the upstream output is the downstream input verbatim. When more than one live inbound edge reaches a node, the one whose source id is smallest wins, so the merge is a pure function of the document.

Resolving agents and tools

A node names its agent by hash and its tool by name; the engine turns those into executables through the [AgentResolver] and [ToolResolver] traits the caller supplies. Tests inject maps; the server wires its own registries in separately. Keeping resolution behind a trait is what lets the engine stay ignorant of where agents and tools actually come from.