mlua_swarm/lib.rs
1//! Swarm engine host for mlua — a long-running stateful runtime that
2//! compiles flow.ir Blueprints and dispatches their agent steps to workers.
3//!
4//! # Architecture
5//!
6//! `mlua-swarm` is the host layer of a four-layer stack:
7//!
8//! ```text
9//! flow.ir programs (Lua / JSON Blueprints authored by users or AIs)
10//! │ parsed / compiled
11//! flow-ir-core IR node & expr types + pure evaluation
12//! │ bridged into Lua
13//! mlua-flow-ir Lua <-> IR bridge (mlua-based)
14//! │ hosted by
15//! mlua-swarm (this) engine, workers, operators, middleware, stores
16//! ```
17//!
18//! A [`Blueprint`] declares a `flow` (step / seq / branch / loop / fanout /
19//! try / assign nodes) plus the `agents` it references. The [`Compiler`]
20//! resolves each agent to a [`SpawnerFactory`] (Lua in-process, Rust fn,
21//! subprocess, or WS operator), and the [`Engine`] drives the flow while
22//! recording task state as an [`Event`] stream.
23//!
24//! # Domain vs. Data separation
25//!
26//! Control-flow values (verdicts, counters, small routing fields) travel in
27//! the shared [`Ctx`]; large worker responses are offloaded to an
28//! [`OutputStore`] and referenced from `Ctx` by [`OutputRef`]. This keeps
29//! flow evaluation cheap and bounded regardless of payload size.
30//!
31//! # Middleware
32//!
33//! Worker dispatch passes through a [`SpawnerStack`] assembled from a
34//! [`LayerRegistry`]: a base layer set plus per-blueprint hints
35//! ([`CompilerHints`]) select layers such as audit, long-hold, main-AI
36//! bridging, senior escalation, and operator delegation.
37//!
38//! # Module map
39//!
40//! - [`types`] / [`errors`] — [`Role`](types::Role) × [`Verb`](types::Verb)
41//! capability gate, [`CapToken`](types::CapToken) (HMAC-SHA256), ID
42//! newtypes, and the [`EngineError`](errors::EngineError) surface.
43//! - [`core`] — engine config, task state machine, [`Ctx`], and the
44//! [`Engine`] itself.
45//! - [`blueprint`] — schema shim, compiler, loader (`$agent_md` file-ref
46//! expansion), and versioned stores (in-memory / git2-backed).
47//! - [`worker`] / [`operator`] — spawner adapters, process spawning, output
48//! events, and WS operator sessions.
49//! - [`middleware`] — the layer registry and the individual layers.
50//! - [`lua`] / [`agent_block`] — Lua blueprint bridge, `agents/*.md` loader,
51//! and the agent-block SDK spawner integration.
52//! - [`service`] / [`application`] / [`enhance`] — task-launch orchestration,
53//! application façades, and the self-enhancement (patch / verify / commit)
54//! flow.
55//! - [`store`] — persistence traits and default backends for outputs,
56//! issues, and enhance settings/logs.
57//!
58//! # Worker I/O contract (why IN is a fetch and OUT is a file)
59//!
60//! Every worker step follows one asymmetric I/O shape, and the asymmetry
61//! is deliberate — each side sits where an LLM worker is *reliable*:
62//!
63//! - **IN — one authenticated HTTP fetch.** The worker pulls its prompt
64//! and context with `GET /v1/worker/prompt` (Bearer = capability
65//! token). The server assembles the view fresh per attempt (system
66//! prompt, directive, `AgentContextView`, prior-step pointers), so a
67//! fetch always returns the current attempt's truth — no stale files
68//! to pre-write or clean up, and the payload never has to travel
69//! through the orchestrating operator's own context window (the Spawn
70//! directive relays only a short handle). The fetch doubles as the
71//! trust handshake: the capability token scopes *which* task's IN this
72//! worker may read.
73//! - **OUT — one tool call, never a self-chosen file.** Producing OUT
74//! happens at the *end* of a worker's run — the point where a
75//! long-context LLM is least dependable about paths and formats.
76//! Letting it pick a file name there structurally invites hallucinated
77//! paths and plausible-looking-but-wrong files. So the exit is pinned
78//! to calls that carry no path and no format choice: `POST
79//! /v1/worker/submit` for the final body, `POST
80//! /v1/worker/artifact?name=<name>` per named part.
81//! - **Files are the server's job.** Turning submitted OUT into the IN
82//! files the *next* step reads (plain `Read` on a path — the cheapest,
83//! most reliable worker primitive, with partial reads for free) is
84//! owned by the submit-time projection sink and
85//! [`FileProjectionAdapter`](core::projection::FileProjectionAdapter):
86//! the final body lands as `<ctx-dir>/<step>.md`, each staged part
87//! lands raw as `<ctx-dir>/<name>`. Placement, naming, and format are
88//! adapter policy — deliberately *not* baked into worker defaults, so
89//! workers stay generic and the policy stays swappable
90//! ([`ProjectionPlacement`]).
91//!
92//! See `mse://guides/worker-io-contract` (an `mse mcp` resource) for the
93//! consumer-side view of the same contract.
94
95#![warn(missing_docs)]
96
97pub mod application;
98pub mod binding;
99pub mod blueprint;
100pub mod core;
101pub mod enhance;
102pub mod lua;
103pub mod middleware;
104pub mod operator;
105pub mod service;
106pub mod store;
107pub mod types;
108pub mod worker;
109
110// Symbol re-exports (preserve external API surface).
111pub use application::{
112 Application, BlueprintRef, EnhanceApplication, EnhanceApplicationConfig,
113 EnhanceApplicationError, EnhanceApplicationInput, TaskApplication, TaskApplicationError,
114 TaskApplicationInput, TaskApplicationOutput, TickOutcome, VersionSelector,
115};
116pub use binding::{
117 attest_bound_agents, binding_request_for_snapshot, binding_requests,
118 validate_bound_agent_snapshot, validate_bound_agent_snapshots, AgentBindingProvider,
119 BindingProviderError, LegacyWorkerBindingPolicy, ManifestBindingProvider, UnboundAgent,
120};
121pub use blueprint::compiler::{
122 agents_with_all_verdict_values_unread, unhandled_verdict_values, AgentContractUnread,
123 CompileError, CompiledAgentTable, CompiledBlueprint, Compiler, HostBridge,
124 LuaInProcessSpawnerFactory, LuaScriptSource, OperatorSpawnerFactory,
125 RustFnInProcessSpawnerFactory, SpawnerFactory, SpawnerFactoryKind, SpawnerRegistry,
126 SubprocessProcessSpawnerFactory, UnhandledVerdictValue, WORKER_BINDING_REQUIRED_MSG_PREFIX,
127};
128pub use blueprint::loader::{expand_file_refs, load_blueprint_from_path, LoadError};
129pub use blueprint::{
130 current_schema_version, AgentDef, AgentKind, AgentMeta, AgentProviderCapability,
131 AgentProviderManifest, BindOutcome, BindReceipt, BindRequest, BindingAttestation,
132 BindingBackend, BindingDigest, Blueprint, BlueprintMetadata, BlueprintOrigin, CompilerHints,
133 CompilerStrategy, EngineDispatcher, SpawnerHints, CURRENT_SCHEMA_VERSION,
134};
135pub use core::config::{EngineCfg, LongHoldConfig};
136pub use core::ctx::{
137 collapse_operator_kind, Ctx, CtxMeta, OperatorInfo, OperatorKind, SeniorBridge, SpawnHook,
138};
139pub use core::engine::Engine;
140pub use core::errors::EngineError;
141pub use core::projection_placement::{
142 ProjectionPlacement, ProjectionPlacementError, RootPreference,
143};
144pub use core::state::{
145 CapTokenConsumeError, CapTokenRecord, DispatchOutcome, Event, EventStream, OperatorSession,
146 ResumeKey, ResumePending, TaskSpec, TaskState, TaskStatus,
147};
148pub use core::step_naming::{StepNameEntry, StepNaming, StepNamingError, StepNamingWarning};
149pub use lua::bridge::{parse_lua_blueprint, parse_lua_blueprint_with_ctx};
150pub use middleware::lua_layer::LuaMiddleware;
151pub use middleware::project_name_alias::{ProjectNameAliasMiddleware, PROJECT_NAME_ALIAS_KEY};
152pub use middleware::resolver::{AgentResolver, FnResolver, ResolverMiddleware};
153pub use middleware::{
154 AuditMiddleware, LayerFactory, LayerRegistry, LongHoldMiddleware, MainAIMiddleware,
155 OperatorDelegateMiddleware, SeniorEscalationMiddleware, SpawnerLayer, SpawnerStack,
156};
157/// GH #79: the unified diagnostic vocabulary crate, re-exported so
158/// downstream users of `mlua-swarm` reach the `Diagnostic` /
159/// `LintDecl` types (`impl From<&CompileError> for diag::Diagnostic`
160/// lives in [`blueprint::compiler`]) without a separate dependency
161/// declaration.
162pub use mlua_swarm_diag as diag;
163pub use operator::{Operator, OperatorSpawner, WorkerBinding};
164pub use service::{
165 TaskInputSpec, TaskLaunchError, TaskLaunchInput, TaskLaunchOutput, TaskLaunchService,
166};
167pub use store::output::{
168 InMemoryOutputStore, OutputRecord, OutputRef, OutputStore, OutputStoreError,
169};
170pub use types::{
171 default_role_verb_table, CapToken, CapTokenDecodeError, Role, RoleVerbGate, RunId, SessionId,
172 StepId, TaskId, Verb, WorkerId, WorkerPayload,
173};
174pub use worker::adapter::{
175 InProcSpawner, SpawnError, SpawnerAdapter, WorkerError, WorkerFn, WorkerInvocation,
176 WorkerResult,
177};
178pub use worker::agent_block::AgentBlockInProcessSpawnerFactory;
179pub use worker::output::{ContentRef, OutputEvent, OutputSink};
180pub use worker::process_spawner::{ProcessSpawner, StreamMode};
181pub use worker::{MiddlewareWorker, Worker, WorkerJoinHandler};