mlua_swarm/lib.rs
1//! Swarm engine host for mlua — a long-running stateful runtime that
2//! compiles flow.ir Blueprints and dispatches their agent steps to workers.
3//!
4//! # Architecture
5//!
6//! `mlua-swarm` is the host layer of a four-layer stack:
7//!
8//! ```text
9//! flow.ir programs (Lua / JSON Blueprints authored by users or AIs)
10//! │ parsed / compiled
11//! flow-ir-core IR node & expr types + pure evaluation
12//! │ bridged into Lua
13//! mlua-flow-ir Lua <-> IR bridge (mlua-based)
14//! │ hosted by
15//! mlua-swarm (this) engine, workers, operators, middleware, stores
16//! ```
17//!
18//! A [`Blueprint`] declares a `flow` (step / seq / branch / loop / fanout /
19//! try / assign nodes) plus the `agents` it references. The [`Compiler`]
20//! resolves each agent to a [`SpawnerFactory`] (Lua in-process, Rust fn,
21//! subprocess, or WS operator), and the [`Engine`] drives the flow while
22//! recording task state as an [`Event`] stream.
23//!
24//! # Domain vs. Data separation
25//!
26//! Control-flow values (verdicts, counters, small routing fields) travel in
27//! the shared [`Ctx`]; large worker responses are offloaded to an
28//! [`OutputStore`] and referenced from `Ctx` by [`OutputRef`]. This keeps
29//! flow evaluation cheap and bounded regardless of payload size.
30//!
31//! # Middleware
32//!
33//! Worker dispatch passes through a [`SpawnerStack`] assembled from a
34//! [`LayerRegistry`]: a base layer set plus per-blueprint hints
35//! ([`CompilerHints`]) select layers such as audit, long-hold, main-AI
36//! bridging, and senior escalation. Dispatching a step to an
37//! [`Operator`](operator::Operator) is not one of them: that is the
38//! AgentSpec axis, where the agent's declared seat resolves its current
39//! holder per dispatch (see [`OperatorSpawnerFactory`]).
40//!
41//! # Module map
42//!
43//! - [`types`] / [`errors`] — [`Role`](types::Role) × [`Verb`](types::Verb)
44//! capability gate, [`CapToken`](types::CapToken) (HMAC-SHA256), ID
45//! newtypes, and the [`EngineError`](errors::EngineError) surface.
46//! - [`core`] — engine config, task state machine, [`Ctx`], and the
47//! [`Engine`] itself.
48//! - [`blueprint`] — schema shim, compiler, loader (`$agent_md` file-ref
49//! expansion), and versioned stores (in-memory / git2-backed).
50//! - [`worker`] / [`operator`] — spawner adapters, process spawning, output
51//! events, and WS operator sessions.
52//! - [`middleware`] — the layer registry and the individual layers.
53//! - [`lua`] / [`agent_block`] — Lua blueprint bridge, `agents/*.md` loader,
54//! and the agent-block SDK spawner integration.
55//! - [`service`] / [`application`] / [`enhance`] — task-launch orchestration,
56//! application façades, and the self-enhancement (patch / verify / commit)
57//! flow.
58//! - [`store`] — persistence traits and default backends for outputs,
59//! issues, and enhance settings/logs.
60//!
61//! # Worker I/O contract (why IN is a fetch and OUT is a file)
62//!
63//! Every worker step follows one asymmetric I/O shape, and the asymmetry
64//! is deliberate — each side sits where an LLM worker is *reliable*:
65//!
66//! - **IN — one authenticated HTTP fetch.** The worker pulls its prompt
67//! and context with `GET /v1/worker/prompt` (Bearer = capability
68//! token). The server assembles the view fresh per attempt (system
69//! prompt, directive, `AgentContextView`, prior-step pointers), so a
70//! fetch always returns the current attempt's truth — no stale files
71//! to pre-write or clean up, and the payload never has to travel
72//! through the orchestrating operator's own context window (the Spawn
73//! directive relays only a short handle). The fetch doubles as the
74//! trust handshake: the capability token scopes *which* task's IN this
75//! worker may read.
76//! - **OUT — one tool call, never a self-chosen file.** Producing OUT
77//! happens at the *end* of a worker's run — the point where a
78//! long-context LLM is least dependable about paths and formats.
79//! Letting it pick a file name there structurally invites hallucinated
80//! paths and plausible-looking-but-wrong files. So the exit is pinned
81//! to calls that carry no path and no format choice: `POST
82//! /v1/worker/submit` for the final body, `POST
83//! /v1/worker/artifact?name=<name>` per named part.
84//! - **Files are the server's job.** Turning submitted OUT into the IN
85//! files the *next* step reads (plain `Read` on a path — the cheapest,
86//! most reliable worker primitive, with partial reads for free) is
87//! owned by the submit-time projection sink and
88//! [`FileProjectionAdapter`](core::projection::FileProjectionAdapter):
89//! the final body lands as `<ctx-dir>/<step>.md`, each staged part
90//! lands raw as `<ctx-dir>/<name>`. Placement, naming, and format are
91//! adapter policy — deliberately *not* baked into worker defaults, so
92//! workers stay generic and the policy stays swappable
93//! ([`ProjectionPlacement`]).
94//!
95//! See `mse://guides/worker-io-contract` (an `mse mcp` resource) for the
96//! consumer-side view of the same contract.
97
98#![warn(missing_docs)]
99
100pub mod application;
101pub mod binding;
102pub mod blueprint;
103pub mod core;
104pub mod enhance;
105pub mod lua;
106pub mod middleware;
107pub mod operator;
108pub mod service;
109pub mod store;
110pub mod types;
111pub mod worker;
112
113// Symbol re-exports (preserve external API surface).
114pub use application::{
115 Application, BlueprintRef, EnhanceApplication, EnhanceApplicationConfig,
116 EnhanceApplicationError, EnhanceApplicationInput, TaskApplication, TaskApplicationError,
117 TaskApplicationInput, TaskApplicationOutput, TickOutcome, VersionSelector,
118};
119pub use binding::{
120 attest_bound_agents, binding_request_for_snapshot, binding_requests,
121 validate_bound_agent_snapshot, validate_bound_agent_snapshots, AgentBindingProvider,
122 BindingProviderError, LegacyWorkerBindingPolicy, ManifestBindingProvider, UnboundAgent,
123};
124pub use blueprint::compiler::{
125 agents_with_all_verdict_values_unread, removed_spawner_hint_reason, unhandled_verdict_values,
126 AgentContractUnread, CompileError, CompiledAgentTable, CompiledBlueprint, Compiler, HostBridge,
127 LuaInProcessSpawnerFactory, LuaScriptSource, OperatorSpawnerFactory,
128 RustFnInProcessSpawnerFactory, SpawnerFactory, SpawnerFactoryKind, SpawnerRegistry,
129 SubprocessProcessSpawnerFactory, UnhandledVerdictValue, WORKER_BINDING_REQUIRED_MSG_PREFIX,
130};
131pub use blueprint::loader::{expand_file_refs, load_blueprint_from_path, LoadError};
132pub use blueprint::{
133 current_schema_version, AgentDef, AgentKind, AgentMeta, AgentProviderCapability,
134 AgentProviderManifest, BindOutcome, BindReceipt, BindRequest, BindingAttestation,
135 BindingBackend, BindingDigest, Blueprint, BlueprintMetadata, BlueprintOrigin, CompilerHints,
136 CompilerStrategy, EngineDispatcher, SpawnerHints, CURRENT_SCHEMA_VERSION,
137};
138pub use core::config::{EngineCfg, LongHoldConfig};
139pub use core::ctx::{
140 collapse_operator_kind, Ctx, CtxMeta, OperatorInfo, OperatorKind, SeniorBridge, SpawnHook,
141};
142pub use core::engine::Engine;
143pub use core::errors::EngineError;
144pub use core::projection_placement::{
145 ProjectionPlacement, ProjectionPlacementError, RootPreference,
146};
147pub use core::state::{
148 CapTokenConsumeError, CapTokenRecord, DispatchOutcome, Event, EventStream, LaunchEnvelope,
149 ResumeKey, ResumePending, TaskSpec, TaskState, TaskStatus,
150};
151pub use core::step_naming::{StepNameEntry, StepNaming, StepNamingError, StepNamingWarning};
152pub use lua::bridge::{parse_lua_blueprint, parse_lua_blueprint_with_ctx};
153pub use middleware::lua_layer::LuaMiddleware;
154pub use middleware::project_name_alias::{ProjectNameAliasMiddleware, PROJECT_NAME_ALIAS_KEY};
155pub use middleware::resolver::{AgentResolver, FnResolver, ResolverMiddleware};
156pub use middleware::{
157 AuditMiddleware, LayerFactory, LayerRegistry, LongHoldMiddleware, MainAIMiddleware,
158 SeniorEscalationMiddleware, SpawnerLayer, SpawnerStack,
159};
160/// GH #79: the unified diagnostic vocabulary crate, re-exported so
161/// downstream users of `mlua-swarm` reach the `Diagnostic` /
162/// `LintDecl` types (`impl From<&CompileError> for diag::Diagnostic`
163/// lives in [`blueprint::compiler`]) without a separate dependency
164/// declaration.
165pub use mlua_swarm_diag as diag;
166pub use operator::{Operator, OperatorSlotResolver, OperatorSpawner, WorkerBinding};
167pub use service::{
168 TaskInputSpec, TaskLaunchError, TaskLaunchInput, TaskLaunchOutput, TaskLaunchService,
169};
170pub use store::output::{
171 InMemoryOutputStore, OutputRecord, OutputRef, OutputStore, OutputStoreError,
172};
173pub use types::{
174 default_role_verb_table, CapToken, CapTokenDecodeError, EmptyOperatorRef, OperatorRef, Role,
175 RoleVerbGate, RunId, SessionId, StepId, TaskId, Verb, WorkerId, WorkerPayload,
176};
177pub use worker::adapter::{
178 InProcSpawner, SpawnError, SpawnerAdapter, WorkerError, WorkerFn, WorkerInvocation,
179 WorkerResult,
180};
181pub use worker::agent_block::AgentBlockInProcessSpawnerFactory;
182pub use worker::output::{ContentRef, OutputEvent, OutputSink};
183pub use worker::process_spawner::{ProcessSpawner, StreamMode};
184pub use worker::{MiddlewareWorker, Worker, WorkerJoinHandler};