mlua_swarm/lib.rs
1//! Swarm engine host for mlua — a long-running stateful runtime that
2//! compiles flow.ir Blueprints and dispatches their agent steps to workers.
3//!
4//! # Architecture
5//!
6//! `mlua-swarm` is the host layer of a four-layer stack:
7//!
8//! ```text
9//! flow.ir programs (Lua / JSON Blueprints authored by users or AIs)
10//! │ parsed / compiled
11//! flow-ir-core IR node & expr types + pure evaluation
12//! │ bridged into Lua
13//! mlua-flow-ir Lua <-> IR bridge (mlua-based)
14//! │ hosted by
15//! mlua-swarm (this) engine, workers, operators, middleware, stores
16//! ```
17//!
18//! A [`Blueprint`] declares a `flow` (step / seq / branch / loop / fanout /
19//! try / assign nodes) plus the `agents` it references. The [`Compiler`]
20//! resolves each agent to a [`SpawnerFactory`] (Lua in-process, Rust fn,
21//! subprocess, or WS operator), and the [`Engine`] drives the flow while
22//! recording task state as an [`Event`] stream.
23//!
24//! # Domain vs. Data separation
25//!
26//! Control-flow values (verdicts, counters, small routing fields) travel in
27//! the shared [`Ctx`]; large worker responses are offloaded to an
28//! [`OutputStore`] and referenced from `Ctx` by [`OutputRef`]. This keeps
29//! flow evaluation cheap and bounded regardless of payload size.
30//!
31//! # Middleware
32//!
33//! Worker dispatch passes through a [`SpawnerStack`] assembled from a
34//! [`LayerRegistry`]: a base layer set plus per-blueprint hints
35//! ([`CompilerHints`]) select layers such as audit, long-hold, main-AI
36//! bridging, senior escalation, and operator delegation.
37//!
38//! # Module map
39//!
40//! - [`types`] / [`errors`] — [`Role`](types::Role) × [`Verb`](types::Verb)
41//! capability gate, [`CapToken`](types::CapToken) (HMAC-SHA256), ID
42//! newtypes, and the [`EngineError`](errors::EngineError) surface.
43//! - [`core`] — engine config, task state machine, [`Ctx`], and the
44//! [`Engine`] itself.
45//! - [`blueprint`] — schema shim, compiler, loader (`$agent_md` file-ref
46//! expansion), and versioned stores (in-memory / git2-backed).
47//! - [`worker`] / [`operator`] — spawner adapters, process spawning, output
48//! events, and WS operator sessions.
49//! - [`middleware`] — the layer registry and the individual layers.
50//! - [`lua`] / [`agent_block`] — Lua blueprint bridge, `agents/*.md` loader,
51//! and the agent-block SDK spawner integration.
52//! - [`service`] / [`application`] / [`enhance`] — task-launch orchestration,
53//! application façades, and the self-enhancement (patch / verify / commit)
54//! flow.
55//! - [`store`] — persistence traits and default backends for outputs,
56//! issues, and enhance settings/logs.
57//!
58//! # Worker I/O contract (why IN is a fetch and OUT is a file)
59//!
60//! Every worker step follows one asymmetric I/O shape, and the asymmetry
61//! is deliberate — each side sits where an LLM worker is *reliable*:
62//!
63//! - **IN — one authenticated HTTP fetch.** The worker pulls its prompt
64//! and context with `GET /v1/worker/prompt` (Bearer = capability
65//! token). The server assembles the view fresh per attempt (system
66//! prompt, directive, `AgentContextView`, prior-step pointers), so a
67//! fetch always returns the current attempt's truth — no stale files
68//! to pre-write or clean up, and the payload never has to travel
69//! through the orchestrating operator's own context window (the Spawn
70//! directive relays only a short handle). The fetch doubles as the
71//! trust handshake: the capability token scopes *which* task's IN this
72//! worker may read.
73//! - **OUT — one tool call, never a self-chosen file.** Producing OUT
74//! happens at the *end* of a worker's run — the point where a
75//! long-context LLM is least dependable about paths and formats.
76//! Letting it pick a file name there structurally invites hallucinated
77//! paths and plausible-looking-but-wrong files. So the exit is pinned
78//! to calls that carry no path and no format choice: `POST
79//! /v1/worker/submit` for the final body, `POST
80//! /v1/worker/artifact?name=<name>` per named part.
81//! - **Files are the server's job.** Turning submitted OUT into the IN
82//! files the *next* step reads (plain `Read` on a path — the cheapest,
83//! most reliable worker primitive, with partial reads for free) is
84//! owned by the submit-time projection sink and
85//! [`FileProjectionAdapter`](core::projection::FileProjectionAdapter):
86//! the final body lands as `<ctx-dir>/<step>.md`, each staged part
87//! lands raw as `<ctx-dir>/<name>`. Placement, naming, and format are
88//! adapter policy — deliberately *not* baked into worker defaults, so
89//! workers stay generic and the policy stays swappable
90//! ([`ProjectionPlacement`]).
91//!
92//! See `mse://guides/worker-io-contract` (an `mse mcp` resource) for the
93//! consumer-side view of the same contract.
94
95#![warn(missing_docs)]
96
97pub mod application;
98pub mod blueprint;
99pub mod core;
100pub mod enhance;
101pub mod lua;
102pub mod middleware;
103pub mod operator;
104pub mod service;
105pub mod store;
106pub mod types;
107pub mod worker;
108
109// Symbol re-exports (preserve external API surface).
110pub use application::{
111 Application, BlueprintRef, EnhanceApplication, EnhanceApplicationConfig,
112 EnhanceApplicationError, EnhanceApplicationInput, TaskApplication, TaskApplicationError,
113 TaskApplicationInput, TaskApplicationOutput, TickOutcome, VersionSelector,
114};
115pub use blueprint::compiler::{
116 CompileError, CompiledAgentTable, CompiledBlueprint, Compiler, HostBridge,
117 LuaInProcessSpawnerFactory, LuaScriptSource, OperatorSpawnerFactory,
118 RustFnInProcessSpawnerFactory, SpawnerFactory, SpawnerFactoryKind, SpawnerRegistry,
119 SubprocessProcessSpawnerFactory,
120};
121pub use blueprint::loader::{expand_file_refs, load_blueprint_from_path, LoadError};
122pub use blueprint::{
123 current_schema_version, AgentDef, AgentKind, AgentMeta, Blueprint, BlueprintMetadata,
124 BlueprintOrigin, CompilerHints, CompilerStrategy, EngineDispatcher, SpawnerHints,
125 CURRENT_SCHEMA_VERSION,
126};
127pub use core::config::{EngineCfg, LongHoldConfig};
128pub use core::ctx::{
129 collapse_operator_kind, Ctx, CtxMeta, OperatorInfo, OperatorKind, SeniorBridge, SpawnHook,
130};
131pub use core::engine::Engine;
132pub use core::errors::EngineError;
133pub use core::projection_placement::{
134 ProjectionPlacement, ProjectionPlacementError, RootPreference,
135};
136pub use core::state::{
137 CapTokenConsumeError, CapTokenRecord, DispatchOutcome, Event, EventStream, OperatorSession,
138 ResumeKey, ResumePending, TaskSpec, TaskState, TaskStatus,
139};
140pub use core::step_naming::{StepNameEntry, StepNaming, StepNamingError, StepNamingWarning};
141pub use lua::bridge::{parse_lua_blueprint, parse_lua_blueprint_with_ctx};
142pub use middleware::lua_layer::LuaMiddleware;
143pub use middleware::project_name_alias::{ProjectNameAliasMiddleware, PROJECT_NAME_ALIAS_KEY};
144pub use middleware::resolver::{AgentResolver, FnResolver, ResolverMiddleware};
145pub use middleware::{
146 AuditMiddleware, LayerFactory, LayerRegistry, LongHoldMiddleware, MainAIMiddleware,
147 OperatorDelegateMiddleware, SeniorEscalationMiddleware, SpawnerLayer, SpawnerStack,
148};
149pub use operator::{Operator, OperatorSpawner, WorkerBinding};
150pub use service::{
151 TaskInputSpec, TaskLaunchError, TaskLaunchInput, TaskLaunchOutput, TaskLaunchService,
152};
153pub use store::output::{
154 InMemoryOutputStore, OutputRecord, OutputRef, OutputStore, OutputStoreError,
155};
156pub use types::{
157 default_role_verb_table, CapToken, CapTokenDecodeError, Role, RoleVerbGate, RunId, SessionId,
158 StepId, TaskId, Verb, WorkerId, WorkerPayload,
159};
160pub use worker::adapter::{
161 InProcSpawner, SpawnError, SpawnerAdapter, WorkerError, WorkerFn, WorkerInvocation,
162 WorkerResult,
163};
164pub use worker::agent_block::AgentBlockInProcessSpawnerFactory;
165pub use worker::output::{ContentRef, OutputEvent, OutputSink};
166pub use worker::process_spawner::{ProcessSpawner, StreamMode};
167pub use worker::{MiddlewareWorker, Worker, WorkerJoinHandler};