# Mobs
This is the detailed reference for Meerkat mobs across Rust SDK, CLI, MCP, REST, RPC, Python SDK, and TypeScript SDK.
## Positioning
- Mobs are an optional extension for multi-agent orchestration.
- Base Meerkat workflows remain session/turn centered.
- On CLI, primary mob UX is tool-driven through `run`/`run --resume` with `--tools full` or config `tools.mob_enabled=true`.
- Direct `rkat mob ...` is the helper/artifact operational surface. Lifecycle creation, wiring, and member management use agent `mob_*` tools or RPC `mob/*`.
## Runtime model
Core entities:
- `Mob`: persisted aggregate (definition, members, status, events).
- `Mob member`: spawned runtime participant identified by `agent_identity`.
- `Profile`: role contract (model/tools/skills posture).
- `Wiring`: peer graph edges.
- `Mob event`: append-only lifecycle records.
- `Flow run` (optional): DAG execution record with step and failure ledgers.
Lifecycle:
1. create
2. spawn
3. wire
4. turn and/or flow runs
5. stop/resume/complete
6. destroy
### Member runtime mode (current default)
- Default is `autonomous_host` when `runtime_mode` is omitted.
- `autonomous_host` members are long-lived peers; mob dispatch routes via injector/subscription path.
- `turn_driven` is explicit opt-in; mob dispatch routes via `start_turn`.
Override points:
- profile-level: `[profiles.<name>].runtime_mode`
- spawn-level: `runtime_mode` argument on spawn tool/command
## Definition model
Common sections:
- `[mob]`
- `[profiles.<name>]`
- `[skills.<name>]` (optional)
- `[wiring]`
- `[topology]` (optional)
- `[supervisor]` (optional)
- `[limits]` (optional)
- `[flows.<flow_id>]` (optional)
Important semantics:
- `orchestrator` chooses the orchestration role.
- `external_addressable` gates external turnability.
- `wiring.auto_wire_orchestrator` and `wiring.role_wiring` shape default graph edges.
- topology rules can enforce strict role-level communication policy.
- `runtime_mode` omitted means `autonomous_host` (new default).
## Rust SDK (detailed)
Primary crates:
- `meerkat_mob` for runtime and state.
- `meerkat_mob_mcp` for mob tool dispatcher and in-memory state helper.
### Core Rust types
- `MobDefinition`
- `MobStorage` (SQLite persistent via `SqliteMobStores`, in-memory for tests/WASM)
- `MobBuilder`
- `MobHandle`
- `MobSessionService`
- `MobState`
- `MobRun`, `MobRunStatus`
- `FlowRunConfig`
### `MobBuilder` API
- `MobBuilder::new(definition, storage)`
- `MobBuilder::from_mobpack(definition, packed_skills, storage)` — create from mobpack with inline skills
- `MobBuilder::for_resume(storage)`
- `.with_session_service(Arc<dyn MobSessionService>)`
- `.allow_ephemeral_sessions(bool)`
- `.notify_orchestrator_on_resume(bool)`
- `.with_default_llm_client(client)` — override LLM client (primarily for testing)
- `.register_tool_bundle(name, dispatcher)`
- `.create().await`
- `.resume().await`
### `MobHandle` API
- inspection: `status()`, `definition()`, `mob_id()`, `roster()`, `list_members()`, `list_all_members()`, `get_member()`, `events()`, `mcp_server_states()`
- membership: `spawn_spec(spec)`, `spawn_many(specs)`, `retire(identity)`, `respawn(identity)`, `retire_all()`, `set_spawn_policy()` — all identity-keyed via `AgentIdentity`
- graph: `wire()`, `unwire()`
- turns: `member(id).send(...)`, `internal_turn()`
- lifecycle: `stop()`, `resume()`, `complete()`, `reset()`, `destroy()`, `shutdown()`
- flows: `list_flows()`, `run_flow()`, `run_flow_with_stream()`, `flow_status()`, `cancel_flow()`
- subscriptions: `subscribe_agent_events()`, `subscribe_all_agent_events()`, `subscribe_mob_events()`, `subscribe_mob_events_with_config()`
- tasks: `task_create()`, `task_update()`, `task_list()`, `task_get()`
### Rust example: full lifecycle via `MobBuilder` + `MobHandle`
```rust
use std::sync::Arc;
use meerkat_mob::{
AgentIdentity, FlowId, MobBuilder, MobDefinition, MobSessionService, MobStorage,
SpawnMemberSpec,
};
async fn run_mob(
definition_toml: &str,
session_service: Arc<dyn MobSessionService>,
) -> Result<(), Box<dyn std::error::Error>> {
let definition = MobDefinition::from_toml(definition_toml)?;
let storage = MobStorage::persistent("./mob.db")?; // SQLite/WAL-backed
let handle = MobBuilder::new(definition, storage)
.with_session_service(session_service)
.create()
.await?;
handle
.spawn_spec(SpawnMemberSpec::new("lead", AgentIdentity::from("lead-1")))
.await?;
handle
.spawn_spec(SpawnMemberSpec::new("worker", AgentIdentity::from("worker-1")))
.await?;
handle
.wire(
AgentIdentity::from("lead-1"),
AgentIdentity::from("worker-1"),
)
.await?;
handle
.member(&AgentIdentity::from("lead-1"))
.await?
.send(
"Coordinate a short execution plan.".to_string(),
meerkat_core::types::HandlingMode::Queue,
)
.await?;
let run_id = handle
.run_flow(FlowId::from("release_flow"), serde_json::json!({"severity":"critical"}))
.await?;
let _run = handle.flow_status(run_id).await?;
handle.complete().await?;
Ok(())
}
```
### Rust example: high-level in-memory mob state helper
```rust
use meerkat_mob::{AgentIdentity, MobDefinition, ProfileName};
use meerkat_mob_mcp::MobMcpState;
async fn in_memory() -> Result<(), Box<dyn std::error::Error>> {
let state = MobMcpState::new_in_memory();
let definition = MobDefinition::from_toml(r#"
[mob]
id = "my-mob"
orchestrator = "lead"
[profiles.lead]
model = "claude-opus-4-6"
external_addressable = true
[profiles.lead.tools]
builtins = true
comms = true
mob = true
[profiles.worker]
model = "claude-sonnet-4-6"
[profiles.worker.tools]
builtins = true
comms = true
"#)?;
let mob_id = state.mob_create_definition(definition).await?;
state
.mob_spawn(
&mob_id,
ProfileName::from("lead"),
AgentIdentity::from("lead-1"),
None,
None,
)
.await?;
let _status = state.mob_status(&mob_id).await?;
Ok(())
}
```
## Shared integration model (`meerkat-mob-mcp`)
Outside direct `meerkat_mob` usage, mob capability is provided by composing
`meerkat_mob_mcp::MobMcpDispatcher` into `SessionBuildOptions.external_tools`.
```rust
use std::sync::Arc;
use meerkat_core::service::SessionBuildOptions;
use meerkat_core::AgentToolDispatcher;
use meerkat_mob::MobSessionService;
use meerkat_mob_mcp::{MobMcpDispatcher, MobMcpState};
fn mob_external_tools(
session_service: Arc<dyn MobSessionService>,
) -> Arc<dyn AgentToolDispatcher> {
let state = Arc::new(MobMcpState::new(session_service));
Arc::new(MobMcpDispatcher::new(state))
}
let build = SessionBuildOptions {
external_tools: Some(mob_external_tools(session_service)),
..Default::default()
};
```
## Surface matrix
| CLI `run` / `run --resume` | `mob_*` tools in prompt-driven runs when mob tools are enabled | Primary CLI mob UX |
| CLI `rkat mob ...` | helper/artifact commands | Secondary operational surface |
| CLI `rkat mob pack/deploy/web build` | artifact and browser distribution | Portable deploy + web target |
| RPC | explicit `mob/*` methods | canonical typed substrate for SDKs; `mob/tools` / `mob/call` are escape hatches |
| REST | session HTTP endpoints | compact mob lifecycle via `/mob/tools` + `/mob/call` plus SSE observe |
| MCP | `meerkat_*` session tools | tool-oriented mob access for LLM ergonomics |
| Python SDK | `Mob` class via `create_mob()` | first-class mob lifecycle, member mgmt, flow control, event subscriptions |
| TypeScript SDK | `Mob` class via `createMob()` | first-class mob lifecycle, member mgmt, flow control, event subscriptions |
| Web SDK | `Mob` class via `createMob()` | same WASM-backed mob lifecycle with typed `EventSubscription<T>` |
Runtime-mode behavior is shared across these surfaces because dispatch comes from the same mob runtime:
- autonomous members: event injection/subscription dispatch
- turn-driven members: direct `start_turn` dispatch
### Spawn startup policy
- Mob member spawn uses deferred initial turn semantics.
- Session creation for spawn registers the session without immediately running a model turn.
- Autonomous members then start host loops explicitly from mob actor lifecycle control.
- First model work is triggered by real dispatch (`external_turn`, peer message, or flow step).
- Concurrent spawns provision in parallel; actor finalization stays serialized for deterministic state transitions.
- `spawn_many(Vec<SpawnMemberSpec>)` exposes this as first-class runtime API.
## Multi-surface examples
### CLI tool-driven (primary)
```bash
rkat run --tools full "Create a mob with one lead and three workers, wire lead to all workers, and report status."
rkat run --tools full --resume <session_id> "Retire worker-2 and add worker-4, then summarize."
```
### CLI direct commands (explicit operational)
```bash
rkat mob spawn-helper team-mob "Join as lead-1" --profile lead --agent-identity lead-1
rkat mob fork-helper team-mob lead-1 "Investigate the failing test cluster." --profile worker --json
rkat mob member-status team-mob lead-1 --json
rkat mob force-cancel team-mob worker-1
rkat mob respawn team-mob worker-1 --initial-message "restart"
rkat mob run-flow team-mob --flow triage --stream
```
### CLI artifact + web deployment
```bash
rkat mob pack ./mobs/release-triage -o ./dist/release-triage.mobpack \
--sign ./keys/release.key --signer-id team@example.com # --sign requires --signer-id
rkat mob inspect ./dist/release-triage.mobpack
rkat mob validate ./dist/release-triage.mobpack
rkat mob deploy ./dist/release-triage.mobpack "triage latest regressions" --trust-policy strict
rkat mob web build ./dist/release-triage.mobpack -o ./dist/release-triage-web
```
Web build prerequisites:
```bash
cargo install wasm-pack
export PATH="$HOME/.cargo/bin:$PATH"
```
### WASM browser surface
The web build produces a real meerkat surface — same agent loop, providers, and streaming as CLI/RPC/REST.
**How it works:**
- `meerkat-core` + `meerkat-client` compile to wasm32 via `tokio_with_wasm` (drop-in tokio replacement)
- `reqwest` uses browser `fetch` on wasm32 — no custom JS bridge
- `web-time` replaces `std::time` types (SystemTime, Instant) for browser compatibility
- Anthropic CORS header added automatically on wasm32 targets
**Available in browser:** agent loop, all LLM providers, sessions, JSON schema validation, budget enforcement, events, skills types, MCP config types, tool/compactor/memory traits.
**Not available in browser:** filesystem config loading (programmatic config instead), stdio MCP servers (no processes), MCP protocol client (rmcp depends on tokio/mio — types work but connections blocked), shell tool, file-based persistence.
**WASM API (28 exports):**
See `SKILL.md` WASM section for the full export list. Key mob-related exports:
```
mob_create(definition_json) → mob_id string [async]
mob_spawn(mob_id, specs_json) → result JSON [async]
mob_wire / mob_unwire / mob_retire / mob_respawn [async]
mob_list_members / mob_send_message / mob_events(mob_id, after_cursor: u32, limit: u32) / mob_status / mob_list
mob_lifecycle(mob_id, action) [async]
mob_run_flow → run_id string [async] / mob_flow_status / mob_cancel_flow [async]
wire_cross_mob(mob_id, a, b) [async]
mob_member_subscribe [async] / mob_subscribe_events [async] / poll_subscription / close_subscription
```
### RPC
```json
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}
{"jsonrpc":"2.0","id":2,"method":"session/create","params":{"prompt":"Use mob_* tools to create a lead/worker mob and return status."}}
```
### REST
```bash
curl -X POST http://127.0.0.1:8080/sessions \
-H "Content-Type: application/json" \
-d '{"prompt":"Use mob_* tools to create a lead/worker mob and return status."}'
```
### MCP
```json
{
"name": "meerkat_run",
"arguments": {
"prompt": "Use mob_* tools to create a lead/worker mob and return status."
}
}
```
### Python SDK
```python
from meerkat import MeerkatClient
client = MeerkatClient()
await client.connect(realm_id="team-alpha")
result = await client.create_session("Design a mob topology for release triage.")
print(result.text)
await client.close()
```
### TypeScript SDK
```typescript
import { MeerkatClient } from "@rkat/sdk";
const client = new MeerkatClient();
await client.connect({ realmId: "team-alpha" });
const result = await client.createSession({
prompt: "Use mob_* tools to create a lead/worker mob and return status.",
});
console.log(result.text);
await client.close();
```
## Flows (subfeature)
Flows add DAG orchestration to mobs.
### v1 flows (flat step DAG)
Flow essentials:
- `depends_on` + `depends_on_mode` (`all`/`any`)
- `dispatch_mode` (`one_to_one`/`fan_out`/`fan_in`)
- `collection_policy` (`any`/`all`/`quorum`)
- optional `condition` and `branch`
- persisted `step_ledger` and `failure_ledger`
### v2 flows (frame-based execution with loops)
v2 flows carry `FlowSpec.root: FrameSpec` as the execution root. When `root` is present, the `FlowFrameEngine` drives execution instead of flat topological-sort dispatch.
Key types:
- `FrameSpec` — a set of `FlowNodeSpec` nodes forming a dependency graph within a frame
- `FlowNodeSpec` — either `Step(FrameStepSpec)` or `RepeatUntil(RepeatUntilSpec)`
- `RepeatUntilSpec` — loop with `loop_id`, `depends_on`, `body: FrameSpec`, `until: ConditionExpr`, `max_iterations: u32`
Execution model:
- `MobMachine` owns per-frame state (node readiness, completion tracking)
- `MobMachine` owns loop body/evaluate lifecycle
- `MobMachine` owns scheduler grants (`GrantNodeSlot`, `GrantBodyFrameStart`), frame-step projection, and terminalization
- `flow_run`, `flow_frame`, and `loop_iteration` are MobMachine-owned fail-closed projection reducers used to materialize `MobRun` snapshots. They are not standalone machines.
- Frame-step outcomes route back through MobMachine-owned transitions; direct mutation from executor code is prohibited
- Recovery handles ready-frame / pending-body-frame drift and returns typed incompatibility for pre-v2 active runs
Definition example:
```toml
[flows.release_flow]
description = "Iterative release check"
[flows.release_flow.root]
nodes.check_quality = { type = "repeat_until", loop_id = "quality_loop", body = { nodes = { run_tests = { type = "step", role = "tester", message = "Run tests" } } }, until = "all_pass", max_iterations = 5, depends_on = [] }
nodes.ship = { type = "step", role = "lead", message = "Ship it", depends_on = ["check_quality"] }
```
### Operational flow controls
- list flows
- run flow
- check flow status
- cancel flow
### Agent-facing delegation tools
Agents can orchestrate mobs programmatically via tools exposed by `AgentMobToolSurface` (`meerkat-mob-mcp/src/agent_tools.rs`):
| `delegate` | Quick helper spawn — creates implicit mob on first use, spawns member, auto-wires comms |
| `mob_create` | Create a mob from a definition |
| `mob_destroy` | Destroy a mob and archive all members |
| `mob_spawn_member` | Spawn a member into any mob |
| `mob_retire_member` | Archive a member and its session |
| `mob_check_member` | Check a member's execution status and output |
| `mob_list_members` | List members of a mob |
| `mob_list` | List all mobs |
| `mob_wire` | Wire a member to a local or external peer (creates comms trust) |
| `mob_unwire` | Remove a wiring relationship between a member and a peer |
When a realm profile store is configured, six additional profile-management tools are surfaced — they treat profiles as reusable, versioned member templates:
| `mob_profile_create` | Register a named profile in the realm |
| `mob_profile_get` | Read a profile (with revision) |
| `mob_profile_list` | List profiles in the realm |
| `mob_profile_update` | Update a profile with `expected_revision` for CAS |
| `mob_profile_delete` | Delete a profile with `expected_revision` for CAS |
| `mob_profile_list_sources` | List the provenance sources contributing profiles |
These tools are composed into the agent's tool dispatcher via `MobToolsFactory` late-binding. Operator authority is injected at runtime; ambient mob enablement alone does not surface operator tools on resume.
## Practical guidance
Use mobs when you need:
- long-lived role-based multi-agent systems,
- explicit peer graph control,
- durable operational history.
Use plain sessions when:
- single-agent execution is sufficient,
- no shared graph/lifecycle state is required.