# mentra
Mentra is an agent runtime for building tool-using LLM applications.
MSRV: Rust 1.88.
## Current Features
- streaming model response handling
- provider-neutral token usage reporting across OpenAI, OpenRouter, Anthropic, Gemini, Ollama, and LM Studio
- optional tool authorization with structured previews and fail-closed execution blocking
- recoverable malformed tool-call input handling that feeds retry guidance back to the model
- custom tool execution through `ToolDefinition + ToolExecutor`, with `ToolSpec::builder(...)` as the convenience metadata API
- builtin `shell`, `background_run`, `check_background`, and `files` tools
- builtin `task` subagents with isolated child context and parent-side tracking
- persistent agent teams with `team_spawn`, `team_send`, `broadcast`, `team_read_inbox`, and generic request-response protocols via `team_request`, `team_respond`, and `team_list_requests`
- context management with optional request-only tool-result elision (disabled
by default), auto-summary compaction, and a builtin `compact` tool
- Model Context Protocol servers over stdio and the legacy HTTP+SSE transport, with their tools bridged into the runtime
- agent events and snapshots for CLI or UI watchers
- Anthropic provider support
- Gemini Developer API provider support
- OpenAI provider support via the Responses API
- OpenRouter provider support via the Responses API
- Ollama provider support via the OpenAI-compatible Responses API
- LM Studio provider support via the OpenAI-compatible Responses API
- image inputs for OpenAI and Anthropic, plus inline image bytes for Gemini
## Quickstart Example
Clone the repository and run the workspace quickstart example:
```bash
cargo run -p mentra-examples --example quickstart -- "Summarize the benefits of tool-using agents."
```
The quickstart example accepts a prompt from CLI args or stdin. Set `MENTRA_MODEL` to force a specific OpenAI model; otherwise it resolves the newest available OpenAI model automatically.
## Building A Runtime
Use `Runtime::builder()` when you want Mentra's builtin runtime tools, or `Runtime::empty_builder()` when you want to opt into every tool explicitly.
```rust,no_run
use mentra::{BuiltinProvider, Runtime};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
.with_optional_provider(
BuiltinProvider::OpenRouter,
std::env::var("OPENROUTER_API_KEY").ok(),
)
.with_optional_provider(
BuiltinProvider::Gemini,
std::env::var("GEMINI_API_KEY").ok(),
)
.with_ollama()
.with_lmstudio()
.build()?;
let _ = runtime;
Ok(())
}
```
`with_ollama()` targets `http://127.0.0.1:11434/` and `with_lmstudio()` targets
`http://127.0.0.1:1234/`, using each server's OpenAI-compatible API surface.
## Rebuilding With Fresh Provider Session State
An embedding host that rebuilds a private runtime can reuse the selected
provider configuration without carrying response-chain, turn-affinity,
WebSocket, or in-flight state into the replacement:
```rust,no_run
use mentra::Runtime;
# fn rebuild(runtime: Runtime) -> Result<Runtime, Box<dyn std::error::Error>> {
let provider = runtime.fresh_provider_session_scope(None)?;
drop(runtime);
let replacement = Runtime::builder()
.with_provider_instance(provider)
.build()?;
# Ok(replacement)
# }
```
`None` selects the default provider; pass a `ProviderId` to select another.
Ordinary clones of the returned `ProviderSessionScope` share one newly minted
scope, which lets a runtime and a retained handle observe the same connection
state. Calling `Provider::fresh_session_scope` again creates another independent
scope. The host remains responsible for quiescing and dropping work attached to
the old runtime before reusing its resources.
Scope minting is deliberately cold and local. It preserves credentials,
configuration, HTTP connection pools, and endpoint knowledge, but it performs no
network I/O and does not prewarm a WebSocket. A host that requires prewarming
must keep its explicit concrete-provider factory/warm path; the high-level scope
does not expose provider-specific warm operations or silently replace that
lifecycle.
## Custom Compatible Providers
If you need a non-default OpenAI-compatible or Anthropic-compatible endpoint,
register a provider-core instance with a customized `ProviderDefinition`.
Using a distinct provider ID lets you keep the builtin provider alongside your
custom endpoint.
```rust,no_run
use mentra::{ModelSelector, ProviderId, Runtime};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let mut definition = mentra::provider_core::responses::openai_definition();
definition.descriptor.id = ProviderId::new("custom-openai-compatible");
definition.descriptor.display_name = Some("Custom OpenAI-Compatible".to_string());
definition.base_url = Some("https://llm.example.com/".to_string());
let runtime = Runtime::builder()
.with_registered_provider(mentra::provider_core::responses::ResponsesProvider::new(
definition,
mentra::provider_core::StaticCredentialSource::new(std::env::var("CUSTOM_API_KEY")?),
))
.build()?;
let model = runtime
.resolve_model(
ProviderId::new("custom-openai-compatible"),
ModelSelector::NewestAvailable,
)
.await?;
# let _ = model;
# Ok(())
# }
```
Anthropic-compatible endpoints follow the same pattern:
```rust,no_run
use mentra::{ProviderId, Runtime};
# fn demo() -> Result<(), Box<dyn std::error::Error>> {
let mut definition = mentra::provider_core::anthropic::definition();
definition.descriptor.id = ProviderId::new("custom-anthropic-compatible");
definition.descriptor.display_name = Some("Custom Anthropic-Compatible".to_string());
definition.base_url = Some("https://claude.example.com/".to_string());
let runtime = Runtime::builder()
.with_registered_provider(
mentra::provider_core::anthropic::AnthropicProvider::with_definition_and_credential_source(
definition,
mentra::provider_core::StaticCredentialSource::new(std::env::var("CUSTOM_API_KEY")?),
),
)
.build()?;
# let _ = runtime;
# Ok(())
# }
```
If your compatible endpoint needs different auth or extra headers, mutate the
definition's `auth_scheme`, `headers`, `query_params`, or `retry` fields before
registering it.
## Architecture
Mentra is organized around four runtime subsystems:
- execution: model providers, runtime policy, hooks, turn execution, and shell/background command routing
- persistence: agent records, run state, task snapshots, leases, team state, background notifications, and memory
- tooling: builtin and custom tools, optional skills, and typed app context
- collaboration: persistent teammates, team inbox/request flows, and background task wakeups
Persistent teammates are hosted as async actors on a shared Tokio runtime. Live actors are wake-driven rather than steady-state polled: inbox appends, protocol updates, background task completion, explicit resume, and autonomy timers wake the actor to process durable state already written to the store. After a restart, the persisted team inbox, protocol requests, and background notifications remain the source of truth, and `Runtime::resume(...)` revives teammate actors against that stored state.
## Resolving A Model
Use `Runtime::resolve_model(...)` when you want provider-aware model selection without reimplementing discovery or `ModelInfo` construction in application code.
```rust,no_run
use mentra::{BuiltinProvider, ModelSelector, Runtime};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
.build()?;
let model = runtime
.resolve_model(
BuiltinProvider::OpenAI,
std::env::var("MENTRA_MODEL")
.map(ModelSelector::Id)
.unwrap_or(ModelSelector::NewestAvailable),
)
.await?;
let _ = model;
# Ok(())
# }
```
## Coding Agent Setup
`Runtime::builder()` registers Mentra's builtin tools, including `shell`, `background_run`, `check_background`, `files`, and the runtime/task/team intrinsics. Shell and background execution remain disabled by default, so coding-agent setups must opt in with a runtime policy. If you want semantic review before tools execute, install a `ToolAuthorizer`.
The builtin local executor is a host executor, not a filesystem or network
sandbox. `RuntimePolicy::permissive()` therefore grants the model the same host
access as the Mentra process. Use it only inside a disposable container or
another boundary you trust. On a normal host, install an OS-enforced custom
executor with `RuntimeBuilder::with_executor(...)`; authorization and shell
validation decide whether a command may start, but they do not contain an
allowed command.
For Responses API transport, xipe-compatible endpoints, and provider-side state
options, see the workspace
[`Responses Coding Agent Guide`](../docs/responses-coding-agent.md).
```rust,no_run
use async_trait::async_trait;
use mentra::{BuiltinProvider, Runtime, RuntimePolicy};
use mentra::tool::{
ToolAuthorizationDecision, ToolAuthorizationRequest, ToolAuthorizer,
};
struct AllowAllAuthorizer;
#[async_trait]
impl ToolAuthorizer for AllowAllAuthorizer {
async fn authorize(
&self,
_request: &ToolAuthorizationRequest,
) -> Result<ToolAuthorizationDecision, mentra::error::RuntimeError> {
Ok(ToolAuthorizationDecision::allow())
}
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
// Full host shell access. Use only inside a trusted external sandbox.
.with_policy(RuntimePolicy::permissive())
.with_tool_authorizer(AllowAllAuthorizer)
.build()?;
let _ = runtime;
Ok(())
}
```
## Runtime Policy Defaults
Mentra's builtin runtime tools are available by default, but command execution is not:
- `Runtime::builder()` registers the builtin shell, background, file, task, team, and memory-oriented intrinsics
- foreground shell execution is disabled by default
- background command execution is disabled by default
- `RuntimePolicy::permissive()` enables both shell and background command execution
- `RuntimePolicy::workspace_bounded(...)` and `RuntimePolicy::read_only(...)` keep shell execution disabled; their roots constrain builtin file tools and the requested shell working directory, not shell process effects
- builtin shell commands run through `/bin/sh -c` on Unix and `cmd.exe /C` on Windows
- the local executor clears unlisted environment variables and enforces timeouts, output caps, and process-tree cleanup on timeout, but it does not restrict filesystem or network access
- semantic review is opt-in through `RuntimeBuilder::with_tool_authorizer(...)`
Use the default policy when you want a safer runtime surface. Opt into
`RuntimePolicy::permissive()` only when an external sandbox already contains the
entire Mentra process and full host access is intentional.
If you need different command semantics, such as PowerShell on Windows, or
filesystem/network confinement, replace the default local executor with
`RuntimeBuilder::with_executor(...)`. A workspace-bounded or read-only policy
can then explicitly enable foreground and background shell switches; Mentra
treats that executor as a trusted enforcement boundary and does not fall back
to the local executor.
### Per-session runtime policies
A shared runtime can attach a different complete policy to each live session:
```rust,no_run
use mentra::{AgentConfig, ModelInfo, Runtime, RuntimePolicy, Session};
use mentra::runtime::{SessionOptions, SessionResumeOptions};
fn create_workspace_session(
runtime: &Runtime,
model: ModelInfo,
workspace: &std::path::Path,
) -> Result<Session, mentra::error::RuntimeError> {
runtime.create_session_with_options(
"workspace",
model,
SessionOptions {
config: AgentConfig {
workspace: mentra::agent::WorkspaceConfig {
base_dir: workspace.to_path_buf(),
..Default::default()
},
..Default::default()
},
policy: Some(RuntimePolicy::workspace_bounded(workspace)),
..Default::default()
},
)
}
fn resume_workspace_session(
runtime: &Runtime,
agent_id: &str,
workspace: &std::path::Path,
) -> Result<Session, mentra::error::RuntimeError> {
runtime.resume_session_with_options(
agent_id,
SessionResumeOptions {
policy: Some(RuntimePolicy::workspace_bounded(workspace)),
..Default::default()
},
)
}
```
`None` inherits the runtime builder policy. `Some(policy)` is the authoritative
complete replacement for that live session; Mentra does not merge it with the
runtime policy. The attachment is not serialized into `AgentConfig`, so pass
the current policy again when resuming. Disposable subagents and teammates
inherit it from their live parent. This scoping does not turn `RuntimePolicy`
into an OS sandbox: builtin checks remain best-effort, and an allowed shell
command retains the authority of the configured executor.
## Tool Authorization
Mentra can run a caller-provided authorization pass before any tool executes. This is the recommended integration point for LLM-based security review, human approval, or custom policy engines.
- no authorizer installed: tools run under the remaining hard runtime constraints
- authorizer returns `Allow`: the tool executes
- authorizer returns `Deny`: Mentra blocks execution and returns an error `tool_result`
- authorizer returns `Prompt`: a raw `Agent` run blocks, while a `Session` uses the permission flow described below
- authorizer timeout or error: Mentra fails closed and blocks execution
`RuntimeBuilder::with_tool_authorizer` is the default for agents and sessions
created from that runtime. A host serving several conversations from one
runtime can replace it for one live session with
`session.with_tool_authorizer(authorizer)`. The session keeps its permission
bridge outside that replacement: `Prompt` emits `PermissionRequested` and
waits for `resolve_permission`, while `Allow` and `Deny` keep their ordinary
meaning. Sibling sessions keep the runtime default, and descendants created
from the decorated session inherit its replacement.
Every session has a process-local remembered-rule rung ahead of its runtime
store. `PermissionRuleScope::Process` belongs to that one live
`SessionPermissionHandle` binding: handle clones and the installed authorizer
share it, but a separately created or resumed session starts empty. It is never
written to `PermissionRuleStore`, and a match answers without reading the
durable backend. The durable `Session` namespace is instead the persisted agent
id—not the fresh UI `SessionId` created on resume—and an optional project id
adds project-wide inheritance. Lookup order is `Process`, `Session`, `Project`,
then `Global`.
`SessionPermissionHandle::{remember_rule, revoke_rule, clear_scope,
remembered_rules}` route each mutation to the live or durable namespace it
names; project and global changes are visible to other already-live sessions on
their next authorization lookup. Effective-rule listing includes both sources
and remains fallible because it reads the durable store. No store attachment or
manual rule reload is required.
The attachment is live execution state rather than persisted agent
configuration. Attach the current authorizer again after `resume_session`.
Stateful authorizers can hold a shared mode or policy and change their answer
between calls without replacing the attachment.
Every authorization request includes a `ToolAuthorizationPreview` with tool metadata plus structured input. Builtin tools provide more specific previews:
- `shell` and `background_run` include the raw command, resolved working directory, timeout, background flag, and justification
- `files` includes resolved paths and operation kinds such as `read`, `search`, `set`, `move`, and `delete`, without file contents
```rust,no_run
use async_trait::async_trait;
use mentra::tool::{
ToolAuthorizationDecision, ToolAuthorizationRequest, ToolAuthorizer,
};
struct DenyDeletes;
#[async_trait]
impl ToolAuthorizer for DenyDeletes {
async fn authorize(
&self,
request: &ToolAuthorizationRequest,
) -> Result<ToolAuthorizationDecision, mentra::error::RuntimeError> {
let structured = &request.preview.structured_input;
let denies_delete = structured
.get("operations")
.and_then(|value| value.as_array())
.is_some_and(|ops| ops.iter().any(|op| op.get("op").and_then(|v| v.as_str()) == Some("delete")));
if request.tool_name == "files" && denies_delete {
Ok(ToolAuthorizationDecision::deny("delete operations require manual approval"))
} else {
Ok(ToolAuthorizationDecision::allow())
}
}
}
```
Registering a skills directory also makes the builtin `load_skill` tool available:
```rust,no_run
use mentra::{BuiltinProvider, Runtime};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
.with_skills_dir("./skills")?
.build()?;
let _ = runtime;
Ok(())
}
```
## App Context
If your tools need access to typed host-side state, register it on the runtime and retrieve it from `ToolContext` or `ParallelToolContext`:
```rust,no_run
use std::sync::Arc;
use async_trait::async_trait;
use mentra::{
BuiltinProvider, Runtime,
tool::{ToolContext, ToolDefinition, ToolExecutor, ToolResult, ToolSpec},
};
use serde_json::{Value, json};
struct AppState {
api_base: String,
}
struct InspectStateTool;
impl ToolDefinition for InspectStateTool {
fn descriptor(&self) -> ToolSpec {
ToolSpec::builder("inspect_state")
.description("Return the configured API base URL.")
.input_schema(json!({
"type": "object",
"properties": {}
}))
.build()
}
}
#[async_trait]
impl ToolExecutor for InspectStateTool {
async fn execute_mut(&self, ctx: ToolContext<'_>, _input: Value) -> ToolResult {
let state = ctx.app_context::<AppState>()?;
Ok(state.api_base.clone())
}
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
.with_context(Arc::new(AppState {
api_base: "https://api.example.com".to_string(),
}))
.with_tool(InspectStateTool)
.build()?;
let _ = runtime;
Ok(())
}
```
## Custom Tools
Use `ToolSpec::builder(...)` to define custom tools without hand-assembling the metadata struct:
```rust,no_run
use async_trait::async_trait;
use mentra::tool::{
ParallelToolContext, ToolCapability, ToolDefinition, ToolDurability, ToolExecutor,
ToolResult, ToolSideEffectLevel, ToolSpec,
};
use serde_json::{Value, json};
struct UppercaseTool;
impl ToolDefinition for UppercaseTool {
fn descriptor(&self) -> ToolSpec {
ToolSpec::builder("uppercase_text")
.description("Uppercase the provided text")
.input_schema(json!({
"type": "object",
"properties": {
"text": { "type": "string" }
},
"required": ["text"]
}))
.capability(ToolCapability::ReadOnly)
.side_effect_level(ToolSideEffectLevel::None)
.durability(ToolDurability::ReplaySafe)
.execution_timeout(std::time::Duration::from_secs(5))
.build()
}
}
#[async_trait]
impl ToolExecutor for UppercaseTool {
async fn execute(&self, _ctx: ParallelToolContext, input: Value) -> ToolResult {
let text = input
.get("text")
.and_then(|value| value.as_str())
.ok_or_else(|| "text is required".to_string())?;
Ok(text.to_uppercase())
}
}
```
`ToolSpec::execution_timeout(...)` is enforced by Mentra around the tool future itself, which is useful for network-backed tools that need a tighter budget than the overall agent run.
Internally, Mentra translates `ToolSpec` into a runtime-only `RuntimeToolDescriptor`, but custom runtime integrations should continue to treat `ToolSpec::builder(...)` as the supported public metadata surface. `ExecutableTool` remains available in this release as a compatibility trait alias over `ToolDefinition + ToolExecutor`.
When a tool needs disposable delegated work, `ParallelToolContext::spawn_subagent()` can create a child agent that inherits the current runtime and model defaults. See the `subagent_tool` example in the workspace examples crate for a complete usage pattern.
Override `ToolExecutor::authorization_preview(...)` when your custom tool needs to expose structured metadata to the installed `ToolAuthorizer`. The default preview includes the resolved working directory, tool capabilities, side-effect level, durability, the raw JSON input, and the same JSON as `structured_input`.
## Audience-Scoped Tools
Use `ToolAudience` when one live runtime serves several workspaces or tenants
whose custom tool sets must remain distinct. The identity is opaque routing
context, not a permission or credential. Keep the returned registration guard
alive for as long as the tool should be available, and attach the audience
explicitly whenever an agent or session is created or resumed:
```rust,no_run
use mentra::runtime::{SessionOptions, SessionResumeOptions};
use mentra::tool::ExecutableTool;
use mentra::{AudienceToolRegistration, ModelInfo, Runtime, Session, ToolAudience};
fn open_workspace<T>(
runtime: &Runtime,
model: ModelInfo,
tool: T,
) -> Result<(Session, AudienceToolRegistration), Box<dyn std::error::Error>>
where
T: ExecutableTool + 'static,
{
let audience = ToolAudience::new("workspace-open-42");
let registration = runtime.try_register_tool_for_audience(audience.clone(), tool)?;
let session = runtime.create_session_with_options(
"Acme workspace",
model,
SessionOptions {
tool_audience: Some(audience),
..Default::default()
},
)?;
Ok((session, registration))
}
fn resume_workspace(
runtime: &Runtime,
agent_id: &str,
audience: ToolAudience,
) -> Result<Session, mentra::error::RuntimeError> {
runtime.resume_session_with_options(
agent_id,
SessionResumeOptions {
tool_audience: Some(audience),
..Default::default()
},
)
}
```
For raw agents, use `spawn_with_config_for_audience`,
`resume_agent_for_audience`, or `resume_for_audience`. The ordinary spawn and
resume methods deliberately attach no audience.
Resolution is deterministic: an exact-agent intrinsic wins first, then a tool
from the matching audience, then a global tool. Different audiences may use
the same name. Safe audience registration rejects a global or same-audience
collision, while safe global registration rejects a collision in any scope.
The legacy infallible global registration APIs deliberately evict every
same-name scoped entry.
Dropping `AudienceToolRegistration` (or consuming it with `unregister`) removes
only that exact generation; an already admitted call may still finish. The
descriptor is evaluated once and is available from the guard. Registrations
are shared live, so matching sessions that already exist observe later
registration and removal. `Runtime::tools`, `Runtime::tool_descriptor`, and
`Runtime::unregister_tool` remain global-only.
Audiences are not persisted in `AgentConfig`; pass them again on resume.
Disposable subagents and teammates inherit their live parent's audience.
`ToolProfile` can only narrow the roster already available through that scope;
it is not ownership or security provenance. A guessed foreign tool name is
rejected before hooks, authorization, or tool execution.
## Live Execution Hooks
`RuntimeBuilder::with_pre_hook` and `with_post_hook` install permanent hooks
before a runtime exists. A long-lived runtime can add hooks later with
`Runtime::register_pre_hook` and `register_post_hook`. Keep each returned guard
alive while the hook should apply; dropping it, or consuming it with
`unregister`, removes only that exact registration. Existing agents and sessions
use the live registry, so their next hook snapshot observes the change.
Use `register_pre_hook_for_audience` and
`register_post_hook_for_audience` for workspace- or tenant-owned hooks. The
scope is the opaque `ToolAudience` carried by the live runtime handle, not the
hook context's `working_directory`: two audiences may point at the same path
without seeing one another's hooks, and an agent with no audience sees only
global hooks. Audiences are routing identity, not authentication, and must be
reattached on resume.
Within each seam, all applicable hooks share one order. Permanent builder hooks
come first, then live global and matching-audience hooks in the order registered
for that seam. Pre-execution walks its order forward; post-execution walks its
own exact reverse, preserving the outer-hook wrapping contract. The two seams
are registered independently. Registering the same hook independently as both
global and matching-audience intentionally invokes it twice.
Each pre or post invocation snapshots its applicable hooks before awaiting user
code. Removing a registration affects later snapshots but does not cancel one
already running. The two seams snapshot independently: if a host needs them to
bracket a tool call, it must retain both guards until the call has quiesced.
Dropping a post guard after pre admission does not guarantee the result will be
reviewed.
## Ordered Mixed Execution Hooks
Use `ExecutionHookParticipant` when in-process and transported participants
must occupy one exact order. Each participant has a required `name` and default
`before`/`after` methods. `BeforeDecision` can continue, deny, or modify input;
`AfterDecision` can continue, deny, or replace a result while optionally
preserving its current `is_error`. Denials are named and short-circuit, while
every modification's participant and attribution are retained in order.
The mixed chain is independent of the legacy hook containers and runs forward
on both sides. The complete runtime order is:
```text
legacy pre hooks (forward)
mixed participants before (forward)
tool execution
mixed participants after (forward)
legacy post hooks (reverse)
```
Forward-after is intentional and differs from legacy post hooks: a host adapter
placed before a workspace subprocess can redact output before the subprocess
receives it. A mixed after-denial prevents remaining mixed participants and the
legacy post block from running, then reaches the model as an error result. A
mixed replacement is threaded into every later mixed participant and then the
legacy post block.
Install permanent participants with `RuntimeBuilder::with_execution_hook` or
`with_execution_hooks`. Live runtimes provide matching single and atomic-batch
`register_execution_*` methods, including `*_for_audience`. Use the batch API
for heterogeneous participants whose relative order must become visible as one
unit; one must-use guard owns the entire batch lifetime.
One matching-audience snapshot is captured after legacy pre hooks and retained
through each admitted tool call. The same participants run after a genuine
serial or parallel execution even if the guard drops meanwhile. Late
registration governs the next admission and cannot appear only on the way out.
Participant futures run without registry locks. A participant `Err` propagates
as `RuntimeError`; participant adapters should translate their own expected
fail-open/fail-closed, reporting, subprocess, or panic policy into typed
decisions.
## Tooling Layers
Mentra now separates tool contracts into explicit layers:
- `ProviderToolSpec` in `mentra-provider` for provider-facing serialization
- `RuntimeToolDescriptor` in Mentra for scheduling, approval, and durability metadata
- `ToolDefinition + ToolExecutor` for executable runtime tools
Provider adapters should serialize provider-facing tool specs only. Runtime integrations should continue to implement custom tools with `ToolSpec::builder(...)`, `ToolDefinition`, and `ToolExecutor`.
## Hosted Tool Search
Mentra can mark custom tools as deferred and let a provider load them on demand with native hosted tool search.
Mark a tool as deferred in its `ToolSpec`:
```rust,no_run
use async_trait::async_trait;
use mentra::tool::{ParallelToolContext, ToolDefinition, ToolExecutor, ToolResult, ToolSpec};
use serde_json::{Value, json};
struct LookupOrderTool;
impl ToolDefinition for LookupOrderTool {
fn descriptor(&self) -> ToolSpec {
ToolSpec::builder("lookup_order")
.description("Look up an order by id.")
.input_schema(json!({
"type": "object",
"properties": {
"order_id": { "type": "string" }
},
"required": ["order_id"]
}))
.defer_loading(true)
.build()
}
}
#[async_trait]
impl ToolExecutor for LookupOrderTool {
async fn execute(&self, _ctx: ParallelToolContext, _input: Value) -> ToolResult {
Ok("order loaded".to_string())
}
}
```
Enable hosted tool search per agent with `ProviderRequestOptions`:
```rust,no_run
use mentra::agent::AgentConfig;
use mentra::provider::{ProviderRequestOptions, ReasoningEffort, ReasoningOptions, ToolSearchMode};
let config = AgentConfig {
provider_request_options: ProviderRequestOptions {
tool_search_mode: ToolSearchMode::Hosted,
reasoning: Some(ReasoningOptions {
effort: Some(ReasoningEffort::Medium),
summary: None,
}),
..Default::default()
},
..Default::default()
};
```
Current provider support:
- OpenAI: supported through the Responses API hosted `tool_search` surface
- Anthropic: supported through the Messages API BM25 tool-search server tool
- Gemini: deferred custom tools are not supported; Mentra returns `InvalidRequest`
Reasoning effort support:
- The shared levels are `low`, `medium`, `high`, `xhigh`, and `max`; omitting
effort leaves the provider default unchanged.
- OpenAI and OpenRouter: Mentra forwards all five levels as
`reasoning.effort` on the Responses API.
- Anthropic: Mentra writes the requested level to `output_config.effort` and
enables adaptive thinking on models that support it. Opus 4.5 accepts
`low`/`medium`/`high` effort without adaptive thinking; availability of
`xhigh` and `max` depends on the Claude model.
- Gemini: Mentra maps the shared `low`, `medium`, and `high` levels to
`thinkingLevel` on Gemini 3 models, subject to that model's accepted values.
`xhigh` and `max` return `InvalidRequest` instead of being silently
downgraded.
- Anthropic models without effort support and Gemini models older than 3 return
`InvalidRequest` when unified reasoning effort is set.
Deferred tools are filtered through `ToolProfile` just like immediate tools. If you force a deferred tool with `ToolChoice::Tool { name }`, Mentra serializes that specific tool as immediate for the request so explicit invocation still works.
## Model Context Protocol Servers
Mentra connects to external MCP servers and bridges every tool they advertise
into the runtime under a namespaced `mcp__<server>__<tool>` name. Bridged tools
run through the same authorization, result limiter, and paging path as builtin
and custom tools.
Three transports are supported, selected by which configuration type you
register.
**stdio** spawns the server as a child process:
```rust,no_run
use mentra::{BuiltinProvider, McpServerConfig, Runtime};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::Anthropic, std::env::var("ANTHROPIC_API_KEY")?)
.with_mcp_server(McpServerConfig {
name: "filesystem".to_string(),
command: "npx".to_string(),
args: vec![
"-y".to_string(),
"@modelcontextprotocol/server-filesystem".to_string(),
"/tmp".to_string(),
],
env: Default::default(),
cwd: None,
})
.build_async()
.await?;
# let _ = runtime;
# Ok(())
# }
```
**Streamable HTTP** reaches a hosted server over the network, and is the
transport current MCP servers ship:
```rust,no_run
use mentra::{BuiltinProvider, McpStreamableHttpServerConfig, Runtime};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::Anthropic, std::env::var("ANTHROPIC_API_KEY")?)
.with_mcp_streamable_http_server(
McpStreamableHttpServerConfig::new("observability", "https://mcp.example.com/mcp")
.with_bearer_token(std::env::var("MCP_TOKEN")?),
)
.build_async()
.await?;
# let _ = runtime;
# Ok(())
# }
```
Every JSON-RPC message is a `POST` to the one configured URL, sent with
`Accept: application/json, text/event-stream` — a streamable-only server
answers `406` to a request that does not offer both, because it chooses the
framing per reply. The reply comes back on that same response, either as one
JSON body or as an event stream the server opens in it. If `initialize` assigns
an `Mcp-Session-Id`, every later request carries it and `shutdown` ends the
session with a `DELETE`.
Because a reply arrives on the response to the request that asked for it, this
client needs no background reader and no pending-request map. It still matches
each reply to its JSON-RPC id: on the streaming path a reply for a different id
is skipped rather than returned, so a server cannot hand you another call's
result by putting it on the wire first.
Nothing is retried or replayed. A `tools/call` whose request may have reached
the server but whose reply never arrived surfaces as
`McpStreamableHttpError::RequestIndeterminate` — the tool may have run. A `4xx`
is exempt, since those are rejections the server makes before dispatching the
message; a `5xx` is not, because a server can fail after running the tool.
**Legacy HTTP+SSE** reaches a hosted server that predates Streamable HTTP:
```rust,no_run
use mentra::{BuiltinProvider, McpSseServerConfig, Runtime};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::Anthropic, std::env::var("ANTHROPIC_API_KEY")?)
.with_mcp_sse_server(
McpSseServerConfig::new("observability", "https://mcp.example.com/sse")
.with_bearer_token(std::env::var("MCP_TOKEN")?),
)
.build_async()
.await?;
# let _ = runtime;
# Ok(())
# }
```
A server that answers `404` on `/mcp` but serves `/sse` needs this transport.
Reach for Streamable HTTP first; this one exists for servers that never
implemented it.
### HTTP+SSE is not Streamable HTTP
`McpSseServerConfig` speaks the transport from MCP protocol revision
`2024-11-05`, which is a different protocol from the newer Streamable HTTP:
| Endpoints | a `GET` stream plus a separate `POST` URL | one URL for both |
| POST target | named by the server in an `endpoint` event | the configured URL |
| Responses | always on the `GET` stream | in the POST response or a stream |
| Session | a query parameter in the endpoint URL | the `Mcp-Session-Id` header |
The client opens the configured URL with `Accept: text/event-stream`, waits for
an `endpoint` event naming the POST URL, then posts `initialize`, a
`notifications/initialized` notification, and a paginated `tools/list`. Servers
answer each POST `202 Accepted` and deliver the actual JSON-RPC result as a
`message` event on the stream.
### Security and failure behavior
The endpoint URL is chosen by the server, so it is validated before anything is
sent to it. A resolved endpoint must match the configured URL's scheme, host,
and effective port; a cross-origin endpoint, a protocol-relative `//other.host`
value, embedded credentials, and non-`http(s)` schemes are all refused. Redirects
are never followed on either request.
Configured headers are sent on both the stream and every POST, stored as
`SecretString` so they never appear in `Debug` output, errors, or logs.
Configuring headers against a plaintext `http://` URL on a non-loopback host is
rejected unless `allowing_plaintext_credentials()` is set. No error carries a
response body or SSE payload, so a malicious server cannot write text into your
logs.
Losing the stream ends the session — the client fails closed rather than
hanging, and never reconnects or re-sends a `tools/call`. A call whose response
never arrived surfaces as `McpSseError::RequestIndeterminate`, because the POST
and the response travel on different connections: the tool may have run. Treat
that differently from a rejected POST, which definitely did not execute.
### Using the client directly
Hosts that need their own allowlist, redaction, or evidence policy can drive
`McpSseClient` without registering anything:
```rust,no_run
use mentra::{McpSseClient, McpSseServerConfig};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let config = McpSseServerConfig::new("observability", "https://mcp.example.com/sse")
.with_bearer_token(std::env::var("MCP_TOKEN")?);
let client = McpSseClient::connect(&config).await?;
for tool in client.tools() {
println!("{}", tool.name);
}
let result = client
.call_tool("search_logs", Some(serde_json::json!({"query": "error"})))
.await?;
println!("{}", result.is_error);
client.shutdown().await;
# Ok(())
# }
```
## Tool Profiles
`ToolProfile` filters the roster already visible to an agent; it cannot grant a
tool from another audience. Register tools once on the runtime, then use
`AgentConfig::tool_profile` to expose different subsets for different operating
modes.
```rust,no_run
use mentra::{BuiltinProvider, ModelSelector, Runtime};
use mentra::agent::{AgentConfig, ToolProfile};
# async fn demo() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::builder()
.with_provider(BuiltinProvider::OpenAI, std::env::var("OPENAI_API_KEY")?)
.build()?;
let model = runtime
.resolve_model(
BuiltinProvider::OpenAI,
ModelSelector::Id("gpt-5.4-mini".to_string()),
)
.await?;
let queue_mode = AgentConfig {
tool_profile: ToolProfile::only([
"shell",
"background_run",
"check_background",
"files",
"task",
]),
..Default::default()
};
let direct_mode = AgentConfig {
tool_profile: ToolProfile::hide(["task", "background_run"]),
..Default::default()
};
let _queue_agent = runtime.spawn_with_config("Queue Agent", model.clone(), queue_mode)?;
let _direct_agent = runtime.spawn_with_config("Direct Agent", model, direct_mode)?;
# Ok(())
# }
```
This is the recommended pattern when one application needs multiple tool surfaces such as a queue-backed agent with delegation enabled and a direct mode that keeps the same runtime but hides long-running or task-oriented tools.
## CLI Integration Pattern
For CLI-style coding or analysis tools, the usual setup is:
- register a superset of builtin and custom tools on one runtime
- scope shell and file access with `RuntimePolicy`
- keep application-specific output paths in app context for custom tools
- switch behavior per mode by changing `AgentConfig::tool_profile`, not by rebuilding the runtime
- inspect `agent.history()` after the run when you want to render a compact tool log or transcript summary
The `cli_runtime` example in the workspace examples crate shows this pattern end to end with custom tools, policy setup, mode-specific tool surfaces, and transcript inspection.
## Disposable Tasks vs Persistent Teams
Mentra supports two different delegation models:
- use the builtin `task` tool or `ParallelToolContext::spawn_subagent()` for short-lived disposable delegation that should return a single summary to the parent
- use `team_spawn`, `team_send`, `team_read_inbox`, `team_request`, and `team_respond` when you want a persistent teammate with a durable mailbox and request/response workflow across turns
The `task` path is ideal for one-off decomposition inside a single run. The `team_*` tools are for longer-lived collaborators that should keep state, receive follow-up work, and participate in approval or shutdown flows.
## Sending Images
You can attach image blocks alongside text when sending a user turn:
```rust,no_run
# use mentra::{ContentBlock, Agent};
# async fn demo(agent: &mut Agent) -> Result<(), Box<dyn std::error::Error>> {
agent
.send(vec![
ContentBlock::text("What is happening in this screenshot?"),
ContentBlock::image_bytes("image/png", std::fs::read("screenshot.png")?),
])
.await?;
# Ok(())
# }
```
For already-hosted assets, use `ContentBlock::image_url(...)` instead. Gemini currently supports inline `image_bytes(...)` inputs only and rejects `image_url(...)`.
## Long-Term Memory
Agents automatically recall from long-term memory by default. When you use `Runtime::builder()`, the builtin runtime intrinsics include:
- `memory_search` for explicit recall
- `memory_pin` for writing important facts
- `memory_forget` for tombstoning a specific memory record
`MemoryConfig` controls recall and write behavior per agent. The default configuration enables automatic recall and memory write tools, which is useful for long-running assistants and teammate workflows. Disable write tools when you want recall without model-initiated mutation.
## Context Compaction
Mentra separates canonical summary compaction from two request-only projection policies:
- **Summary compaction** is enabled by default. When estimated request context
crosses its threshold, Mentra writes the full transcript to the default
transcript directory and replaces older history with a model-generated
summary. The default threshold is 75% of a known model context window, with
50k tokens as the fallback when the window is unknown. The model can also call
the builtin `compact` tool explicitly. This changes canonical history and
emits the normal compaction events.
- **Request-only tool-result elision** is disabled by default. Setting
`keep_recent_tool_results` to a finite `N` leaves the newest `N` projected
results unchanged and replaces eligible older payloads over 100 bytes with
`[Previous: used <tool>]` in each main model request. The canonical transcript
is unchanged by this projection. Each changed projection emits
`AgentEvent::RequestToolResultsElided`; a `Session` maps the same facts to
`SessionEvent::RequestToolResultsElided`.
- **Request-only tool-result budgeting** is also disabled by default. Setting
`projected_tool_result_budget` selects it instead of the legacy recent-count
policy. `max_bytes` is a hard aggregate cap over final provider-neutral tool
result body bytes—not roles, call ids, JSON/wire framing, tool definitions, or
other request content. A floor keeps short originals or descriptive markers
where possible, then degrades lower-priority results to ellipsis or empty text
when even those markers exceed the strict cap. Whole recent bodies, bounded
UTF-8 head/tail previews, and whole historical bodies then receive budget in
that order. Recent priority is not an exemption from the cap, and structured
JSON is never sliced.
A finite recent count is a lossy heuristic, not a byte or token limit: recent
results remain unbounded, short old results survive, and markers accumulate.
Use it only when old results are disposable. Tool-result paging can provide
sequential window access within the same live agent when its reader is offered;
it is text-only, and one line longer than a page is deliberately hard-cut. It
runs after the output limiter, so to use its default 64 KiB threshold, raise the
limiter above its default 50 KiB cap.
Budget mode adds no recovery channel. It operates on whatever canonical result
the limiter, post-execution hook, and optional pager placed in history. An
existing paging trailer may survive as ordinary tail text, but paging state is
live-agent-only and is not restored after resume. Auto-compaction measures the
same budget-shaped main-request projection that is ultimately sent, so enabling
the budget can delay or avoid summary compaction.
You can tune or disable this per-agent with `CompactionConfig`:
```rust
use mentra::agent::{AgentConfig, CompactionConfig, ProjectedToolResultBudget};
let config = AgentConfig {
compaction: CompactionConfig {
auto_compact_threshold_tokens: Some(75_000),
auto_compact_threshold_percent: Some(80),
projected_tool_result_budget: Some(ProjectedToolResultBudget {
max_bytes: 128 * 1024,
prioritize_recent_results: 4,
max_preview_bytes: 8 * 1024,
}),
..Default::default()
},
..Default::default()
};
```
`ProjectedToolResultBudget` intentionally has no default: all lossy limits must
be explicit. Its persisted field requires Mentra 0.22 or later; older binaries
ignore it and therefore cannot enforce the cap.
`auto_compact_trigger` decides which of those two numbers is consulted, and
whether auto-compaction runs at all. The default, `AutoCompactTrigger::Thresholds`,
resolves them exactly as earlier versions did — the window share when the window
is known, the absolute number otherwise, and off when the absolute number is
`None`. `AutoCompactTrigger::WindowShareOnly` compacts strictly at
`auto_compact_threshold_percent` of a *known* context window and never
auto-compacts when the window is unknown, so a host with window-relative policy
does not have to invent an absolute count that goes live for exactly the models
whose window it does not know. `AutoCompactTrigger::Off` turns auto-compaction
off without discarding either number, and `CompactionConfig::auto_compact_enabled`
reports that state without resolving a threshold.
## Data And Persistence Defaults
For non-test builds, Mentra keeps all default persisted state under a workspace-scoped app-data directory:
- store: `<platform data dir>/mentra/workspaces/<workspace-hash>/runtime.sqlite`
- runtime-scoped stores: `<platform data dir>/mentra/workspaces/<workspace-hash>/runtime-<runtime-id>.sqlite`
- team state: `<platform data dir>/mentra/workspaces/<workspace-hash>/team/`
- task state: `<platform data dir>/mentra/workspaces/<workspace-hash>/tasks/`
- transcripts: `<platform data dir>/mentra/workspaces/<workspace-hash>/transcripts/`
If the platform data directory cannot be resolved, Mentra falls back to `.mentra/workspaces/<workspace-hash>/...` inside the current workspace.
Override these defaults when needed:
- use `Runtime::builder().with_store(...)` for the SQLite store
- customize `AgentConfig::task.tasks_dir`, `AgentConfig::team.team_dir`, and `AgentConfig::compaction.transcript_dir` for task, team, and transcript storage
## Persistence Extension Points
The public persistence surface is intentionally split into narrower traits:
- `AgentStore` for agent records and working-memory snapshots
- `RunStore` for turn and run lifecycle tracking
- `TaskStore` for the dependency-aware task board
- `LeaseStore` for runtime ownership and resume coordination
`RuntimeStore` composes those traits with `TeamStore`, `BackgroundStore`, and `MemoryStore`. `SqliteRuntimeStore` is the default all-in-one backend. `HybridRuntimeStore` keeps SQLite runtime state and swaps in the hybrid memory engine for richer long-term memory behavior.
## Testing With MockRuntime
Enable the `test-utils` feature when you want a deterministic scripted runtime for unit and integration tests.
`mentra::test::MockRuntime` wraps a real runtime with:
- a scripted provider
- a `VolatileRuntimeStore`, so a mock writes nothing to disk and two mocks never
share state — pass `MockRuntimeBuilder::with_store` a `SqliteRuntimeStore`
when a test needs state that outlives the mock
- deterministic per-turn helper methods for assistant text, streamed text, tool-call turns, and provider failures
This is the recommended way to test Mentra-based agents and tools without live API keys.
The common pattern is:
- build a `MockRuntime`
- register the same custom tools you use in production
- spawn an agent with the `AgentConfig` or `ToolProfile` you want to verify
- assert against `mock.recorded_requests()` to confirm the runtime exposed the expected tools and tool-choice hints
See `mentra::test` and the crate tests for a full example of asserting runtime assembly with custom tools and filtered tool surfaces.
## Interactive Repo Example
Clone the repository when you want the richer interactive demo with provider selection, persisted runtime inspection, skills loading, and team/task visibility.
Set `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY`, then run. The example lets you choose a provider and shows up to 10 models from that provider ordered newest to oldest.
```bash
cargo run -p mentra-examples --example chat
```
Additional focused examples live in the same crate:
```bash
cargo run -p mentra-examples --example custom_tool
cargo run -p mentra-examples --example subagent_tool
cargo run -p mentra-examples --example team_collaboration
cargo run -p mentra-examples --example cli_runtime -- --mode direct
```
`cli_runtime` is the closest example to a real integration. It combines runtime policy setup, custom tools, mode-specific `ToolProfile` selection, and transcript inspection after the run.
## Run Checks
```bash
cargo fmt --all --check
cargo +1.88.0 check --workspace --all-targets --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features
cargo test -p mentra --no-default-features
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --all-features
```