Module tool 

Tool execution engine.

This module provides the runtime layer for executing tools that LLMs invoke during generation. It builds on the foundational types from chat (ToolCall, ToolResult) and provider (ToolDefinition, JsonSchema).

Architecture

  ToolHandler        — defines a single tool (schema + execute fn)
      │
  ToolRegistry       — stores handlers by name, validates & dispatches
      │
  tool_loop()           — automates generate → execute → feedback cycle
  tool_loop_stream()    — unified LoopEvent stream (LLM deltas + loop lifecycle)
  ToolLoopHandle        — caller-driven resumable variant (borrowed refs)
  OwnedToolLoopHandle   — caller-driven resumable variant (Arc, Send + 'static)

Example

use llm_stack::tool::{ToolRegistry, tool_fn, ToolLoopConfig, tool_loop};
use llm_stack::{ChatParams, ChatMessage, JsonSchema, ToolDefinition};
use serde_json::{json, Value};

let mut registry: ToolRegistry<()> = ToolRegistry::new();
registry.register(tool_fn(
    ToolDefinition {
        name: "add".into(),
        description: "Add two numbers".into(),
        parameters: JsonSchema::new(json!({
            "type": "object",
            "properties": {
                "a": {"type": "number"},
                "b": {"type": "number"}
            },
            "required": ["a", "b"]
        })),
        retry: None,
    },
    |input: Value| async move {
        let a = input["a"].as_f64().unwrap_or(0.0);
        let b = input["b"].as_f64().unwrap_or(0.0);
        Ok(format!("{}", a + b))
    },
));

let params = ChatParams {
    messages: vec![ChatMessage::user("What is 2 + 3?")],
    tools: Some(registry.definitions()),
    ..Default::default()
};

let result = tool_loop(provider, &registry, params, ToolLoopConfig::default(), &()).await?;
println!("Final answer: {:?}", result.response.text());

Using Context

Tools often need access to shared state like database connections, user identity, or configuration. Use tool_fn_with_ctx to create tools that receive context:

use llm_stack::tool::{tool_fn_with_ctx, ToolRegistry, ToolError, ToolOutput, tool_loop, ToolLoopConfig, LoopContext};
use llm_stack::{ToolDefinition, JsonSchema, ChatParams, ChatMessage};
use serde_json::{json, Value};

#[derive(Clone)]
struct AppState {
    user_id: String,
    api_key: String,
}

type AppCtx = LoopContext<AppState>;

let handler = tool_fn_with_ctx(
    ToolDefinition {
        name: "get_user_data".into(),
        description: "Fetch data for the current user".into(),
        parameters: JsonSchema::new(json!({"type": "object"})),
        retry: None,
    },
    |_input: Value, ctx: &AppCtx| {
        // Clone data from context before the async block
        let user_id = ctx.state.user_id.clone();
        async move {
            Ok(ToolOutput::new(format!("Data for user: {}", user_id)))
        }
    },
);

let mut registry: ToolRegistry<AppCtx> = ToolRegistry::new();
registry.register(handler);

let ctx = LoopContext::new(AppState {
    user_id: "user123".into(),
    api_key: "secret".into(),
});

let params = ChatParams {
    messages: vec![ChatMessage::user("Get my data")],
    tools: Some(registry.definitions()),
    ..Default::default()
};

let result = tool_loop(provider, &registry, params, ToolLoopConfig::default(), &ctx).await?;

Note on lifetimes: The closure passed to tool_fn_with_ctx uses higher-ranked trait bounds (for<'c> Fn(Value, &'c Ctx) -> Fut). This means the future returned by your closure must be 'static — it cannot borrow from the context reference. Clone any data you need from the context before creating the async block.

Structs

Completed

Terminal: the loop completed successfully.

FnToolHandler

A tool handler backed by an async closure.

LoopContext

Generic context wrapper with built-in depth tracking.

LoopDetectionConfig

Configuration for detecting repeated tool calls (stuck agents).

NoCtxToolHandler

A tool handler without context, created by super::tool_fn.

OwnedToolLoopHandle

Arc-owned resumable tool loop.

OwnedYielded

Handle returned when tools were executed on an OwnedToolLoopHandle.

StopContext

Context provided to stop condition callbacks.

ToolError

Error returned by tool execution.

ToolLoopConfig

Configuration for tool_loop and tool_loop_stream.

ToolLoopHandle

Caller-driven resumable tool loop.

ToolLoopResult

The result of a completed tool loop.

ToolOutput

Output returned by a tool handler.

ToolRegistry

A registry of tool handlers, indexed by name.

TurnError

Terminal: the loop errored.

Yielded

Handle returned when tools were executed. Borrows the ToolLoopHandle mutably, so the caller cannot call next_turn() again until this is consumed via resume(), continue_loop(), inject_and_continue(), or stop().

Enums

LoopAction

Action to take when a tool call loop is detected.

LoopCommand

Commands sent by the caller to control the resumable loop.

LoopEvent

Unified event emitted during tool loop execution.

OwnedTurnResult

Result of one turn of the owned tool loop.

StopDecision

Decision returned by a stop condition callback.

TerminationReason

Why a tool loop terminated.

ToolApproval

Result of approving a tool call before execution.

TurnResult

Result of one turn of the tool loop.

Traits

LoopDepth

Trait for contexts that support automatic depth tracking in nested tool loops.

ToolHandler

A single tool that can be invoked by the LLM.

Functions

tool_fn

Creates a ToolHandler<()> from a closure (no context).

tool_fn_with_ctx

Creates a ToolHandler<Ctx> from a closure that receives context.

tool_loop

Runs the LLM in a tool-calling loop until completion.

tool_loop_stream

Streaming variant of tool_loop.

Type Aliases

LoopStream

A pinned, boxed, Send stream of LoopEvent results.

StopConditionFn

Callback type for stop conditions.

ToolApprovalFn

Callback type for tool call approval.