Module tool 

Tool execution engine.

This module provides the runtime layer for executing tools that LLMs invoke during generation. It builds on the foundational types from chat (ToolCall, ToolResult) and provider (ToolDefinition, JsonSchema).

Architecture

  ToolHandler        — defines a single tool (schema + execute fn)
      │
  ToolRegistry       — stores handlers by name, validates & dispatches
      │
  tool_loop()        — automates generate → execute → feedback cycle
  tool_loop_stream() — streaming variant
  ToolLoopHandle     — caller-driven resumable variant

Example

use llm_stack::tool::{ToolRegistry, tool_fn, ToolLoopConfig, tool_loop};
use llm_stack::{ChatParams, ChatMessage, JsonSchema, ToolDefinition};
use serde_json::{json, Value};

let mut registry: ToolRegistry<()> = ToolRegistry::new();
registry.register(tool_fn(
    ToolDefinition {
        name: "add".into(),
        description: "Add two numbers".into(),
        parameters: JsonSchema::new(json!({
            "type": "object",
            "properties": {
                "a": {"type": "number"},
                "b": {"type": "number"}
            },
            "required": ["a", "b"]
        })),
        retry: None,
    },
    |input: Value| async move {
        let a = input["a"].as_f64().unwrap_or(0.0);
        let b = input["b"].as_f64().unwrap_or(0.0);
        Ok(format!("{}", a + b))
    },
));

let params = ChatParams {
    messages: vec![ChatMessage::user("What is 2 + 3?")],
    tools: Some(registry.definitions()),
    ..Default::default()
};

let result = tool_loop(provider, &registry, params, ToolLoopConfig::default(), &()).await?;
println!("Final answer: {:?}", result.response.text());

Using Context

Tools often need access to shared state like database connections, user identity, or configuration. Use tool_fn_with_ctx to create tools that receive context:

use llm_stack::tool::{tool_fn_with_ctx, ToolRegistry, ToolError, ToolOutput, tool_loop, ToolLoopConfig, LoopDepth};
use llm_stack::{ToolDefinition, JsonSchema, ChatParams, ChatMessage};
use serde_json::{json, Value};

// Your application context - must implement Clone for LoopDepth
#[derive(Clone)]
struct AppContext {
    user_id: String,
    api_key: String,
    depth: u32,
}

// Implement LoopDepth for automatic depth tracking in nested loops
impl LoopDepth for AppContext {
    fn loop_depth(&self) -> u32 { self.depth }
    fn with_depth(&self, depth: u32) -> Self {
        Self { depth, ..self.clone() }
    }
}

// Create a tool that uses context
let handler = tool_fn_with_ctx(
    ToolDefinition {
        name: "get_user_data".into(),
        description: "Fetch data for the current user".into(),
        parameters: JsonSchema::new(json!({"type": "object"})),
        retry: None,
    },
    |_input: Value, ctx: &AppContext| {
        // Clone data from context before the async block
        let user_id = ctx.user_id.clone();
        async move {
            // Use the cloned data in the async block
            Ok(ToolOutput::new(format!("Data for user: {}", user_id)))
        }
    },
);

// Register with a typed registry
let mut registry: ToolRegistry<AppContext> = ToolRegistry::new();
registry.register(handler);

// Create context and run
let ctx = AppContext {
    user_id: "user123".into(),
    api_key: "secret".into(),
    depth: 0,
};

let params = ChatParams {
    messages: vec![ChatMessage::user("Get my data")],
    tools: Some(registry.definitions()),
    ..Default::default()
};

let result = tool_loop(provider, &registry, params, ToolLoopConfig::default(), &ctx).await?;

Note on lifetimes: The closure passed to tool_fn_with_ctx uses higher-ranked trait bounds (for<'c> Fn(Value, &'c Ctx) -> Fut). This means the future returned by your closure must be 'static — it cannot borrow from the context reference. Clone any data you need from the context before creating the async block.

Structs

Completed

Terminal: the loop completed successfully.

FnToolHandler

A tool handler backed by an async closure.

LoopDetectionConfig

Configuration for detecting repeated tool calls (stuck agents).

NoCtxToolHandler

A tool handler without context, created by super::tool_fn.

StopContext

Context provided to stop condition callbacks.

ToolError

Error returned by tool execution.

ToolLoopConfig

Configuration for tool_loop and tool_loop_stream.

ToolLoopHandle

Caller-driven resumable tool loop.

ToolLoopResult

The result of a completed tool loop.

ToolOutput

Output returned by a tool handler.

ToolRegistry

A registry of tool handlers, indexed by name.

TurnError

Terminal: the loop errored.

Yielded

Handle returned when tools were executed. Borrows the ToolLoopHandle mutably, so the caller cannot call next_turn() again until this is consumed via resume(), continue_loop(), inject_and_continue(), or stop().

Enums

LoopAction

Action to take when a tool call loop is detected.

LoopCommand

Commands sent by the caller to control the resumable loop.

StopDecision

Decision returned by a stop condition callback.

TerminationReason

Why a tool loop terminated.

ToolApproval

Result of approving a tool call before execution.

ToolLoopEvent

Events emitted during tool loop execution for observability.

TurnResult

Result of one turn of the tool loop.

Traits

LoopDepth

Trait for contexts that support automatic depth tracking in nested tool loops.

ToolHandler

A single tool that can be invoked by the LLM.

Functions

tool_fn

Creates a ToolHandler<()> from a closure (no context).

tool_fn_with_ctx

Creates a ToolHandler<Ctx> from a closure that receives context.

tool_loop

Runs the LLM in a tool-calling loop until completion.

tool_loop_channel

Channel-based tool loop with bounded buffer for backpressure.

tool_loop_resumable

Convenience function to create a resumable tool loop.

tool_loop_stream

Streaming variant of tool_loop.

Type Aliases

StopConditionFn

Callback type for stop conditions.

ToolApprovalFn

Callback type for tool call approval.

ToolLoopEventFn

Callback type for tool loop events.