Crate open_agent 

Open Agent SDK - Rust Implementation

A production-ready, streaming-first Rust SDK for building AI agents with local OpenAI-compatible servers.

Overview

This SDK provides a clean, ergonomic API for working with local LLM servers such as:

• LM Studio
• Ollama
• llama.cpp
• vLLM

Key Features

• Zero API Costs: Run models on your own hardware
• Privacy-First: All data stays local on your machine
• High Performance: Native async/await with Tokio runtime
• Streaming Responses: Real-time token-by-token streaming
• Tool Calling: Define and execute tools with automatic schema generation
• Lifecycle Hooks: Intercept and control execution at key points
• Interrupts: Gracefully cancel long-running operations
• Context Management: Manual token estimation and history truncation
• Retry Logic: Exponential backoff with jitter for reliability

Two Interaction Modes

1. Simple Query Function (query())

For single-turn interactions without conversation state:

use open_agent::{query, AgentOptions, ContentBlock};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Configure the agent with required settings
    let options = AgentOptions::builder()
        .system_prompt("You are a helpful assistant")
        .model("qwen2.5-32b-instruct")
        .base_url("http://localhost:1234/v1")
        .build()?;

    // Send a single query and stream the response
    let mut stream = query("What's the capital of France?", &options).await?;

    // Process each content block as it arrives
    while let Some(block) = stream.next().await {
        match block? {
            ContentBlock::Text(text_block) => {
                print!("{}", text_block.text);
            }
            ContentBlock::ToolUse(tool_block) => {
                println!("Tool called: {}", tool_block.name());
            }
            ContentBlock::ToolResult(_) => {
                // Tool results can be ignored in simple queries
            }
            ContentBlock::Image(_) => {
                // Images not expected in this example
            }
        }
    }

    Ok(())
}

2. Client Object (Client)

For multi-turn conversations with persistent state:

use open_agent::{Client, AgentOptions, ContentBlock};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let options = AgentOptions::builder()
        .system_prompt("You are a helpful assistant")
        .model("qwen2.5-32b-instruct")
        .base_url("http://localhost:1234/v1")
        .build()?;

    // Create a stateful client that maintains conversation history
    let mut client = Client::new(options)?;

    // First turn
    client.send("What's 2+2?").await?;
    while let Some(block) = client.receive().await? {
        match block {
            ContentBlock::Text(text) => print!("{}", text.text),
            ContentBlock::ToolUse(_) | ContentBlock::ToolResult(_) | ContentBlock::Image(_) => {}
        }
    }

    // Second turn - client remembers previous context
    client.send("What about if we multiply that by 3?").await?;
    while let Some(block) = client.receive().await? {
        match block {
            ContentBlock::Text(text) => print!("{}", text.text),
            ContentBlock::ToolUse(_) | ContentBlock::ToolResult(_) | ContentBlock::Image(_) => {}
        }
    }

    Ok(())
}

Architecture

The SDK is organized into several modules, each with a specific responsibility:

• client: Core streaming query engine and multi-turn client
• types: Data structures for messages, content blocks, and configuration
• tools: Tool definition system with automatic JSON schema generation
• hooks: Lifecycle event system for intercepting execution
• config: Provider-specific configuration helpers
• error: Comprehensive error types and conversions
• context: Token estimation and message truncation utilities
• retry: Exponential backoff retry logic with jitter
• utils: Internal utilities for SSE parsing and tool aggregation

Modules

prelude

Convenience module containing the most commonly used types and functions. Import with use open_agent::prelude::*; to get everything you need for typical usage.

retry

Retry utilities with exponential backoff and jitter. Made public as a module so users can access retry configuration and functions for their own operations that need retry logic. Retry utilities with exponential backoff

Structs

AgentOptions

Configuration options for an AI agent instance.

AgentOptionsBuilder

Builder for constructing AgentOptions with validation.

BaseUrl

Validated base URL with compile-time type safety.

Client

Stateful client for multi-turn conversations with automatic history management.

HookDecision

Decision returned by a hook handler to control agent execution flow.

Hooks

Container for registering and managing lifecycle hooks.

ImageBlock

Image content block for vision-capable models.

Message

A complete message in a conversation.

ModelName

Validated model name with compile-time type safety.

PostToolUseEvent

Event fired after a tool completes execution, enabling audit, filtering, or validation.

PreToolUseEvent

Event fired before a tool is executed, enabling validation, modification, or blocking.

Temperature

Validated temperature value with compile-time type safety.

TextBlock

Simple text content in a message.

Tool

Tool definition for OpenAI-compatible function calling.

ToolBuilder

Builder for creating tools with a fluent API.

ToolResultBlock

Tool execution result sent back to the model.

ToolUseBlock

Tool use request from the AI model.

UserPromptSubmitEvent

Event fired before processing user input, enabling content moderation and prompt enhancement.

Enums

ContentBlock

Multi-modal content blocks that can appear in messages.

Error

Comprehensive error type covering all failure modes in the SDK.

ImageDetail

Image detail level for vision API calls.

MessageRole

Identifies the sender/role of a message in the conversation.

OpenAIContent

OpenAI API message format for serialization.

OpenAIContentPart

A single content part in an OpenAI message.

Provider

Enum representing supported local LLM server providers.

Constants

HOOK_POST_TOOL_USE

String constant for the PostToolUse hook event name.

HOOK_PRE_TOOL_USE

String constant for the PreToolUse hook event name.

HOOK_USER_PROMPT_SUBMIT

String constant for the UserPromptSubmit hook event name.

Functions

estimate_tokens

Estimate token count for message list

get_base_url

Get the base URL for API requests with environment variable support.

get_model

Get the model name with optional environment variable override.

is_approaching_limit

Check if history is approaching a token limit

query

Simple query function for single-turn interactions without conversation history.

tool

Create a tool using the builder pattern (convenience function).

truncate_messages

Truncate message history, keeping recent messages

Type Aliases

Result

Type alias for Result<T, Error> used throughout the SDK.