Crate open_agent

Source

Expand description

§Open Agent SDK - Rust Implementation

A production-ready, streaming-first Rust SDK for building AI agents with local OpenAI-compatible servers.

§Overview

This SDK provides a clean, ergonomic API for working with local LLM servers such as:

LM Studio
Ollama
llama.cpp
vLLM

§Key Features

Zero API Costs: Run models on your own hardware
Privacy-First: All data stays local on your machine
High Performance: Native async/await with Tokio runtime
Streaming Responses: Real-time token-by-token streaming
Finish Reasons: Every stream reports why generation stopped
Tool Calling: Define and execute tools with automatic schema generation
Lifecycle Hooks: Intercept and control execution at key points
Interrupts: Gracefully cancel long-running operations
Context Management: Manual token estimation and history truncation
Retry Logic: Exponential backoff with jitter for reliability

§Two Interaction Modes

§1. Simple Query Function (`query()`)

For single-turn interactions without conversation state:

use open_agent::{query, AgentOptions, ContentBlock, FinishReason, StreamEvent};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Configure the agent with required settings
    let options = AgentOptions::builder()
        .system_prompt("You are a helpful assistant")
        .model("qwen2.5-32b-instruct")
        .base_url("http://localhost:1234/v1")
        .build()?;

    // Send a single query and stream the response
    let mut stream = query("What's the capital of France?", &options).await?;

    // Process each event as it arrives; the stream always ends with one Finish
    while let Some(event) = stream.next().await {
        match event? {
            StreamEvent::Block(ContentBlock::Text(text_block)) => {
                print!("{}", text_block.text);
            }
            StreamEvent::Block(ContentBlock::ToolUse(tool_block)) => {
                println!("Tool called: {}", tool_block.name());
            }
            StreamEvent::Finish(FinishReason::Length) => {
                eprintln!("response was truncated at the token cap");
            }
            _ => {}
        }
    }

    Ok(())
}

§2. Client Object (`Client`)

For multi-turn conversations with persistent state:

use open_agent::{Client, AgentOptions, ContentBlock};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let options = AgentOptions::builder()
        .system_prompt("You are a helpful assistant")
        .model("qwen2.5-32b-instruct")
        .base_url("http://localhost:1234/v1")
        .build()?;

    // Create a stateful client that maintains conversation history
    let mut client = Client::new(options)?;

    // First turn
    client.send("What's 2+2?").await?;
    while let Some(block) = client.receive().await? {
        match block {
            ContentBlock::Text(text) => print!("{}", text.text),
            ContentBlock::ToolUse(_) | ContentBlock::ToolResult(_) | ContentBlock::Image(_) => {}
        }
    }

    // Second turn - client remembers previous context
    client.send("What about if we multiply that by 3?").await?;
    while let Some(block) = client.receive().await? {
        match block {
            ContentBlock::Text(text) => print!("{}", text.text),
            ContentBlock::ToolUse(_) | ContentBlock::ToolResult(_) | ContentBlock::Image(_) => {}
        }
    }

    Ok(())
}

§Architecture

The SDK is organized into several modules, each with a specific responsibility:

client: Core streaming query engine and multi-turn client
types: Data structures for messages, content blocks, and configuration
tools: Tool definition system with automatic JSON schema generation
hooks: Lifecycle event system for intercepting execution
config: Provider-specific configuration helpers
error: Comprehensive error types and conversions
context: Token estimation and message truncation utilities
retry: Exponential backoff retry logic with jitter
utils: Internal utilities for SSE parsing and tool aggregation

Modules§

prelude: Convenience module containing the most commonly used types and functions. Import with use open_agent::prelude::*; to get everything you need for typical usage.
retry: Retry utilities with exponential backoff and jitter. Made public as a module so users can access retry configuration and functions for their own operations that need retry logic. Retry utilities with exponential backoff

Structs§

AgentOptions: Configuration options for an AI agent instance.
AgentOptionsBuilder: Builder for constructing AgentOptions with validation.
AnthropicErrorBody: The body of a mid-stream error event.
AnthropicMessage: One conversation turn.
AnthropicMessageDelta: Top-level message changes, carrying the reason generation stopped.
AnthropicRequest: Request payload for POST {base_url}/messages.
BaseUrl: Validated base URL with compile-time type safety.
Client: Stateful client for multi-turn conversations with automatic history management.
HookDecision: Decision returned by a hook handler to control agent execution flow.
Hooks: Container for registering and managing lifecycle hooks.
ImageBlock: Image content block for vision-capable models.
Message: A complete message in a conversation.
ModelName: Validated model name with compile-time type safety.
OpenAIFunction: OpenAI function call details.
OpenAIMessage
OpenAIRequest: Complete request payload for OpenAI chat completions API.
OpenAIToolCall: OpenAI tool call representation in API messages.
PostToolUseEvent: Event fired after a tool completes execution, enabling audit, filtering, or validation.
PreToolUseEvent: Event fired before a tool is executed, enabling validation, modification, or blocking.
Temperature: Validated temperature value with compile-time type safety.
TextBlock: Simple text content in a message.
Tool: Tool definition for OpenAI-compatible function calling.
ToolBuilder: Builder for creating tools with a fluent API.
ToolResultBlock: Tool execution result sent back to the model.
ToolUseBlock: Tool use request from the AI model.
UserPromptSubmitEvent: Event fired before processing user input, enabling content moderation and prompt enhancement.

Enums§

AnthropicBlockStart: The declaration that opens a content block.
AnthropicDelta: One fragment appended to an open block.
AnthropicEvent: One event from an Anthropic streaming response.
ApiProtocol: The wire protocol an endpoint exposes.
ContentBlock: Multi-modal content blocks that can appear in messages.
Error: Comprehensive error type covering all failure modes in the SDK.
FinishReason: Why the model stopped generating.
ImageDetail: Image detail level for vision API calls.
MessageRole: Identifies the sender/role of a message in the conversation.
OpenAIContent: OpenAI API message format for serialization.
OpenAIContentPart: A single content part in an OpenAI message.
Provider: Enum representing supported local LLM server providers.
StreamEvent: One item in the stream returned by query().

Constants§

HOOK_POST_TOOL_USE: String constant for the PostToolUse hook event name.
HOOK_PRE_TOOL_USE: String constant for the PreToolUse hook event name.
HOOK_USER_PROMPT_SUBMIT: String constant for the UserPromptSubmit hook event name.

Functions§

anthropic_finish_reason: Maps an Anthropic stop_reason onto the SDK’s protocol-neutral FinishReason.
estimate_tokens: Estimate token count for message list
get_base_url: Get the base URL for API requests with environment variable support.
get_model: Get the model name with optional environment variable override.
is_approaching_limit: Check if history is approaching a token limit
query: Simple query function for single-turn interactions without conversation history.
tool: Create a tool using the builder pattern (convenience function).
truncate_messages: Truncate message history, keeping recent messages

Type Aliases§

EventStream: A pinned, boxed stream of events from the model.
Result: Type alias for Result<T, Error> used throughout the SDK.

Crate open_agent

Crate open_agent Copy item path

§Open Agent SDK - Rust Implementation

§Overview

§Key Features

§Two Interaction Modes

§1. Simple Query Function (query())

§2. Client Object (Client)

§Architecture

Modules§

Structs§

Enums§

Constants§

Functions§

Type Aliases§

Crate open_agent

§1. Simple Query Function (`query()`)

§2. Client Object (`Client`)