Crate tower_llm 

Tower LLM

A powerful Tower-based framework for building multi-agent workflows with OpenAI LLMs. This SDK provides composable, type-safe components using Tower’s service architecture for maximum flexibility and performance.

Core Concepts

• Agent: A Tower service that processes chat requests through an LLM with tools
• Tools: Type-safe functions that agents can call, with automatic schema generation
• Layers: Tower middleware for cross-cutting concerns (retry, timeout, observability)
• Static DI: All dependencies are injected at construction time - no runtime lookups

Getting Started

Set your OpenAI API key in the OPENAI_API_KEY environment variable.

use tower_llm::{Agent, tool_typed, policies, CompositePolicy, run_user};
use async_openai::{config::OpenAIConfig, Client};
use schemars::JsonSchema;
use serde::Deserialize;
use std::sync::Arc;
use tower::{Service, ServiceExt};

#[derive(Debug, Deserialize, JsonSchema)]
struct AddArgs {
    a: f64,
    b: f64,
}

// Create OpenAI client
let client = Arc::new(Client::<OpenAIConfig>::new());

// Define a tool
let calculator = tool_typed(
    "add",
    "Add two numbers",
    |args: AddArgs| async move {
        Ok(serde_json::json!({ "sum": args.a + args.b }))
    },
);

// Build an agent with per-agent instructions
let agent = Agent::builder(client)
    .model("gpt-4o")
    .instructions("You are a helpful math assistant")
    .tool(calculator)
    .policy(CompositePolicy::new(vec![policies::until_no_tool_calls()]))
    .build();

// Use the agent with Tower's Service trait
let mut agent = agent;
// Send only a user message; the agent injects its instructions as a system message
let response = run_user(&mut agent, "What is 2 + 2?").await?;

println!("Agent: {:?}", response);

Re-exports

pub use error::AgentsError;

pub use error::Result;

pub use memory::Session;

pub use result::RunResult;

pub use result::RunResultWithContext;

pub use result::StreamingRunResult;

pub use sqlite_session::SqliteSession;

Modules

approvals

Guardrails and approvals

auto_compaction

Auto-compaction layer for managing conversation context length

budgets

Budgets: tokens, time, and tool invocations

codec

Bijection codec between RunItem and raw OpenAI chat messages

concurrency

Parallel tool execution and concurrency controls

error

Error Handling for the Agents SDK

groups

Multi-agent orchestration and handoffs

items

Items (orientation)

memory

Session trait for compatibility with SqliteSession

observability

Observability: tracing and metrics

policies

Built-in policies

provider

Model provider abstractions (streaming and non-streaming)

recording

Recording and replay I/O surfaces

resilience

Resilience layers: timeout, retry, rate-limit, circuit-breaker

result

Result types for agent execution

sessions

Sessions and persistence (memory/replay) for the next stack

sqlite_session

SQLite-Based Session Storage

streaming

Streaming step/agent variants

tap

Generic, zero-interference tap layer for observing requests, responses, and errors.

validation

Conversation validation utilities (pure, test-focused).

Structs

Agent

Thin facade to build an agent stack from tools, model, and policy.

AgentBuilder

AgentLoop

AgentLoopLayer

Layer to wrap a step service with an agent loop controlled by a policy.

AgentRun

Final run summary from the agent loop.

Client

Client is a container for config, backoff and http_client used to make API calls.

CompositePolicy

Composite policy: stop when any sub-policy returns a stop reason.

CreateChatCompletionRequest

CreateChatCompletionRequestArgs

Builder for CreateChatCompletionRequest.

LoopState

Loop state visible to policies.

OpenAIConfig

Configuration for OpenAI API

Policy

Chainable policy builder.

PolicyFn

Function-backed policy for ergonomic composition.

Step

One-step agent service parameterized by a routed tool service S.

StepAux

Auxiliary accounting captured per step.

StepLayer

Layer that lifts a routed tool service S into a Step<S> service.

ToolDef

Definition of a tool: function spec (for OpenAI) + service implementation.

ToolInvocation

Uniform tool invocation passed to routed tool services.

ToolOutput

Uniform tool output produced by tool services.

ToolRouter

Simple router service over tools using a name → index table.

Enums

AgentStopReason

Stop reasons reported by the agent loop.

ChatCompletionRequestMessage

ReasoningEffort

StepOutcome

Outcome of a single agent step.

ToolJoinPolicy

Join policy for parallel tool execution

Traits

AgentPolicy

Policy interface controlling loop termination.

LLMInstructionProvider

Layer

Decorates a Service, transforming either the request or the response.

Service

An asynchronous function from a Request to a Response.

ServiceExt

An extension trait for Services that provides a variety of convenient adapters

Functions

run

Convenience: run a prompt through an agent service.

run_user

Convenience: run a user message through an agent service. System instructions come from the agent.

simple_chat_request

Build a simple chat request from plain strings.

simple_user_request

Build a simple chat request with only a user message.

tool_typed

DX sugar: create a tool from a typed handler.

Type Aliases

AgentSvc

Boxed agent service type for ergonomic returns.

ToolSvc

Boxed tool service type alias.