Crate llm_stack 

llm-stack

Provider-agnostic types and traits for interacting with large language models.

This crate defines the shared vocabulary that every LLM provider implementation speaks: messages, responses, tool calls, streaming events, usage tracking, and errors. It intentionally contains zero provider-specific code — concrete providers live in sibling crates and implement Provider (or its object-safe counterpart DynProvider).

Provider Crates

Official provider implementations:

Crate	Provider	Features
llm-stack-anthropic	Claude (Anthropic)	Streaming, tools, vision, caching
llm-stack-openai	GPT (OpenAI)	Streaming, tools, structured output
llm-stack-ollama	Ollama (local)	Streaming, tools

Architecture

 ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
 │ llm-stack-anthropic│ │  llm-stack-openai │ │  llm-stack-ollama │
 └─────────┬─────────┘ └─────────┬─────────┘ └─────────┬─────────┘
           │                     │                     │
           └──────────┬──────────┴──────────┬──────────┘
                      │                     │
                      ▼                     ▼
             ┌─────────────────────────────────────┐
             │             llm-stack               │  ← you are here
             │  (Provider trait, ChatParams, etc.) │
             └─────────────────────────────────────┘

Quick start

use llm_stack::{ChatMessage, ChatParams, Provider};

let params = ChatParams {
    messages: vec![ChatMessage::user("Explain ownership in Rust")],
    max_tokens: Some(1024),
    ..Default::default()
};

let response = provider.generate(&params).await?;

Modules

Module	Purpose
chat	Messages, content blocks, tool calls, and responses
context	Token-budgeted conversation history management
error	Unified LlmError across all providers
intercept	Unified interceptor system for LLM calls and tool executions
provider	The Provider trait and request parameters
stream	Server-sent event types and the ChatStream alias
structured	Typed LLM responses with schema validation (feature-gated)
tool	Tool execution engine with registry and approval hooks
registry	Dynamic provider instantiation from configuration
usage	Token counts and cost tracking

Re-exports

pub use chat::ChatMessage;

pub use chat::ChatResponse;

pub use chat::ChatRole;

pub use chat::ContentBlock;

pub use chat::ImageSource;

pub use chat::StopReason;

pub use chat::ToolCall;

pub use chat::ToolResult;

pub use error::LlmError;

pub use provider::Capability;

pub use provider::ChatParams;

pub use provider::DynProvider;

pub use provider::JsonSchema;

pub use provider::Provider;

pub use provider::ProviderMetadata;

pub use provider::RetryPredicate;

pub use provider::ToolChoice;

pub use provider::ToolDefinition;

pub use provider::ToolRetryConfig;

pub use stream::ChatStream;

pub use stream::StreamEvent;

pub use tool::Completed;

pub use tool::FnToolHandler;

pub use tool::LoopAction;

pub use tool::LoopCommand;

pub use tool::LoopContext;

pub use tool::LoopDepth;

pub use tool::LoopDetectionConfig;

pub use tool::LoopEvent;

pub use tool::LoopStream;

pub use tool::NoCtxToolHandler;

pub use tool::OwnedToolLoopHandle;

pub use tool::OwnedTurnResult;

pub use tool::OwnedYielded;

pub use tool::StopConditionFn;

pub use tool::StopContext;

pub use tool::StopDecision;

pub use tool::TerminationReason;

pub use tool::ToolApproval;

pub use tool::ToolError;

pub use tool::ToolHandler;

pub use tool::ToolLoopConfig;

pub use tool::ToolLoopHandle;

pub use tool::ToolLoopResult;

pub use tool::ToolOutput;

pub use tool::ToolRegistry;

pub use tool::TurnError;

pub use tool::TurnResult;

pub use tool::Yielded;

pub use tool::tool_fn;

pub use tool::tool_fn_with_ctx;

pub use usage::Cost;

pub use usage::ModelPricing;

pub use usage::Usage;

pub use usage::UsageTracker;

pub use context::ContextWindow;

pub use context::estimate_message_tokens;

pub use context::estimate_tokens;

pub use registry::ProviderConfig;

pub use registry::ProviderFactory;

pub use registry::ProviderRegistry;

pub use mcp::McpError;

pub use mcp::McpRegistryExt;

pub use mcp::McpService;

pub use structured::GenerateObjectConfig;

pub use structured::GenerateObjectResult;

pub use structured::PartialObject;

pub use structured::collect_stream_object;

pub use structured::generate_object;

pub use structured::stream_object_async;

pub use mock::MockError;

pub use mock::MockProvider;

Modules

chat

Conversation primitives: messages, content blocks, and responses.

context

Context window management with token budgeting.

error

Unified error type for all LLM operations.

intercept

Unified interceptor system for LLM calls and tool executions.

mcp

MCP (Model Context Protocol) integration for ToolRegistry.

mock

Mock provider for testing.

provider

Provider trait and request types.

registry

Dynamic provider registry for configuration-driven provider instantiation.

stream

Streaming response types.

structured

Structured output — typed LLM responses with schema validation.

test_helpers

Pre-built helpers for testing code that uses llm-core types.

tool

Tool execution engine.

usage

Token usage and cost tracking.