llm-stack 0.7.0

Core traits, types, and tools for the llm-stack SDK
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112

//! # llm-stack
//!
//! Provider-agnostic types and traits for interacting with large language models.
//!
//! This crate defines the shared vocabulary that every LLM provider implementation
//! speaks: messages, responses, tool calls, streaming events, usage tracking, and
//! errors. It intentionally contains **zero** provider-specific code — concrete
//! providers live in sibling crates and implement [`Provider`] (or its
//! object-safe counterpart [`DynProvider`]).
//!
//! # Provider Crates
//!
//! Official provider implementations:
//!
//! | Crate | Provider | Features |
//! |-------|----------|----------|
//! | [`llm-stack-anthropic`](https://docs.rs/llm-stack-anthropic) | Claude (Anthropic) | Streaming, tools, vision, caching |
//! | [`llm-stack-openai`](https://docs.rs/llm-stack-openai) | GPT (`OpenAI`) | Streaming, tools, structured output |
//! | [`llm-stack-ollama`](https://docs.rs/llm-stack-ollama) | Ollama (local) | Streaming, tools |
//!
//! # Architecture
//!
//! ```text
//!  ┌─────────────────────┐ ┌───────────────────┐ ┌───────────────────┐
//!  │ llm-stack-anthropic │ │  llm-stack-openai  │ │  llm-stack-ollama │
//!  └──────────┬──────────┘ └─────────┬─────────┘ └─────────┬─────────┘
//!             │                      │                     │
//!             └───────────┬──────────┴──────────┬──────────┘
//!                         │                     │
//!                         ▼                     ▼
//!              ┌─────────────────────────────────────┐
//!              │             llm-stack               │  ← you are here
//!              │  (Provider trait, ChatParams, etc.) │
//!              └─────────────────────────────────────┘
//! ```
//!
//! # Quick start
//!
//! ```rust,no_run
//! use llm_stack::{ChatMessage, ChatParams, Provider};
//!
//! # async fn example(provider: impl Provider) -> Result<(), llm_stack::LlmError> {
//! let params = ChatParams {
//!     messages: vec![ChatMessage::user("Explain ownership in Rust")],
//!     max_tokens: Some(1024),
//!     ..Default::default()
//! };
//!
//! let response = provider.generate(&params).await?;
//! # Ok(())
//! # }
//! ```
//!
//! # Modules
//!
//! | Module | Purpose |
//! |--------|---------|
//! | [`chat`] | Messages, content blocks, tool calls, and responses |
//! | [`context`] | Token-budgeted conversation history management |
//! | [`error`] | Unified [`LlmError`] across all providers |
//! | [`intercept`] | Unified interceptor system for LLM calls and tool executions |
//! | [`provider`] | The [`Provider`] trait and request parameters |
//! | [`stream`] | Server-sent event types and the [`ChatStream`] alias |
//! | [`structured`] | Typed LLM responses with schema validation (feature-gated) |
//! | [`tool`] | Tool execution engine with registry and approval hooks |
//! | [`registry`] | Dynamic provider instantiation from configuration |
//! | [`usage`] | Token counts and cost tracking |

#![warn(missing_docs)]

pub mod chat;
pub mod context;
pub mod error;
pub mod intercept;
pub mod provider;
pub mod registry;
pub mod stream;
pub mod structured;
pub mod tool;
pub mod usage;

pub mod mcp;

#[cfg(any(test, feature = "test-utils"))]
pub mod mock;

#[cfg(any(test, feature = "test-utils"))]
pub mod test_helpers;

// ── Core re-exports ────────────────────────────────────────────────
//
// Only the types that appear in nearly every program are re-exported
// at the crate root. Everything else lives in its submodule:
//
//   llm_stack::tool::*        — tool execution, loop handles, events
//   llm_stack::provider::*    — capabilities, metadata, retry config
//   llm_stack::chat::*        — StopReason, ImageSource, ChatRole
//   llm_stack::stream::*      — ChatStream, StreamEvent
//   llm_stack::usage::*       — Cost, ModelPricing, UsageTracker
//   llm_stack::context::*     — ContextWindow, token estimation
//   llm_stack::registry::*    — ProviderRegistry, ProviderFactory
//   llm_stack::mcp::*         — McpService, McpRegistryExt
//   llm_stack::structured::*  — generate_object, stream_object_async
//   llm_stack::mock::*        — MockProvider (test-utils feature)

pub use chat::{ChatMessage, ChatResponse, ContentBlock, ToolCall, ToolResult};
pub use error::LlmError;
pub use provider::{ChatParams, DynProvider, JsonSchema, Provider, ToolChoice, ToolDefinition};
pub use registry::ProviderRegistry;
pub use stream::{ChatStream, StreamEvent};
pub use tool::{ToolHandler, ToolLoopConfig, ToolRegistry};
pub use usage::Usage;