Module llm 

Async LLM client with OpenAI and Anthropic backends.

The LLMClient trait provides a unified interface for chat completion and SSE streaming. Implementations: OpenAIClient, AnthropicClient.

The json_extract module handles extracting structured JSON from raw LLM text output (code fences, raw JSON, etc.).

Requires the client-async feature.

Re-exports

pub use cache::CacheClient;

pub use client::AnthropicClient;

pub use client::LLMClient;

pub use client::OpenAIClient;

pub use history::ConversationHistory;

pub use json_extract::JsonExtractor;

pub use json_extract::extract_json;

pub use json_extract::parse_json;

pub use middleware::LLMClientMiddleware;

pub use middleware::MiddlewareClient;

pub use middleware::NoopMiddleware;

pub use prompt::render_prompt;

pub use retry::RetryClient;

pub use retry::RetryConfig;

pub use template::PromptTemplate;

pub use tool::ToolCall;

pub use tool::ToolDefinition;

pub use tool::ToolResult;

pub use types::LLMStream;

pub use types::ChatMessage;

pub use types::ContentPart;

pub use types::LLMRequest;

pub use types::LLMRequestBuilder;

pub use types::LLMResponse;

pub use types::MessageRole;

pub use types::ModelConfig;

pub use types::ResponseFormat;

pub use types::StreamEvent;

pub use types::TokenUsage;

Modules

cache

Response cache wrapper for LLMClient over a KvStore.

client

Async LLM client implementations (OpenAI, Anthropic).

history

Conversation history with token-budget-aware truncation. Conversation history management with token-budget-aware truncation.

json_extract

JSON extraction from raw LLM text output. Extract structured JSON from raw LLM text output.

middleware

Middleware hooks for LLMClient request/response lifecycle. Middleware hooks for LLMClient request/response lifecycle.

prompt

Prompt template rendering.

retry

Exponential backoff retry wrapper for LLMClient. Exponential backoff retry wrapper for LLMClient.

template

Prompt templates with variable substitution and few-shot examples. Prompt templates with variable substitution and few-shot examples.

tool

Tool/function calling types. Tool/function calling types for LLM APIs.

types

Core LLM request/response types. Core types for the LLM client module.