Crate oxi_ai

Expand description

oxi-ai: Unified LLM API for oxi

This crate provides a unified interface for interacting with multiple LLM providers. It handles streaming, tool calling, context management, and cross-provider handoffs.

Re-exports§

pub use multi_provider::MultiProvider;
pub use model_db::get_all_models;
pub use model_db::get_cheapest_models;
pub use model_db::get_model_entry;
pub use model_db::get_provider_models;
pub use model_db::get_reasoning_models;
pub use model_db::get_vision_models;
pub use model_db::model_count;
pub use model_db::search_models;
pub use model_db::ModelEntry;
pub use fallback_chain::FallbackChain;
pub use fallback_chain::FallbackChainError;
pub use partial_response::PartialResponse;
pub use circuit_breaker::CircuitBreakerConfig;
pub use circuit_breaker::CircuitOpenError;
pub use circuit_breaker::ProviderCircuitBreaker;
pub use env_api_keys::find_env_keys;
pub use env_api_keys::get_all_env_keys;
pub use env_api_keys::get_env_api_key;
pub use provider_registry::OAuthTokenInfo;
pub use provider_registry::ProviderAuth;
pub use provider_registry::ProviderAuthRegistry;

Modules§

circuit_breaker: Per-provider circuit breaker implementation.
env_api_keys: Environment variable-based API key resolution (LAST RESORT)
fallback_chain: Fallback chain for ordered model failover.
model_db: Comprehensive model database for oxi-ai
multi_provider: MultiProvider for intelligent routing with fallback support. MultiProvider — intelligent routing with fallback
oauth: OAuth authentication system for oxi-ai
partial_response: Partial response accumulator for stream recovery.
prelude: Standard imports for oxi-ai usage.
provider_pool: Provider pool — rate limiting and concurrency control for provider calls.
provider_registry: Provider authentication registry
register_builtins
secret: Wrapper type to prevent accidental exposure of sensitive data (e.g., API keys).
utils: Utility modules for AI API handling

Structs§

AssistantMessage: Re-export AssistantMessage from messages Assistant message returned by the model.
CompactedContext: Compaction strategies and managers for long conversations. Result of context compaction
CompactionManager: Compaction strategies and managers for long conversations. Context manager that handles compaction automatically
CompatSettings: Core type definitions (tokens, cost, etc.). Compatibility settings for OpenAI-compatible APIs.
Context: Shared conversation context. Conversation context for LLM interactions.
Cost: Core type definitions (tokens, cost, etc.). Cost structure – prices per million tokens.
DefaultRouter: Complexity-based model routing. Default implementation of ComplexityRouter.
ImageContent: Message types for constructing conversations. Image content block (base64-encoded).
LlmCompactor: Compaction strategies and managers for long conversations. LLM-based compactor that uses the model itself to summarize
Model: Core type definitions (tokens, cost, etc.). LLM model definition.
ModelRegistry: Runtime model registry for dynamically registered models.
OpenAiProvider: OpenAI-compatible provider implementation. OpenAI-compatible provider
OpenAiResponsesProvider: OpenAI Responses API provider. OpenAI Responses API provider
ProviderRegistry: Provider trait, streaming options, and provider registry. Runtime registry for providers (custom + built-in resolution).
StreamOptions: Provider trait, streaming options, and provider registry. Options for streaming requests
TextContent: Message types for constructing conversations. Text content block
ThinkingContent: Message types for constructing conversations. Thinking content block (extended thinking / chain-of-thought output).
Tool: Tool definition and argument validation. Tool definition with JSON Schema parameters
ToolCall: Message types for constructing conversations. Tool call content block emitted by the model.
ToolResult: Core type definitions (tokens, cost, etc.). Tool result returned by agent tool execution.
ToolResultMessage: Message types for constructing conversations. Tool result message carrying the output of a tool invocation.
TransformOptions: Message transformation between provider formats. Options that control how messages are transformed between providers.
Usage: Core type definitions (tokens, cost, etc.). Token usage statistics.
UserMessage: Message types for constructing conversations. User message sent to the model.

Enums§

Api: Core type definitions (tokens, cost, etc.). Provider API identifier.
AssistantRole: Message types for constructing conversations. AssistantRole.
CacheRetention: Cache retention control for provider requests. Core type definitions (tokens, cost, etc.). Cache retention preference
CompactionStrategy: Compaction strategies and managers for long conversations. Compaction strategy determining when to compact
Complexity: Core type definitions (tokens, cost, etc.). Task complexity level for routing decisions.
ContentBlock: Message types for constructing conversations. Content block union (untagged for flexibility).
Error: Result type alias for oxi-ai operations. Unified error type for oxi-ai
ImageContentType: Message types for constructing conversations. ImageContentType.
InputModality: Core type definitions (tokens, cost, etc.). Input modalities
MaxTokensField: Core type definitions (tokens, cost, etc.). Which JSON field to use for the maximum output token count.
Message: Message types for constructing conversations. Message union tagged by role.
MessageContent: Message types for constructing conversations. Message content – either a plain text string or a list of structured blocks.
ProviderError: Provider-specific error type for LLM operations. Provider-specific errors
ProviderEvent: Provider trait, streaming options, and provider registry. Streaming events emitted by providers
StopReason: Core type definitions (tokens, cost, etc.). Stop reason – why the model finished generating.
TextContentType: Message types for constructing conversations. TextContentType.
ThinkingContentType: Message types for constructing conversations. ThinkingContentType.
ThinkingFormat: Core type definitions (tokens, cost, etc.). Provider-specific wire format for extended thinking.
ThinkingLevel: Core type definitions (tokens, cost, etc.). Model thinking/reasoning level
ToolCallType: Message types for constructing conversations. ToolCallType.
ToolResultRole: Message types for constructing conversations. ToolResultRole.
ToolValidationError: Tool definition and argument validation. Validation error
UserRole: Message types for constructing conversations. UserRole.

Traits§

Compactor: Compaction strategies and managers for long conversations. Trait for context compaction implementations
ComplexityRouter: Complexity-based model routing. Routes tasks to models based on estimated complexity.
Provider: Provider trait, streaming options, and provider registry. LLM provider trait

Functions§

anthropic_to_google: Message transformation between provider formats. Convert Anthropic-format messages to Google format.
anthropic_to_openai: Message transformation between provider formats. Convert Anthropic-format messages to OpenAI format.
complete: High-level completion and token estimation. High-level complete function that collects all streaming events and returns the final assistant message.
context_usage: Token estimation and context usage helpers. Calculate context length usage percentage.
create_builtin_provider: Built-in provider helpers (re-exported from providers). Create a built-in provider by name.
custom_provider_names: Provider trait, streaming options, and provider registry. Return the set of currently registered custom provider names (global registry).
estimate: Token estimation and context usage helpers. Estimate token count using a hybrid algorithm that combines character-based and word-based heuristics.
estimate_tokens: High-level completion and token estimation. Estimate token count using a hybrid algorithm that combines character-based and word-based heuristics.
estimate_words: Token estimation and context usage helpers. Estimate tokens based on word count.
fetch_models_async: Model fetching utilities (async and blocking). Fetch the model list from an OpenAI-compatible /v1/models endpoint asynchronously.
fetch_models_blocking: Model fetching utilities (async and blocking). Fetch the model list from an OpenAI-compatible /v1/models endpoint using the shared reqwest::blocking client.
get_all_provider_names: Built-in provider helpers (re-exported from providers). Get all provider names (primary names only).
get_builtin_provider: Built-in provider helpers (re-exported from providers). Look up a built-in provider by name or alias.
get_model: Runtime model registry for dynamically registered models.
get_models: Runtime model registry for dynamically registered models.
get_provider: Provider trait, streaming options, and provider registry. Get a provider by name
get_provider_env_key: Built-in provider helpers (re-exported from providers). Get the environment variable name for a provider.
get_provider_env_keys: Built-in provider helpers (re-exported from providers). Get all environment variable names for a provider (primary + extras).
get_providers: Runtime model registry for dynamically registered models.
google_to_openai: Message transformation between provider formats. Convert Google-format messages to OpenAI format.
is_builtin_provider: Built-in provider helpers (re-exported from providers). Check if a provider name or alias is a known built-in.
lookup_model: Runtime model registry for dynamically registered models.
normalize_tool_call_id: Message transformation between provider formats. Normalize a tool call ID for cross-provider compatibility.
openai_to_anthropic: Message transformation between provider formats. Convert OpenAI-format messages to Anthropic format.
progress_callback: Tool definition and argument validation. Create a progress callback from a closure
register_model: Runtime model registry for dynamically registered models.
register_provider: Provider trait, streaming options, and provider registry. Register a custom provider at runtime (global registry).
transform_for_provider: Message types for constructing conversations. Transform messages for cross-provider compatibility.
transform_messages: Message transformation between provider formats. Transform a slice of Messages from one provider API to another.
transform_messages_for_model: Message transformation between provider formats. Transform messages for a target model, handling:
unregister_model: Runtime model registry for dynamically registered models.
unregister_provider: Provider trait, streaming options, and provider registry. Unregister a previously registered custom provider (global registry).
validate_args: Tool definition and argument validation. Validate tool arguments against a JSON Schema

Type Aliases§

ProgressCallback: Tool definition and argument validation. Callback type for progress updates
Result: Result type alias for oxi-ai operations. Result type alias

Crate oxi_ai

Crate oxi_ai Copy item path

Re-exports§

Modules§

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate oxi_ai