Crate oxi_ai

Expand description

oxi-ai: Unified LLM API for oxi

This crate provides a unified interface for interacting with multiple LLM providers. It handles streaming, tool calling, context management, and cross-provider handoffs.

Re-exports§

pub use providers::CacheRetention;
pub use compaction::generate_branch_summary;
pub use compaction::CompactedContext;
pub use compaction::CompactionManager;
pub use compaction::CompactionStrategy;
pub use compaction::Compactor;
pub use compaction::ContextTransformer;
pub use compaction::LlmCompactor;
pub use compaction::NoopContextTransformer;
pub use model_db::ModelEntry;
pub use model_db::get_all_models;
pub use model_db::get_cheapest_models;
pub use model_db::get_model_entry;
pub use model_db::get_provider_models;
pub use model_db::get_reasoning_models;
pub use model_db::get_vision_models;
pub use model_db::model_count;
pub use model_db::search_models;
pub use roles::ModelRole;
pub use roles::RoleColor;
pub use roles::RoleInfo;
pub use roles::RoleRegistry;
pub use roles::builtin_role_info;
pub use roles::builtin_visible_ids;
pub use role_switcher::DEFAULT_LONG_CONTEXT_THRESHOLD;
pub use role_switcher::RoleSignals;
pub use role_switcher::decide_role;
pub use role_switcher::resolve_role_to_model;
pub use role_switcher::role_for_tool;
pub use roles::live_role_registry;
pub use roles::set_live_role_registry;
pub use role_routing::RoleRoutingProvider;
pub use partial_response::PartialResponse;
pub use env_api_keys::find_env_keys;
pub use env_api_keys::get_all_env_keys;
pub use env_api_keys::get_env_api_key;
pub use provider_registry::OAuthTokenInfo;
pub use provider_registry::ProviderAuth;
pub use provider_registry::ProviderAuthRegistry;
pub use types::*;

Modules§

catalog: Stable since 0.63.0
Built-in provider catalog loader.
circuit_breaker: Circuit-breaker behavior trait + SDK reference implementation.
compaction: Context compaction for long conversations
compaction_seam: Compaction trait seams — ported from grok-build xai-grok-compaction/src/{item,sampler,token}.rs (Apache-2.0).
dialect: Owned (in-band) tool-calling dialects — tool calls as text for models without native tool support (omp pi-ai/dialect port). Owned (in-band) tool-calling dialects.
env_api_keys: Environment variable-based API key resolution (LAST RESORT)
model_db: Comprehensive model database for oxi-ai
oauth: OAuth authentication system for oxi-ai
partial_response: Partial response accumulator for stream recovery.
prelude: Standard imports for oxi-ai usage.
product_env: Product home-directory resolution (OXI_HOME / ~/.oxi). Product home-directory resolution for oxi-ai.
provider_registry: Provider authentication registry
register_builtins
role_routing: Provider wrapper that routes each request to the role-selected model. Role-routing provider — plugs role switching into the main agent loop.
role_switcher: Role-switching decision engine (signals -> role -> model). Role-switching decision engine — the “switching layer” on top of the crate::roles registry.
roles: Named model roles with pi/<role> alias resolution. Model roles — named model assignments ported from omp’s model-roles.ts.
router: Model auto-routing for oxi.
secret: Wrapper type to prevent accidental exposure of sensitive data (e.g., API keys).
types: Core domain types for oxi-ai
utils: Utility modules for AI API handling

Structs§

AnthropicProvider: Anthropic provider implementation. Anthropic provider
AssistantMessage: Re-export AssistantMessage from messages Assistant message returned by the model.
AzureProvider: Azure OpenAI provider implementation. Azure OpenAI provider
BedrockProvider: AWS Bedrock provider implementation. Amazon Bedrock provider
Context: Stable since 0.63.0
Shared conversation context. Conversation context for LLM interactions.
GeminiCliProvider: Google Gemini CLI transport — typed unsupported-provider error path. Api::GoogleGeminiCli dispatches here; stream() returns ProviderError::NotImplemented because no dedicated CLI transport exists in-tree (upstream collapses google-gemini-cli → google-generative-ai). Thin transport for the google-gemini-cli dialect.
GoogleProvider: Google Generative AI (Gemini) provider implementation. Google Generative AI provider
HttpErrorDetail: Structured HTTP error detail (status/body/provider/request-id) carried by ProviderError::HttpError. Re-exported so downstream crates (oxi-agent, oxi-cli) can construct/inspect structured errors. Structured HTTP error detail.
ImageContent: Stable since 0.63.0
Message types for constructing conversations. Image content block (base64-encoded).
ModelRegistry: Runtime model registry for dynamically registered models.
OllamaProvider: Ollama (local NDJSON server) provider implementation. Ollama provider for local LLM inference.
OpenAiProvider: OpenAI-compatible provider implementation. OpenAI-compatible provider
OpenAiResponsesProvider: OpenAI Responses API provider. OpenAI Responses API provider
ProviderOptions: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Per-provider options for fine-grained control.
ProviderRegistry: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Runtime registry for providers (custom + built-in resolution).
StreamOptions: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Options for streaming requests
TextContent: Stable since 0.63.0
Message types for constructing conversations. Text content block
ThinkingContent: Stable since 0.63.0
Message types for constructing conversations. Thinking content block (extended thinking / chain-of-thought output).
Tool: Stable since 0.63.0
Tool definition and argument validation. Tool definition with JSON Schema parameters
ToolCall: Stable since 0.63.0
Message types for constructing conversations. Tool call content block emitted by the model.
ToolResultMessage: Stable since 0.63.0
Message types for constructing conversations. Tool result message carrying the output of a tool invocation.
TransformOptions: Message transformation between provider formats. Options that control how messages are transformed between providers.
UserMessage: Stable since 0.63.0
Message types for constructing conversations. User message sent to the model.
VertexProvider: Google Vertex AI provider implementation. Google Vertex AI provider

Enums§

AssistantRole: Stable since 0.63.0
Message types for constructing conversations. AssistantRole.
ContentBlock: Stable since 0.63.0
Message types for constructing conversations. Content block union (untagged for flexibility).
Error: Result type alias for oxi-ai operations. Unified error type for oxi-ai
ImageContentType: Stable since 0.63.0
Message types for constructing conversations. ImageContentType.
Message: Stable since 0.63.0
Message types for constructing conversations. Message union tagged by role.
MessageContent: Stable since 0.63.0
Message types for constructing conversations. Message content – either a plain text string or a list of structured blocks.
ProviderError: Stable since 0.63.0
Provider-specific error type for LLM operations. Provider-specific errors. `#[non_exhaustive]` — consumers MUST add a catch-all `_ =>` arm in their `match` expressions. Existing named variants are frozen; their meaning does not change between releases (see `docs/release-process.md`).
ProviderEvent: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Streaming events emitted by providers
TextContentType: Stable since 0.63.0
Message types for constructing conversations. TextContentType.
ThinkingContentType: Stable since 0.63.0
Message types for constructing conversations. ThinkingContentType.
ToolCallType: Stable since 0.63.0
Message types for constructing conversations. ToolCallType.
ToolResultRole: Stable since 0.63.0
Message types for constructing conversations. ToolResultRole.
ToolValidationError: Stable since 0.63.0
Tool definition and argument validation. Validation error
UserRole: Stable since 0.63.0
Message types for constructing conversations. UserRole.

Traits§

Provider: Stable since 0.63.0
Provider trait, streaming options, and provider registry. LLM provider trait

Functions§

anthropic_to_google: Message transformation between provider formats. Convert Anthropic-format messages to Google format.
anthropic_to_openai: Message transformation between provider formats. Convert Anthropic-format messages to OpenAI format.
complete: High-level completion and token estimation. High-level complete function that collects all streaming events and returns the final assistant message.
context_usage: Token estimation and context usage helpers. Calculate context length usage percentage.
create_builtin_provider: Built-in provider helpers (re-exported from providers). Create a built-in provider by name.
create_builtin_provider_with_options: Built-in provider helpers (re-exported from providers). Create a built-in provider with optional credential and base URL overrides.
custom_provider_names: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Return the set of currently registered custom provider names (global registry).
dynamic_models: Runtime model registry for dynamically registered models.
estimate: Token estimation and context usage helpers. Estimate token count using a hybrid algorithm that combines character-based and word-based heuristics.
estimate_tokens: High-level completion and token estimation. Estimate token count using a hybrid algorithm that combines character-based and word-based heuristics.
estimate_words: Token estimation and context usage helpers. Estimate tokens based on word count.
fetch_models_async: Model fetching utilities (async and blocking). Fetch the model list from an OpenAI-compatible /v1/models endpoint asynchronously.
fetch_models_blocking: Model fetching utilities (async and blocking). Fetch the model list from an OpenAI-compatible /v1/models endpoint using the shared reqwest::blocking client.
get_all_provider_names: Built-in provider helpers (re-exported from providers). Get all provider names (primary names only).
get_builtin_provider: Built-in provider helpers (re-exported from providers). Look up a built-in provider by name or alias.
get_model: Runtime model registry for dynamically registered models.
get_models: Runtime model registry for dynamically registered models.
get_provider: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Get a provider by name
get_provider_arc: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Get a provider by name, returning Arc (for router delegation).
get_provider_env_key: Built-in provider helpers (re-exported from providers). Get the environment variable name for a provider.
get_provider_env_keys: Built-in provider helpers (re-exported from providers). Get all environment variable names for a provider (primary + extras).
get_providers: Runtime model registry for dynamically registered models.
google_to_openai: Message transformation between provider formats. Convert Google-format messages to OpenAI format.
is_builtin_provider: Built-in provider helpers (re-exported from providers). Check if a provider name or alias is a known built-in.
lookup_model: Runtime model registry for dynamically registered models.
normalize_messages: Provider-specific message normalization (empty content filtering, tool ID scrubbing, reasoning injection, tool-use ordering fix). Normalize messages for a specific provider.
normalize_tool_call_id: Message transformation between provider formats. Normalize a tool call ID for cross-provider compatibility.
openai_to_anthropic: Message transformation between provider formats. Convert OpenAI-format messages to Anthropic format.
product_home_dir: Product home-directory resolution (OXI_HOME → ~/.oxi). The product home directory.
progress_callback: Stable since 0.63.0
Tool definition and argument validation. Create a progress callback from a closure
register_model: Runtime model registry for dynamically registered models.
register_provider: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Register a custom provider at runtime (global registry).
transform_for_provider: Stable since 0.63.0
Message types for constructing conversations. Transform messages for cross-provider compatibility.
transform_messages: Message transformation between provider formats. Transform a slice of Messages from one provider API to another.
transform_messages_for_model: Message transformation between provider formats. Transform messages for a target model, handling:
unregister_model: Runtime model registry for dynamically registered models.
unregister_provider: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Unregister a previously registered custom provider (global registry).
validate_args: Stable since 0.63.0
Tool definition and argument validation. Validate tool arguments against a JSON Schema

Type Aliases§

ProgressCallback: Stable since 0.63.0
Tool definition and argument validation. Callback type for progress updates
Result: Result type alias for oxi-ai operations. Result type alias
StreamResult: Stable since 0.63.0
Provider trait, streaming options, and provider registry. Stream result type alias

Crate oxi_ai

Crate oxi_ai Copy item path

Re-exports§

Modules§

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate oxi_ai