Struct AiClient 

pub struct AiClient { /* private fields */ }

Unified AI client

Usage example:

use ai_lib::{AiClient, Provider, ChatCompletionRequest, Message, Role};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Switch model provider by changing Provider value
    let client = AiClient::new(Provider::Groq)?;
     
    let request = ChatCompletionRequest::new(
        "test-model".to_string(),
        vec![Message {
            role: Role::User,
            content: ai_lib::types::common::Content::Text("Hello".to_string()),
            function_call: None,
        }],
    );
     
    // Note: Set GROQ_API_KEY environment variable for actual API calls
    // Optional: Set AI_PROXY_URL environment variable to use proxy server
    // let response = client.chat_completion(request).await?;
     
    println!("Client created successfully with provider: {:?}", client.current_provider());
    println!("Request prepared for model: {}", request.model);
     
    Ok(())
}

Proxy Configuration

Configure proxy server by setting the AI_PROXY_URL environment variable:

export AI_PROXY_URL=http://proxy.example.com:8080

Supported proxy formats:

• HTTP proxy: http://proxy.example.com:8080
• HTTPS proxy: https://proxy.example.com:8080
• With authentication: http://user:pass@proxy.example.com:8080

Implementations

impl AiClient

pub fn default_chat_model(&self) -> String

Get the effective default chat model for this client (honors custom override)

pub fn new(provider: Provider) -> Result<Self, AiLibError>

Create a new AI client

Arguments

• provider - The AI model provider to use

Returns

• Result<Self, AiLibError> - Client instance on success, error on failure

Example

use ai_lib::{AiClient, Provider};

let client = AiClient::new(Provider::Groq)?;

pub fn with_failover(self, providers: Vec<Provider>) -> Self

Configure a basic failover chain for automatic provider switching on failures.

When a request fails with a retryable error (network, timeout, rate limit, 5xx), the client will automatically attempt subsequent providers in the specified order.

Execution Behavior

• Error Types: Only retryable errors trigger failover (network, timeout, rate limit, 5xx)
• Order: Providers are tried in the exact order specified in the vector
• Skip Current: The current provider is automatically skipped if it appears in the chain
• State Preservation: Routing selections and request modifications are preserved during failover

Integration with Other Features

• Routing: When used with with_routing_array(), the selected model is preserved across failover attempts. Failover providers will use the same model selection.
• Interceptors: Failover happens after interceptor processing, so interceptors can modify requests before failover attempts.
• Metrics: Failover attempts are tracked with failover.attempts, failover.success, and failover.error metrics.

Examples

use ai_lib::{AiClient, Provider};

// Basic failover configuration
let client = AiClient::new(Provider::OpenAI)?
    .with_failover(vec![Provider::Anthropic, Provider::Groq]);

// Combined with routing (requires routing_mvp feature)
#[cfg(feature = "routing_mvp")]
{
    let mut array = ai_lib::provider::models::ModelArray::new("production");
    // ... configure array
    let client = client
        .with_routing_array(array)
        .with_failover(vec![Provider::Anthropic, Provider::Groq]);
}

// Empty vector disables failover
let client = client.with_failover(vec![]);

Limitations (OSS)

This is a lightweight OSS feature. For advanced capabilities, consider ai-lib-pro:

• Weighted failover based on provider performance
• SLO-aware failover policies
• Cost-based failover decisions
• Advanced health checking and circuit breaking

Arguments

• providers - Ordered list of fallback providers. Empty vector disables failover.

Returns

• Self - Client instance with failover configuration

pub fn with_options( provider: Provider, opts: ConnectionOptions, ) -> Result<Self, AiLibError>

Create client with minimal explicit options (base_url/proxy/timeout). Not all providers support overrides; unsupported providers ignore unspecified fields gracefully.

pub fn connection_options(&self) -> Option<&ConnectionOptions>

pub fn builder(provider: Provider) -> AiClientBuilder

Create a new AI client builder

The builder pattern allows more flexible client configuration:

• Automatic environment variable detection
• Support for custom base_url and proxy
• Support for custom timeout and connection pool configuration

Arguments

• provider - The AI model provider to use

Returns

• AiClientBuilder - Builder instance

Example

use ai_lib::{AiClient, Provider};

// Simplest usage - automatic environment variable detection
let client = AiClient::builder(Provider::Groq).build()?;

// Custom base_url and proxy
let client = AiClient::builder(Provider::Groq)
    .with_base_url("https://custom.groq.com")
    .with_proxy(Some("http://proxy.example.com:8080"))
    .build()?;

pub fn new_with_metrics( provider: Provider, metrics: Arc<dyn Metrics>, ) -> Result<Self, AiLibError>

Create AiClient with injected metrics implementation

pub fn with_metrics(self, metrics: Arc<dyn Metrics>) -> Self

Set metrics implementation on client

pub async fn chat_completion( &self, request: ChatCompletionRequest, ) -> Result<ChatCompletionResponse, AiLibError>

Send chat completion request

This method supports multiple routing and failover strategies that work together:

Execution Flow

1. Routing (if enabled): If request.model == "__route__" and routing_mvp feature is enabled, the client will select the best available model from the configured ModelArray based on health checks and load balancing strategy.
2. Request Execution: The request is sent to the current provider with the selected model.
3. Failover (if enabled): If the request fails with a retryable error (network, timeout, rate limit, 5xx), the client will automatically try the configured failover providers in order, preserving the routing selection.

Feature Interaction

• Routing + Failover: When both are configured, routing selection is preserved during failover attempts. The failover providers will use the same model selection.
• Interceptors: Applied after routing but before failover, allowing for request/response modification and monitoring.

Examples

use ai_lib::{AiClient, Provider, ChatCompletionRequest, Message, Role};
use ai_lib::types::common::Content;

// Basic usage with failover
let client = AiClient::new(Provider::OpenAI)?
    .with_failover(vec![Provider::Anthropic, Provider::Groq]);

// With routing (requires routing_mvp feature)
#[cfg(feature = "routing_mvp")]
{
    let mut array = ai_lib::provider::models::ModelArray::new("production");
    // ... configure array with endpoints
    let client = client.with_routing_array(array);
}

// Request with routing (will select best model from array)
let request = ChatCompletionRequest::new(
    "__route__".to_string(),  // Special sentinel for routing
    vec![Message {
        role: Role::User,
        content: Content::Text("Hello".to_string()),
        function_call: None,
    }],
);

Arguments

• request - Chat completion request

Returns

• Result<ChatCompletionResponse, AiLibError> - Response on success, error on failure

pub async fn chat_completion_stream( &self, request: ChatCompletionRequest, ) -> Result<Box<dyn Stream<Item = Result<ChatCompletionChunk, AiLibError>> + Send + Unpin>, AiLibError>

Streaming chat completion request

This method provides the same routing and failover capabilities as chat_completion(), but returns a streaming response for real-time processing.

Execution Flow

1. Routing (if enabled): If request.model == "__route__" and routing_mvp feature is enabled, the client will select the best available model from the configured ModelArray based on health checks and load balancing strategy.
2. Stream Request Execution: The streaming request is sent to the current provider with the selected model.
3. Failover (if enabled): If the stream request fails with a retryable error (network, timeout, rate limit, 5xx), the client will automatically try the configured failover providers in order, preserving the routing selection.

Feature Interaction

• Routing + Failover: When both are configured, routing selection is preserved during failover attempts. The failover providers will use the same model selection.
• Interceptors: Applied after routing but before failover, allowing for request/response modification and monitoring.
• Backpressure: The configured backpressure controller is applied to the final stream.

Examples

use ai_lib::{AiClient, Provider, ChatCompletionRequest, Message, Role};
use ai_lib::types::common::Content;
use futures::stream::StreamExt;

// Basic usage with failover
let client = AiClient::new(Provider::OpenAI)?
    .with_failover(vec![Provider::Anthropic, Provider::Groq]);

// With routing (requires routing_mvp feature)
#[cfg(feature = "routing_mvp")]
{
    let mut array = ai_lib::provider::models::ModelArray::new("production");
    // ... configure array with endpoints
    let client = client.with_routing_array(array);
}

// Streaming request with routing (will select best model from array)
let request = ChatCompletionRequest::new(
    "__route__".to_string(),  // Special sentinel for routing
    vec![Message {
        role: Role::User,
        content: Content::Text("Hello".to_string()),
        function_call: None,
    }],
);

let mut stream = client.chat_completion_stream(request).await?;
while let Some(chunk) = stream.next().await {
    match chunk {
        Ok(chunk) => println!("Received: {:?}", chunk),
        Err(e) => eprintln!("Stream error: {}", e),
    }
}

Arguments

• request - Chat completion request

Returns

• Result<impl Stream<Item = Result<ChatCompletionChunk, AiLibError>>, AiLibError> - Stream response on success

pub async fn chat_completion_stream_with_cancel( &self, request: ChatCompletionRequest, ) -> Result<(Box<dyn Stream<Item = Result<ChatCompletionChunk, AiLibError>> + Send + Unpin>, CancelHandle), AiLibError>

Streaming chat completion request with cancel control

Arguments

• request - Chat completion request

Returns

• Result<(impl Stream<Item = Result<ChatCompletionChunk, AiLibError>> + Send + Unpin, CancelHandle), AiLibError> - Returns streaming response and cancel handle on success

pub async fn chat_completion_batch( &self, requests: Vec<ChatCompletionRequest>, concurrency_limit: Option<usize>, ) -> Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError>

Batch chat completion requests

Arguments

• requests - List of chat completion requests
• concurrency_limit - Maximum concurrent request count (None means unlimited)

Returns

• Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError> - Returns response results for all requests

Example

use ai_lib::{AiClient, Provider, ChatCompletionRequest, Message, Role};
use ai_lib::types::common::Content;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = AiClient::new(Provider::Groq)?;
     
    let requests = vec![
        ChatCompletionRequest::new(
            "llama3-8b-8192".to_string(),
            vec![Message {
                role: Role::User,
                content: Content::Text("Hello".to_string()),
                function_call: None,
            }],
        ),
        ChatCompletionRequest::new(
            "llama3-8b-8192".to_string(),
            vec![Message {
                role: Role::User,
                content: Content::Text("How are you?".to_string()),
                function_call: None,
            }],
        ),
    ];
     
    // Limit concurrency to 5
    let responses = client.chat_completion_batch(requests, Some(5)).await?;
     
    for (i, response) in responses.iter().enumerate() {
        match response {
            Ok(resp) => println!("Request {}: {}", i, resp.choices[0].message.content.as_text()),
            Err(e) => println!("Request {} failed: {}", i, e),
        }
    }
     
    Ok(())
}

pub async fn chat_completion_batch_smart( &self, requests: Vec<ChatCompletionRequest>, ) -> Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError>

Smart batch processing: automatically choose processing strategy based on request count

Arguments

• requests - List of chat completion requests

Returns

• Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError> - Returns response results for all requests

pub async fn list_models(&self) -> Result<Vec<String>, AiLibError>

Batch chat completion requests

Arguments

• requests - List of chat completion requests
• concurrency_limit - Maximum concurrent request count (None means unlimited)

Returns

• Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError> - Returns response results for all requests

Example

use ai_lib::{AiClient, Provider, ChatCompletionRequest, Message, Role};
use ai_lib::types::common::Content;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = AiClient::new(Provider::Groq)?;
     
    let requests = vec![
        ChatCompletionRequest::new(
            "llama3-8b-8192".to_string(),
            vec![Message {
                role: Role::User,
                content: Content::Text("Hello".to_string()),
                function_call: None,
            }],
        ),
        ChatCompletionRequest::new(
            "llama3-8b-8192".to_string(),
            vec![Message {
                role: Role::User,
                content: Content::Text("How are you?".to_string()),
                function_call: None,
            }],
        ),
    ];
     
    // Limit concurrency to 5
    let responses = client.chat_completion_batch(requests, Some(5)).await?;
     
    for (i, response) in responses.iter().enumerate() {
        match response {
            Ok(resp) => println!("Request {}: {}", i, resp.choices[0].message.content.as_text()),
            Err(e) => println!("Request {} failed: {}", i, e),
        }
    }
     
    Ok(())
}

Get list of supported models

Returns

• Result<Vec<String>, AiLibError> - Returns model list on success, error on failure

pub fn switch_provider(&mut self, provider: Provider) -> Result<(), AiLibError>

Switch AI model provider

Arguments

• provider - New provider

Returns

• Result<(), AiLibError> - Returns () on success, error on failure

Example

use ai_lib::{AiClient, Provider};

let mut client = AiClient::new(Provider::Groq)?;
// Switch from Groq to Groq (demonstrating switch functionality)
client.switch_provider(Provider::Groq)?;

pub fn current_provider(&self) -> Provider

Get current provider

pub fn build_simple_request<S: Into<String>>( &self, prompt: S, ) -> ChatCompletionRequest

Convenience helper: construct a request with the provider’s default chat model. This does NOT send the request. Uses custom default model if set via AiClientBuilder, otherwise uses provider default.

pub fn build_simple_request_with_model<S: Into<String>>( &self, prompt: S, model: S, ) -> ChatCompletionRequest

Convenience helper: construct a request with an explicitly specified chat model. This does NOT send the request.

pub fn build_multimodal_request<S: Into<String>>( &self, prompt: S, ) -> Result<ChatCompletionRequest, AiLibError>

Convenience helper: construct a request with the provider’s default multimodal model. This does NOT send the request. Uses custom default model if set via AiClientBuilder, otherwise uses provider default.

pub fn build_multimodal_request_with_model<S: Into<String>>( &self, prompt: S, model: S, ) -> ChatCompletionRequest

Convenience helper: construct a request with an explicitly specified multimodal model. This does NOT send the request.

pub async fn quick_chat_text<P: Into<String>>( provider: Provider, prompt: P, ) -> Result<String, AiLibError>

One-shot helper: create a client for provider, send a single user prompt using the default chat model, and return plain text content (first choice).

pub async fn quick_chat_text_with_model<P: Into<String>, M: Into<String>>( provider: Provider, prompt: P, model: M, ) -> Result<String, AiLibError>

One-shot helper: create a client for provider, send a single user prompt using an explicitly specified chat model, and return plain text content (first choice).

pub async fn quick_multimodal_text<P: Into<String>>( provider: Provider, prompt: P, ) -> Result<String, AiLibError>

One-shot helper: create a client for provider, send a single user prompt using the default multimodal model, and return plain text content (first choice).

pub async fn quick_multimodal_text_with_model<P: Into<String>, M: Into<String>>( provider: Provider, prompt: P, model: M, ) -> Result<String, AiLibError>

One-shot helper: create a client for provider, send a single user prompt using an explicitly specified multimodal model, and return plain text content (first choice).

pub async fn quick_chat_text_with_options<P: Into<String>>( provider: Provider, prompt: P, options: ModelOptions, ) -> Result<String, AiLibError>

One-shot helper with model options: create a client for provider, send a single user prompt using specified model options, and return plain text content (first choice).

pub async fn upload_file(&self, path: &str) -> Result<String, AiLibError>

Upload a local file using provider’s multipart endpoint and return a URL or file id.

Behavior:

• For config-driven providers, uses their configured upload_endpoint if available.
• For OpenAI, posts to {base_url}/files.
• Providers without a known upload endpoint return UnsupportedFeature.