Struct GenAIProvider 

pub struct GenAIProvider { /* private fields */ }

Thread-safe LLM provider implementation using Arc<RwLock<>>.

This provider can be cheaply cloned and shared across multiple agents. Each clone shares the same underlying client and rate limiting state.

Implementations

impl GenAIProvider

pub fn new() -> Result<Self>

Creates a new GenAI provider with automatic configuration.

pub fn new_with_config( provider_type: Option<&str>, api_key: Option<&str>, ) -> Result<Self>

Creates a new GenAI provider with explicit configuration.

pub async fn get_total_tokens_used(&self) -> usize

Get total tokens used across all requests

pub async fn get_request_count(&self) -> usize

Get total request count

pub async fn add_tokens(&self, count: usize)

Add tokens to the total count

pub async fn get_available_models(&self, provider: &str) -> Result<Vec<String>>

Retrieves all available models for a specific provider.

pub async fn generate_response_simple( &self, model: &str, prompt: &str, ) -> Result<String>

Generates a simple text response without streaming. Includes exponential backoff retry for rate limits and transient errors.

pub async fn generate_response_with_retry( &self, model: &str, prompt: &str, max_retries: usize, ) -> Result<String>

Generates a response with configurable retry count and exponential backoff.

pub async fn generate_response_stream_to_channel( &self, model: &str, prompt: &str, tx: UnboundedSender<String>, ) -> Result<()>

Generates a streaming response and sends chunks via mpsc channel.

pub async fn generate_response_with_history( &self, model: &str, messages: Vec<ChatMessage>, ) -> Result<String>

Generate response with conversation history

pub async fn generate_response_with_options( &self, model: &str, prompt: &str, options: ChatOptions, ) -> Result<String>

Generate response with custom chat options

pub fn get_supported_providers() -> Vec<&'static str>

Get a list of supported providers

pub async fn get_available_providers(&self) -> Result<Vec<String>>

Get all available providers

pub async fn test_model(&self, model: &str) -> Result<bool>

Test if a model is available and working

pub async fn validate_model( &self, model: &str, provider_type: Option<&str>, ) -> Result<String>

Validate and get the best available model for a provider

Trait Implementations

impl Clone for GenAIProvider

fn clone(&self) -> GenAIProvider

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.