Struct ClientPool 

pub struct ClientPool { /* private fields */ }

LLM Client Pool for managing reusable client connections

The pool maintains separate sub-pools for each registered provider, allowing efficient reuse of HTTP connections and client state.

Implementations

impl ClientPool

pub fn new(config: PoolConfig) -> Self

Create a new client pool with the given configuration

pub fn with_defaults() -> Self

Create a new client pool with default configuration

pub fn register_provider(&self, name: &str, provider: Provider)

Register a provider with the pool

This creates a sub-pool for the given provider that will manage client instances for that provider.

pub fn has_provider(&self, name: &str) -> bool

Check if a provider is registered

pub fn provider_names(&self) -> Vec<String>

List all registered provider names

pub async fn get(&self, provider_name: &str) -> Result<PooledClientGuard>

Get a client from the pool for the specified provider

The returned guard will automatically return the client to the pool when dropped.

pub async fn get_with_error( &self, provider_name: &str, ) -> Result<PooledClientGuard, PoolError>

pub async fn try_get( &self, provider_name: &str, ) -> Result<PooledClientGuard, PoolError>

pub fn stats(&self) -> PoolStats

Get pool statistics

pub fn cleanup_stale(&self) -> usize

Clean up stale connections across all providers

Returns the total number of connections removed.

pub fn start_cleanup_task(self: &Arc<Self>) -> JoinHandle<()>

Start a background task that periodically cleans up stale connections

The task runs until the pool is shut down.

pub fn shutdown(&self)

Gracefully shut down the pool

This prevents new clients from being acquired and drains all existing connections.

pub fn is_shutdown(&self) -> bool

Check if the pool is shut down

pub fn snapshot(&self) -> LLMPoolSnapshot

Trait Implementations

impl Default for ClientPool

fn default() -> Self

Returns the “default value” for a type.

impl Display for ClientPool

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.