Struct EnginePool 

pub struct EnginePool { /* private fields */ }

A pool for reusing backend engine instances

Implementations

impl EnginePool

pub fn new(config: EnginePoolConfig) -> Self

pub async fn set_eviction_callback<F>(&self, cb: F)

where F: Fn(u32, usize, Arc<dyn Engine>) + Send + Sync + 'static,

Set an eviction callback that will be invoked whenever an engine is evicted. The callback receives (model_id, device_id, evicted_engine).

pub async fn clear_eviction_callback(&self)

Clear any previously set eviction callback.

pub fn start_health_check_task(&self) -> JoinHandle<()>

pub fn max_size(&self) -> usize

pub fn min_size(&self) -> usize

pub fn ttl(&self) -> Duration

pub fn health_check_interval(&self) -> Duration

pub async fn get( &self, model_id: u32, device_id: usize, ) -> Option<Arc<dyn Engine>>

Get an existing engine from the pool if available Returns None if engine doesn’t exist or fails health check

pub async fn put( &self, model_id: u32, device_id: usize, engine: Arc<dyn Engine>, )

Add an engine to the pool

If adding this engine causes an eviction, the eviction callback (if set) will be invoked asynchronously in a separate task to avoid blocking pool operations.

pub async fn remove(&self, model_id: u32, device_id: usize)

Remove an engine from the pool (e.g. on error or unload)

pub async fn len(&self) -> usize

pub async fn is_empty(&self) -> bool

pub async fn warmup<F, Fut>( &self, engine_configs: Vec<(u32, usize)>, engine_factory: F, ) -> Result<(), EngineError>

where F: Fn(u32, usize) -> Fut, Fut: Future<Output = Result<Arc<dyn Engine>, EngineError>>,

Warm up the pool by pre-creating engines

engine_factory is an async function that creates an engine for given (model_id, device_id)

pub async fn pool_metrics(&self) -> PoolMetrics

Trait Implementations

impl Clone for EnginePool

fn clone(&self) -> EnginePool

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.