Struct GpuPool 

pub struct GpuPool {
    pub workers: Vec<Arc<GpuWorker>>,
}

Pool of GPU workers with placement strategy.

Fields

workers: Vec<Arc<GpuWorker>>

Implementations

impl GpuPool

pub fn worker_by_ordinal(&self, ordinal: usize) -> Option<Arc<GpuWorker>>

Return the worker bound to ordinal, if present in this pool.

pub fn resolve_explicit_placement_gpu( &self, placement: Option<&DevicePlacement>, ) -> Result<Option<usize>, String>

Validate a request/config placement against the active worker pool.

In multi-GPU worker mode a request may explicitly pin components to at most one GPU ordinal. Cross-GPU component placement would bypass the worker-affinity model entirely, so reject it here instead of letting the engines silently allocate on a sibling GPU.

pub fn find_loaded(&self, model_name: &str) -> Option<Arc<GpuWorker>>

Find a non-degraded worker that already has this model loaded on GPU. If multiple workers have it, prefer the one with fewer in-flight requests.

pub fn select_worker( &self, model_name: &str, estimated_vram: u64, ) -> Option<Arc<GpuWorker>>

Select the best worker for a model, using the placement strategy (checked in order):

1. Loaded and idle (model on GPU, no in-flight requests).
2. Loaded but busy — queue behind the warm copy instead of reloading.
3. Idle GPU with no model (spreads cold loads across free GPUs).
4. Non-degraded worker with the most headroom (will evict LRU).

pub fn select_worker_excluding( &self, model_name: &str, estimated_vram: u64, skip: &[usize], ) -> Option<Arc<GpuWorker>>

Same as [select_worker], but skips workers whose ordinal is in skip. Used by the dispatcher to retry after a try_send failure.

pub fn gpu_status(&self) -> Vec<GpuWorkerStatus>

Collect status from all workers.

pub fn worker_count(&self) -> usize

Number of GPU workers in the pool.