Struct PipelinePool 

pub struct PipelinePool { /* private fields */ }

Fixed pool of pipeline slots over ONE shared mmap’d model (roadmap §3 «serving полностью сериализован», этап 5.1): the weights are zero-copy shared through Arc<CmfModel>, each slot owns its KV-cache / recurrent state / sampler / workspace. A request checks a slot out for the duration of one generation, so up to slots requests decode CONCURRENTLY; excess requests queue fairly on the semaphore. This is bounded-concurrency serving, not yet continuous batching (этап 5.2+).

Implementations

impl PipelinePool

pub fn new(pipelines: Vec<Pipeline>) -> Self

pub fn n_slots(&self) -> usize

pub async fn acquire(&self) -> SlotGuard

Wait for a free slot and check it out. With permits == slots, holding a permit guarantees the try_lock scan finds a free slot.