Enum SequencePool 

pub enum SequencePool {
    KvBased(KvCachePool),
    Ssm(SsmStatePool),
}

Dispatch-enum over the two pool backends.

At model-load time the engine inspects the loaded architecture and constructs either a KvBased pool (for any transformer) or an Ssm pool (for Mamba-2 and similar). Both variants expose the same interface through SequencePool’s methods.

KV-based pooling

The KV-based variant stores state in a KvCachePool of page-sized slabs. Slots are identified by page indices returned by KvCachePool::alloc.

SSM pooling

The SSM variant stores the full per-layer recurrent state in an SsmStatePool. The alloc_ssm / release_ssm helpers delegate to it.

Variants

KvBased(KvCachePool)

Attention-transformer pool (KV cache pages).

Ssm(SsmStatePool)

SSM pool (per-layer recurrent hidden states).

Implementations

impl SequencePool

pub fn alloc_kv(&mut self) -> PoolResult<usize>

Allocate a slot from the KV-based pool.

Returns the page index on success.

Errors

PoolError::Exhausted if the pool is full. PoolError::InvalidSlot(usize::MAX) if called on an Ssm variant.

pub fn free_kv(&mut self, page_idx: usize) -> PoolResult<()>

Free a page in the KV-based pool.

Errors

PoolError::InvalidSlot if called on an Ssm variant.

pub fn alloc_ssm(&mut self, request_id: u64) -> PoolResult<usize>

Allocate an SSM slot bound to request_id.

Errors

PoolError::Exhausted if the pool is full. PoolError::InvalidSlot(usize::MAX) if called on a KvBased variant.

pub fn release_ssm(&mut self, idx: usize) -> PoolResult<()>

Release an SSM slot by index.

Errors

PoolError::InvalidSlot if idx is invalid or already free, or if called on a KvBased variant.

pub fn ssm_slot(&self, idx: usize) -> Option<&SequenceSlot>

Get an immutable reference to an SSM slot.

Returns None for KV-based pools or out-of-range indices.

pub fn ssm_slot_mut(&mut self, idx: usize) -> Option<&mut SequenceSlot>

Get a mutable reference to an SSM slot.

Returns None for KV-based pools or out-of-range indices.

pub fn is_kv_based(&self) -> bool

Returns true if this pool uses the KV-cache backend.

pub fn is_ssm(&self) -> bool

Returns true if this pool uses the SSM backend.