Struct TensorMemoryPool 

pub struct TensorMemoryPool {
    pub slots: HashMap<u64, PoolSlot>,
    pub next_slot_id: u64,
    pub max_slots: usize,
    pub total_allocations: u64,
    pub total_deallocations: u64,
}

A slab-based memory pool that pre-allocates tensor buffers organised into size-class buckets to minimise allocation overhead on hot inference paths.

Fields

slots: HashMap<u64, PoolSlot>

All slots, keyed by slot_id.

next_slot_id: u64

Counter used to generate unique slot IDs.

max_slots: usize

Hard upper bound on the number of slots the pool may hold.

total_allocations: u64

Total successful allocations since creation.

total_deallocations: u64

Total successful deallocations since creation.

Implementations

impl TensorMemoryPool

pub fn new(max_slots: usize) -> Self

Create a new pool with the given slot capacity.

pub fn allocate(&mut self, requested_bytes: u64, tick: u64) -> Option<u64>

Attempt to check out a slot suitable for requested_bytes at the given logical tick.

pub fn deallocate(&mut self, slot_id: u64, tick: u64) -> bool

Return slot slot_id to the pool at the given tick.

Returns false if the slot does not exist or was already free.

pub fn evict_idle(&mut self, tick: u64, idle_ticks: u64)

Evict every free slot whose last-used tick is at least idle_ticks ticks in the past.

pub fn slots_for_class(&self, class: SizeClass) -> Vec<&PoolSlot>

Return all slots belonging to class, sorted by slot_id ascending.

pub fn stats(&self) -> MemoryPoolStats

Compute aggregate statistics for the current pool state.