Struct SlotGeometry 

pub struct SlotGeometry {
    pub num_layers: usize,
    pub slots: usize,
    pub row_bytes: Vec<usize>,
}

The shape of one pool: how many slots, how many banks, and how wide a row is in each bank.

row_bytes is per bank because the banks are not interchangeable: gate and up are [ffn_dim, hidden] while down is [hidden, ffn_dim], and a checkpoint may quantize them differently, so one row width for all three would be wrong for at least one of them.

Fields

num_layers: usize

slots: usize

Slots in the pool. This is the cache_size the ExpertCache was built with; the two must agree or a slot the planner names will not exist here.

row_bytes: Vec<usize>

Bytes per expert row, one entry per bank, in the bank order the caller uses everywhere else (conventionally gate, up, down).

Implementations

impl SlotGeometry

pub fn banks(&self) -> usize

pub fn bytes(&self) -> u64

Device bytes the whole pool occupies. What a VRAM budget is checked against before anything is allocated.

Trait Implementations

impl Clone for SlotGeometry

fn clone(&self) -> SlotGeometry

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SlotGeometry

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for SlotGeometry

impl PartialEq for SlotGeometry

fn eq(&self, other: &SlotGeometry) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for SlotGeometry