Struct BankResidency 

pub struct BankResidency { /* private fields */ }

The validated per-layer residency of an attached set of expert banks.

Constructing one is the check; holding one is the proof the three invariants in the module docs hold for this configuration.

Implementations

impl BankResidency

pub fn new( labels: &[HostResidency], num_layers: usize, cpu_layers: &BTreeSet<u32>, prefill_overlap: bool, ) -> Result<Self, ResidencyError>

Attach labels (the achieved ones) to a model of num_layers MoE layers.

cpu_layers is the set already routed to the CPU executor and prefill_overlap is whether the overlap path is still enabled. Both are inputs, not outputs: this refuses a bad combination rather than repairing it, because repairing it here would mean silently moving a layer to another executor after the cache geometry was already sized against the old answer.

pub fn all_pinned(num_layers: usize) -> Self

The default when no plan was in force: every layer pinned, no invariant to check, prefill overlap free to stay on.

pub fn num_layers(&self) -> usize

pub fn labels(&self) -> &[HostResidency]

pub fn label(&self, layer_id: u32) -> HostResidency

Panics

If layer_id is not a layer of this model.

pub fn unpinned_layers(&self) -> &BTreeSet<u32>

Layers with no device address. The copy plan skips their rows entirely.

pub fn is_unpinned(&self, layer_id: u32) -> bool

pub fn has_unpinned(&self) -> bool

Whether any layer is locked or pageable – the same signal that must have turned prefill overlap off.

pub fn copy_route( &self, layer_id: u32, whole_layer: bool, ) -> Result<CopyRoute, ResidencyError>

How a copy staged for layer_id must be carried out.

whole_layer is whether the caller staged the whole-layer materialize (every expert, slot e for expert e) rather than an LRU ensure. An unpinned layer accepts only the former; a pinned layer takes the indexed device path either way, since a registered bank can serve both.

Trait Implementations

impl Clone for BankResidency

fn clone(&self) -> BankResidency

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BankResidency

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for BankResidency

impl PartialEq for BankResidency

fn eq(&self, other: &BankResidency) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for BankResidency