Struct HotExpertCache 

pub struct HotExpertCache { /* private fields */ }

Device slot cache for one MoE layer.

Implementations

impl HotExpertCache

pub fn new(num_experts: usize, num_slots: usize) -> Self

New cache with the first min(num_slots, num_experts) experts pre-loaded, matching ExpertPool::new’s initial residency so the two structures start consistent (no spurious first-refresh reload).

pub fn num_experts(&self) -> usize

pub fn num_slots(&self) -> usize

pub fn slot_of(&self, expert: usize) -> Option<usize>

Device slot holding expert, or None if the expert is cold.

pub fn is_resident(&self, expert: usize) -> bool

Whether expert currently occupies a device slot.

pub fn slot_to_expert(&self) -> &[Option<usize>]

slot -> expert table (None = empty slot).

pub fn expert_to_slot(&self) -> &[Option<usize>]

expert -> slot table (None = cold).

pub fn reconcile(&mut self, resident_mask: &[bool]) -> Vec<SlotLoad>

Bring device slots in line with resident_mask (typically ExpertPool::resident_mask), keeping experts that stay resident in their existing slot (no reload). Returns the minimal set of host→device loads for newly-resident experts.

Two clean passes so leftover slots (when fewer experts are hot than there are slots) are always left consistently empty rather than dangling on a now-cold expert.

pub fn route(&self, expert_idx: &[u32], cold_sentinel: u32) -> SlotRoute

Split per-token routed expert indices into slot indices + a cold-token list. Resident tokens get their device slot; cold tokens get cold_sentinel in slot_idx and an entry in cold.

Trait Implementations

impl Clone for HotExpertCache

fn clone(&self) -> HotExpertCache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HotExpertCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.