Struct ExpertStreamBank 

pub struct ExpertStreamBank { /* private fields */ }

One stacked [n_experts, n, k] GGUF expert bank kept on disk.

Implementations

impl ExpertStreamBank

pub fn open( name: String, path: &Path, base_offset: u64, expert_bytes: usize, n_experts: usize, dtype: GgmlDType, n: usize, k: usize, ) -> Result<Arc<Self>>

pub fn dtype(&self) -> GgmlDType

pub fn n_experts(&self) -> usize

pub fn expert_dims(&self) -> (usize, usize)

pub fn expert_bytes(&self) -> usize

pub fn logical_bytes(&self) -> usize

Full size of the bank as if resident (shape/size accounting only).

pub fn fetch(&self, eid: u32) -> Result<Arc<[u8]>>

pinned -> LRU -> disk.

pub fn pin(&self, eid: u32) -> Result<()>

Load eid into the pinned hot-set (idempotent).

pub fn repin(&self, max_swaps: usize) -> Result<usize>

Live tier adaptation (--repin). Decay the session heat, then swap up to [REPIN_MAX_SWAPS] of the coldest pinned experts for the hottest streamed ones, each only when the heat gap clears the hysteresis margin ([tier_pick_swap]). Correctness neutral: the pinned set changes, never the bytes – a promoted expert is the same file slice it would stream, so indexed_moe_forward output is bit-identical. Cold experts fall back to the LRU (still warm), hot experts come from the LRU or disk. Call at a turn boundary. Returns the number of swaps performed.

pub fn prefetch(self: &Arc<Self>, eid: u32)

Enqueue a router-lookahead prefetch of eid. Best effort and non-blocking: dropped when disabled, out of range, or the worker queue is full – a missed prefetch only costs a later on-demand fetch, never correctness.

pub fn set_cap(&self, cap: usize)

Sized by finalize from RAM; also settable directly.

pub fn stats(&self) -> (usize, usize, usize, u64, u64)

(pinned, cached, cap, hits, misses).

pub fn resident_bytes(&self) -> usize