Struct MoeOffloadEstimate 

pub struct MoeOffloadEstimate {
    pub expert_param_bytes: usize,
    pub num_moe_layers: usize,
    pub num_experts: usize,
    pub gpu_expert_budget_per_layer: usize,
    pub all_expert_weight_bytes: usize,
    pub resident_expert_weight_bytes: usize,
}

Estimate peak memory for running graph on a session bound to registry. Pure analysis — runs the memory planner internally and queries the registry for weight bytes; doesn’t compile or execute. MoE offload sizing (TIDE enable_predictive_expert_offload).

Fields

expert_param_bytes: usize

Bytes for one expert FFN (gate+up+down) at runtime dtype.

num_moe_layers: usize

num_experts: usize

gpu_expert_budget_per_layer: usize

Experts pinned on device per layer after budget clamp.

all_expert_weight_bytes: usize

All experts resident on host+device (upper bound).

resident_expert_weight_bytes: usize

Only gpu_expert_budget_per_layer experts per layer on device.

Implementations

impl MoeOffloadEstimate

pub fn peak_with_offload(&self, base: &MemoryEstimate) -> usize

Resident expert weights + non-expert peak from MemoryEstimate.

Trait Implementations

impl Clone for MoeOffloadEstimate

fn clone(&self) -> MoeOffloadEstimate

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MoeOffloadEstimate

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.