Struct MoeRouteScratch 

pub struct MoeRouteScratch {
    pub output: RouterOutput,
    pub probs: Vec<f32>,
    pub plan: MoeBucketPlan,
}

Reusable host-side scratch for moe_forward_bucketed. Holds the router output, softmax scratch buffer, and bucket plan, all reused across layers so the inner MoE forward path is allocation-free.

At c=32 / Qwen3-MoE / 48 layers, the previous fresh-Vec-per-layer pattern accounted for ~10 ms / token of pure CPU softmax+sort+alloc (25% of MoE wallclock — see docs/bench/cuda-rtx4090-2026-05-08-m3-moe).

Fields

output: RouterOutput

probs: Vec<f32>

Softmax buffer reused across all rows of all layers — sized to num_experts on first use.

plan: MoeBucketPlan

Implementations

impl MoeRouteScratch

pub fn new() -> Self

Trait Implementations

impl Default for MoeRouteScratch

fn default() -> Self

Returns the “default value” for a type.