Struct MoeForwardBucketedParams 

pub struct MoeForwardBucketedParams<'a, B: QuantLlmBackend + BackendMoeFused> {

    pub ctx: &'a mut B::Context,
    pub x: &'a B::Buffer,
    pub router_logits: &'a B::Buffer,
    pub out: &'a mut B::Buffer,
    pub batch: usize,
    pub hidden_size: usize,
    pub expert_intermediate: usize,
    pub num_experts: usize,
    pub top_k: usize,
    pub norm_topk_prob: bool,
    pub experts: &'a ExpertStack<B>,
    pub x_packed: &'a mut B::Buffer,
    pub gate_up_packed: &'a mut B::Buffer,
    pub silu_packed: &'a mut B::Buffer,
    pub down_packed: &'a mut B::Buffer,
    pub route_scratch: &'a mut MoeRouteScratch,
    pub device_route: Option<DeviceRouteScratch<'a, B>>,
}

Bucketed MoE forward: gather → per-expert m=N Marlin GEMM → silu_mul → per-expert m=N Marlin GEMM → moe_combine.

Replaces the batch × top_k m=1 dispatch loop in moe_forward with num_active_experts × 2 m=tokens_per_expert dispatches. For prefill (m=512+), this is a 30× reduction in GEMM launches AND each GEMM runs at a much more efficient m than the m=1 path. For decode (m=1), the number of dispatches is similar but we still benefit from the gather/combine kernel pattern (one launch each instead of 2 per pair).

Requires: scratch buffers x_packed [total_pairs, hidden], gate_up_packed [total_pairs, 2*expert_inter], silu_packed [total_pairs, expert_inter], and down_packed [total_pairs, hidden] provisioned by the caller. The caller is responsible for sizing these to batch * top_k rows (worst-case all top_k pairs alive).

Fields

ctx: &'a mut B::Context

x: &'a B::Buffer

router_logits: &'a B::Buffer

out: &'a mut B::Buffer

batch: usize

hidden_size: usize

expert_intermediate: usize

num_experts: usize

top_k: usize

norm_topk_prob: bool

experts: &'a ExpertStack<B>

x_packed: &'a mut B::Buffer

gate_up_packed: &'a mut B::Buffer

silu_packed: &'a mut B::Buffer

down_packed: &'a mut B::Buffer

route_scratch: &'a mut MoeRouteScratch

device_route: Option<DeviceRouteScratch<'a, B>>