Struct MlaEngine 

pub struct MlaEngine {
    pub embedding: WeightMatrix,
    pub layers: Vec<MlaLayerWeights>,
    pub final_norm: Vec<f32>,
    pub output_head: WeightMatrix,
    pub mla_cfg: MlaConfig,
    pub rms_norm_eps: f32,
    pub hidden_dim: usize,
    pub moe: Option<MlaMoeRuntime>,
}

Multi-layer MLA stack for DeepSeek-2 / Mistral-4-style GGUF serve.

Uses crate::mla::mla_forward_token with asymmetric K/V caches (plain Vec<f32>, not KvCache). Layers [0, leading_dense) use dense SwiGLU; later layers use MoE (ferrox_moe) when the GGUF carries experts — fail-closed at load if expert tensors are missing.

Fields

embedding: WeightMatrix

layers: Vec<MlaLayerWeights>

final_norm: Vec<f32>

output_head: WeightMatrix

mla_cfg: MlaConfig

rms_norm_eps: f32

hidden_dim: usize

moe: Option<MlaMoeRuntime>

Present when any layer uses MlaLayerFfn::Moe.

Implementations

impl MlaEngine

pub fn new_state(&self) -> MlaDecodeState

Trait Implementations

impl Engine for MlaEngine

type State = MlaDecodeState

fn new_state(&self) -> MlaDecodeState

Builds fresh (empty) per-layer state for a new request.

fn vocab_size(&self) -> usize

fn forward_token( &self, token_id: usize, _pos: usize, state: &mut Self::State, ) -> Vec<f32>