Struct MtpHead 

pub struct MtpHead {
    pub enorm: GpuTensor,
    pub hnorm: GpuTensor,
    pub eh_proj: GpuTensor,
    pub attn_norm: GpuTensor,
    pub post_attn_norm: GpuTensor,
    pub mixer: Mixer,
    pub ffn: Ffn,
    pub shared_head_norm: Option<GpuTensor>,
    pub shared_head_head: Option<GpuTensor>,
    pub d2t: Option<Vec<u32>>,
    pub geom: Option<DraftGeom>,
}

Qwen3.5 NextN/MTP head: a full transformer block (attn+FFN, same tensors as a trunk layer) plus the MTP glue (enorm/hnorm/eh_proj that fold the next-token embedding into the trunk hidden, and an optional shared_head_norm/head). Loaded from blk.{n_trunk}.* — the block the trunk loop drops. Used for speculative decode (drafts 1 token per call). See research/mtp/MTP-PLAN.md.

Fields

enorm: GpuTensor

hnorm: GpuTensor

eh_proj: GpuTensor

attn_norm: GpuTensor

post_attn_norm: GpuTensor

mixer: Mixer

ffn: Ffn

shared_head_norm: Option<GpuTensor>

shared_head_head: Option<GpuTensor>

d2t: Option<Vec<u32>>

FR-Spec draft->target vocab map: the draft lm_head is TRIMMED to the highest-frequency tokens (e.g. 32768 rows of the full 248320-row head); d2t[draft_idx] = the target vocab token id of trimmed row draft_idx. None for a full-vocab head (identity map). Host-side: the draft argmax already lands on host as one u32, so the map is a single Vec index.

geom: Option<DraftGeom>

DISTILLED-STUDENT geometry (None = the natural NextN block at trunk shape). A distilled draft (StudentSV) runs the same block structure at a narrower inner width with fewer heads, then up-projects back to n_embd (out_up) — the chain carrier and the head input stay at n_embd, so the trunk/verify interface is unchanged. Selected by the presence of blk.N.nextn.out_up.weight in a MEMRA_MTP_DRAFT file.

Implementations

impl MtpHead

pub fn load_draft( e: &Engine, g: &GgufFile, main_cfg: &ModelConfig, ) -> Result<Self, Box<dyn Error>>

Load an MTP/NextN head from a STANDALONE draft GGUF (MEMRA_MTP_DRAFT override). The draft file carries ONLY the NextN block (blk.N.nextn.* glue + attn/ffn) plus its own lm_head (output.weight) — which for an FR-Spec draft is TRIMMED to the top-frequency rows, with a d2t (i32/i64) tensor mapping trimmed-row index -> target vocab token id. Draft-token embedding still uses the MAIN model’s token_embd (identical weights, saves VRAM), so the draft file’s full-vocab token_embd copy is ignored.