Expand description
Linear-attention cores, selected by arch.linear_core.kind
(descriptor-driven operators — Patent 15 claim 8).
Two tracks (owner decision 2026-07-04):
-
gated_delta_net— the faithful vendor operator (Qwen3.5 / Qwen3-Next). Default for models that ship GDN weights: conversion carries the tensors 1:1 and needs no training. Port of the validatedgated_delta_net(vmfcore/rust/src/forward.rs) against the numpy/torch oracle (vmfcore/gdn_layer.py). -
vmf_phase— the canonical core: token carries a phase θ; kernel φ(θ) = [cos θ; sin θ] gives a linear factorization; the condensate is a recurrent state S[head][p2, dv] with decay exp(−exp(A_log)). Noise-robust and simpler than vendor recurrences. Exotic operators are folded onto it at CONVERT time (--linear-core vmf_phase) and quality is restored by the offline heal — the research track and the production mechanism for Patent-15 skills (mask→heal→compress).
Both cores implement the same contract: *_forward (one position,
advances the state) and *_pair (fused two positions; lane 1
commits, lane 2 is tentative in scratch for speculative verify).
State lives in the layer’s linear_state: Vec<f32> and is resized
lazily by the core itself.
Structs§
- GdnCfg
- GdnWeights
- Weights of one GatedDeltaNet layer (
model.layers.{i}.linear_attn.*, names 1:1 with the source model — no fold, no training). - KdaCfg
- KdaWeights
- Short
Conv Cfg - Short
Conv Weights - Weights of one LFM2 short-convolution mixer
(
model.layers.{i}.short_conv.*, renamed from the vendorconv.*at convert time). No recurrent condensate — the only state is the causal conv ring (the lastkernel−1gated inputs per channel). - VmfPhase
Cfg - VmfPhase
Weights - Weights of one vmf_phase layer (
model.layers.{i}.vmf_attn.*).
Enums§
Functions§
- gdn_
forward - Forward one position through a GatedDeltaNet layer, advancing
state. - gdn_
forward_ batch - Batched GDN forward (prefill-GEMM): the qkv/z/a/b and out_proj projections are matmat over the batch (a weight row once per chunk), the gdn_step recurrence runs sequentially over positions (state is the same as the sequential path; the math is elementwise identical).
- gdn_
pair - Fused two-position forward (speculative verify): lane 1 commits into
state, lane 2 is tentative inscratch(ring + S move together). - kda_
forward - Forward one position through a KDA layer, advancing
state. - kda_
forward_ batch - Batched KDA forward (prefill-GEMM): projections as matmat over the chunk, the recurrence sequential per position — elementwise identical to the single-position path.
- short_
conv_ forward - Forward one position through a short-conv layer, advancing
state. - short_
conv_ forward_ batch - Batched short-conv forward (prefill-GEMM): in_proj/out_proj are matmat over the chunk (a weight row streamed once), the conv walks the positions in order — the chunk is contiguous, so the ring state is exactly the sequential path’s and the math is elementwise identical.
- short_
conv_ pair - Fused two-position forward (speculative verify). Lane 1 commits into
state; lane 2’s tentative ring goes intoscratch— swapped in on draft acceptance, dropped on rejection. LFM2 ships no MTP head, so this is exercised only by the pair-fusion micro-benchmark; kept correct. - vmf_
phase_ forward - Forward one position through a vmf_phase layer, advancing
state. - vmf_
phase_ pair - Fused two-position forward (speculative verify). Lane 1 commits into
state(its token is always committed); lane 2’s tentative state goes intoscratch— the caller swaps it in on draft acceptance and simply drops it on rejection.