Skip to main content

Module linear_core

Module linear_core 

Source
Expand description

Linear-attention cores, selected by arch.linear_core.kind (descriptor-driven operators — Patent 15 claim 8).

Two tracks (owner decision 2026-07-04):

  • gated_delta_net — the faithful vendor operator (Qwen3.5 / Qwen3-Next). Default for models that ship GDN weights: conversion carries the tensors 1:1 and needs no training. Port of the validated gated_delta_net (vmfcore/rust/src/forward.rs) against the numpy/torch oracle (vmfcore/gdn_layer.py).

  • vmf_phase — the canonical core: token carries a phase θ; kernel φ(θ) = [cos θ; sin θ] gives a linear factorization; the condensate is a recurrent state S[head][p2, dv] with decay exp(−exp(A_log)). Noise-robust and simpler than vendor recurrences. Exotic operators are folded onto it at CONVERT time (--linear-core vmf_phase) and quality is restored by the offline heal — the research track and the production mechanism for Patent-15 skills (mask→heal→compress).

Both cores implement the same contract: *_forward (one position, advances the state) and *_pair (fused two positions; lane 1 commits, lane 2 is tentative in scratch for speculative verify). State lives in the layer’s linear_state: Vec<f64> and is resized lazily by the core itself.

Structs§

GdnCfg
GdnWeights
Weights of one GatedDeltaNet layer (model.layers.{i}.linear_attn.*, names 1:1 with the source model — no fold, no training).
VmfPhaseCfg
VmfPhaseWeights
Weights of one vmf_phase layer (model.layers.{i}.vmf_attn.*).

Functions§

gdn_forward
Forward one position through a GatedDeltaNet layer, advancing state.
gdn_forward_batch
Batched GDN forward (prefill-GEMM): the qkv/z/a/b and out_proj projections are matmat over the batch (a weight row once per chunk), the gdn_step recurrence runs sequentially over positions (state is the same as the sequential path; the math is elementwise identical).
gdn_pair
Fused two-position forward (speculative verify): lane 1 commits into state, lane 2 is tentative in scratch (ring + S move together).
vmf_phase_forward
Forward one position through a vmf_phase layer, advancing state.
vmf_phase_pair
Fused two-position forward (speculative verify). Lane 1 commits into state (its token is always committed); lane 2’s tentative state goes into scratch — the caller swaps it in on draft acceptance and simply drops it on rejection.