Struct Lightning 

pub struct Lightning {
    pub n_head: usize,
    pub head_dim: usize,
    pub decay: LightningDecay,
    pub qkv: WeightMatrix,
    pub gate: WeightMatrix,
    pub norm: Vec<f32>,
    pub out_proj: WeightMatrix,
}

One lightning layer’s weights, and the two counts that size them.

Fields

n_head: usize

Query heads. Uniform across the file: minimax-01.cpp:44-51 sizes every recurrent layer from n_head and n_embd_head_k.

head_dim: usize

n_embd_head_k, which llama-hparams.cpp:249-253 also uses as n_embd_head_la to size the recurrent state.

decay: LightningDecay

This layer’s decay, a function of the layer index and the head count alone.

qkv: WeightMatrix

blk.N.attn_qkv.weight, [3 * n_head * head_dim, n_embd], HEAD-major (see the module docs).

gate: WeightMatrix

blk.N.attn_gate.weight, [n_head * head_dim, n_embd].

norm: Vec<f32>

blk.N.attn_norm_2.weight, [n_head * head_dim].

out_proj: WeightMatrix

blk.N.attn_output.weight, [n_embd, n_head * head_dim]. The same tensor NAME a full-attention layer of the same file uses (:53), which is why it is loaded here rather than left on AttnWeights::o_proj: the tail applies it after the gate, not where the attention tail would.

Implementations

impl Lightning

pub fn state_len(n_head: usize, head_dim: usize) -> usize

Rows the per-head state needs: head_dim * head_dim per head.

llama-hparams.cpp:253 spells the same product n_embd_head_la * n_embd_head_la * n_head().

pub fn load( file: &impl TensorSource, layer: usize, n_layer: usize, n_head: usize, head_dim: usize, hidden_dim: usize, ) -> Result<Self, LoadError>

Loads layer layer’s four tensors and checks them against minimax-01.cpp:44-53’s shapes.

n_layer is the LOGICAL layer count the graph loops over, because the decay scale is 1 - il / (n_layer - 1) and nothing in the file carries it.

pub fn zero_state(&self) -> RecurrentState

A fresh sequence’s state: one head_dim x head_dim KV per head and no convolution window.

pub fn forward_rows( &self, normed: &[f32], rows: usize, state: &mut RecurrentState, rms_eps: f32, ) -> Vec<f32>

rows consecutive tokens of ONE sequence (normed is [rows][n_embd]) through the block, advancing state in place.

pub fn forward_row(&self, x: &[f32], state: &mut [f32], eps: f32) -> Vec<f32>

One token through the block, advancing state in place.

NOT called forward_token: that name is reserved for the entry point that enters the CPU worker pool, and engine::entry fails the build for any other declaration of it. forward_row is what the other recurrent blocks call the same thing.

x is the layer input AFTER attn_norm, as the graph has it (:308), and the return is what joins the residual – wo applied, the gate and the norm already inside.