Module inference 

CPU forward-pass primitives: RMS-norm, SiLU, sparse SwiGLU FFN.

Functions

add_rmsnorm_fused_into

Fused residual addition and RMSNorm into a caller-owned buffer: h += delta, then out = rms_norm(h, weight). This avoids a separate pass over h for the residual addition and sum of squares.

gelu_tanh

SiLU activation function. tanh-approximated GELU (Gemma’s GeGLU gate; HF gelu_pytorch_tanh).

rms_norm

RMS normalization with explicit weight semantics.

rms_norm_into

rms_norm writing into a caller-owned buffer — the decode hot path calls this twice per layer; the returning variant allocated each time.

silu

sparse_ffn_forward

Sparse SwiGLU FFN: compute only active_indices neurons.

sparse_ffn_forward_pair

Fused two-position sparse FFN: gate/up/down weight rows are streamed from memory ONCE for both positions. Bit-identical to two single calls (per-position accumulation order is unchanged).