Skip to main content

Module forward

Module forward 

Source
Expand description

Dense forward pass (Stage-1, all f32, prefill of T tokens, batch=1). Matches llama.cpp qwen3 graph: embed → per layer {RMSNorm, QKV, QK-norm, RoPE, SDPA, O, residual, RMSNorm, SwiGLU, residual} → output_norm → lm_head.

Activation layout: x is [n_embd, T] but we store it row-major-per-token as [T, n_embd] (token t at offset t*n_embd) so cuBLASLt linear (m=T tokens, in=n_embd) works directly.

Structs§

PrimeGateVerdict

Enums§

PrimeGateClass
Gate #46 verdict: BATCHED-PRIME last-position logits vs the TOKENWISE-PRIME reference.

Functions§

argmax
argmax helper.
prime_gate_verdict
top2
top-1/top-2 (id, value) pairs — the greedy near-tie margin is v1 - v2.