Struct FullAttnLayer 

pub struct FullAttnLayer {
    pub wq: GpuTensor,
    pub wk: GpuTensor,
    pub wv: GpuTensor,
    pub wo: GpuTensor,
    pub q_norm: GpuTensor,
    pub k_norm: GpuTensor,
    pub attn_gate: Option<GpuTensor>,
}

Fields

wq: GpuTensor

wk: GpuTensor

wv: GpuTensor

wo: GpuTensor

q_norm: GpuTensor

k_norm: GpuTensor

attn_gate: Option<GpuTensor>

step35-class SEPARATE head-wise attention gate: blk.N.attn_gate.weight [n_embd, n_head_l] where n_head_l is this layer’s query-head count (64 full / 96 SWA on Step-3.7-Flash, so the width VARIES per layer). Produces one pre-sigmoid scalar per head from the post-attn_norm hidden state; the forward broadcasts sigmoid(gate) over head_dim and multiplies attn_out before wo (upstream step35.cpp:267-285).

None for every other arch. Do NOT confuse with LinearAttnLayer::wqkv_gate, which reads the SAME tensor name on qwen35’s SSM layers but is a different mechanism (a full-width z-gate, not a per-head scalar), nor with the qwen35 FUSED gate packed inside wq that ModelConfig::attn_out_gate() / q_gate_split handle.