Skip to main content

RMS_BLOCK_DEFAULT

memra_engine

Static RMS_BLOCK_DEFAULT 

Source 
pub static RMS_BLOCK_DEFAULT: AtomicU32
Expand description

Per-model rms_norm block size (per-model numeric-config law: the per-thread partial-sum split changes with blockDim -> different FP order -> battery-arbitrated per model). qwen keeps the shipped 256; gemma4 adopts 1024 (single-row 2816-col norms are one-block latency-bound at 256 threads — 7us/launch measured).