pub fn with_row_exact<R>(f: impl FnOnce() -> R) -> RExpand description
Runs f with row-exact products on this thread: quantized and gathered products of up
to ROW_INDEPENDENT_ROWS rows (tokens, for gathers) evaluated or built inside give
each row the bits of its one-row product, through one matrix-vector product per row
(or nuvec’s few-row kernels with the same arithmetic). Outside it they route as MLX
does, whose multi-vector kernels are faster from 2 rows on but give a row other bits
than its one-row product. Speculative verification runs in this mode, and so
can batched serving that must equal each request run alone.