pub fn q4tp_matvec(
model: &Arc<CmfModel>,
idx: usize,
xs: &[f32],
rows: usize,
cols: usize,
out: &mut [f32],
) -> boolExpand description
Single-token q4tp matvec on the device — the lm_head class. Through the DEDICATED matvec kernel: the batched GEMM at b=1 measured 11.73 ms against the host’s 9.51 on the release head, so the route that was supposed to save eleven milliseconds a token lost its own probe instead.