pub const GPU_MIN_ROWS: usize = 65_536;
Default row threshold: the GPU takes only larger matrices (lm_head class). Below it, the dispatch/readback cost does not pay off on unified memory.