Expand description
Persistent worker pool for row-parallel matvecs.
Threads are spawned once and parked between calls — vmfcore measured spawn-per-matvec at ~+27% decode cost versus a persistent pool. Parallelism is by disjoint row ranges, so results are bit-identical to the serial path (each row’s dot product is computed the same way).
CMF_THREADS env: 0/1 = serial, N = worker count
(default: available_parallelism − 1, capped at 8).
Structs§
- Pool
- Persistent thread pool. Workers park on a channel between jobs.
Functions§
- matvec_
rows - Row-parallel dense matvec:
out[o] = Σ_j w[o·in + j]·x[j]. Bit-identical to the serial loop (row order does not change math). - matvec_
rows2 - Two-input row matvec: one pass over the weight rows serves BOTH inputs — CPU decode is memory-bound, so the second position costs a fraction of the first (this is where MTP speculative verify wins). Per-output accumulation order matches the single-input path exactly → bit-identical results.