Module pool 

Persistent worker pool for row-parallel matvecs.

Threads are spawned once and spin-then-park between calls — vmfcore measured spawn-per-matvec at ~+27% decode cost versus a persistent pool. Parallelism is by disjoint row ranges, so results are bit-identical to the serial path (each row’s dot product is computed the same way).

Dispatch is a single shared job slot + atomic epoch (roadmap §3 P0): the caller publishes one pointer, bumps the epoch and JOINS THE WORK as the extra worker instead of blocking on a latch. The previous design allocated an Arc<Latch> and pushed a message into every worker’s mpsc channel for every matvec (~200 dispatches/token) — with decode-grade matvecs that synchronization was its own budget. Workers spin for CMF_POOL_SPIN iterations before parking. Default 4000: at ~39 dispatches/token, park-immediately pays the unpark syscall on every worker for every dispatch — measured on an M4 (interleaved A/B, current epoch dispatch + parked-flag design): Qwen-0.5B q8 decode 101→115 tok/s, q4t 117→149, the 50M bench model 549→954 at spin=4000 vs spin=0. An early measurement that showed spinning LOSING (−25% on q8) predates the parked-flag skip and the multi-matrix dispatch cuts; it no longer reproduces. Over-spinning still hurts (200k: −15% vs 4k — spinners steal the caller’s serial cycles), so the budget stays bounded. CMF_POOL_SPIN=0 restores park-immediately for share-the-box serving.

CMF_THREADS env: 0/1 = serial, N = worker count (default: available_parallelism − 1, capped at 8).

Structs

Pool

Persistent thread pool: shared job slot, epoch dispatch, caller participation.

Statics

FORCED_THREADS

Embedder override for the pool size (C ABI cortiq_set_threads): 0 = unset, consult CMF_THREADS / topology as before. Read once at pool construction, so set it before the load.

WORKER_TIDS

Kernel thread ids of the CURRENT pool’s workers (Android/Linux) — what ADPF’s PerformanceHintManager needs to attribute work to the governor. Refilled on every pool construction; empty elsewhere.

Functions

dispatch_count

Total pool jobs published since process start (all pools).

matvec_rows

Row-parallel dense matvec: out[o] = Σ_j w[o·in + j]·x[j]. Bit-identical to the serial loop (row order does not change math).

matvec_rows2

Two-input row matvec: one pass over the weight rows serves BOTH inputs — CPU decode is memory-bound, so the second position costs a fraction of the first (this is where MTP speculative verify wins). Per-output accumulation order matches the single-input path exactly → bit-identical results.