Module threads 

CPU worker-pool policy, shared by ferrox (CLI) and ferrox-server.

Three things this module exists to control, all of which were measured to matter far more than any kernel change on Apple Silicon:

1. Thread count. available_parallelism() on an M2 Pro reports 10 (6 performance + 4 efficiency cores). Splitting a decode GEMV across all 10 makes every fork-join wait on the slowest E-core slice. llama.cpp defaults to hw.perflevel0.physicalcpu for exactly this reason (common_cpu_get_num_math), and collapses when forced above it – 346 -> 176 tok/s on SmolLM2-135M Q8_0 going from -t 4 to -t 10. So default to the performance-core count, not the logical-core count.

2. Thread QoS. macOS schedules threads onto E-cores based on their Quality-of-Service class, and QoS is inherited from whichever thread spawned them. Rayon builds its global pool lazily, on first use – which inside ferrox-server is a Tokio spawn_blocking task, not the main thread. If that blocking thread carries a demoted QoS, every rayon worker inherits it and the whole matvec runs on efficiency cores. init_cpu_pool pins the workers to USER_INTERACTIVE explicitly so the pool’s placement does not depend on who happened to touch rayon first.

3. SMT siblings. The same argument as (1), for the other kind of fake core. On a 16C/32T host available_parallelism() reports 32, so an auto-sized pool puts two workers on every physical core. MoE decode is memory-bandwidth-bound: a sibling adds no bandwidth, contends for the same core’s load ports, and turns a spin barrier into a livelock-grade tax once the pool is oversubscribed. So the non-macOS width comes from physical_core_count, which deduplicates thread_siblings_list across this process’s affinity mask; sysfs being unreadable degrades to the available_parallelism answer rather than failing.

Structs

CpuPoolPlan

A sized worker pool: how many workers, which logical CPU each pins to, and the CPU set aside for a coordinator, if any.

CpuTopology

The SMT layout of the logical CPUs this process may run on: for each allowed CPU, in ascending id order, the set of CPUs sharing its physical core.

Constants

SYSFS_CPU_ROOT

Where Linux publishes per-CPU topology. CpuTopology::detect reads {root}/cpu{n}/topology/thread_siblings_list under this directory.

Functions

clamp_intra_op_threads

The intra-op width a second thread pool may still use once plan has claimed its cores: physical_cores - workers - coordinator - 1, clamped into 1..=configured. The trailing -1 is the calling thread itself, which is running the surrounding forward.

current_qos_name

The calling thread’s macOS QoS class, as a human-readable name. None off macOS, where the concept does not exist.

for_each_chunk_init

Chunk-parallel sibling of for_each_row; chunks are never split.

for_each_row

One output row per slot; parallel when should_parallelize says the matvec is big enough to pay for the fork-join. Rows are never split.

init_cpu_pool

Builds the global rayon pool with an explicit width and an explicit QoS, so neither depends on which thread first touched rayon. Safe to call more than once and from either binary; a pool that already exists is left alone.

parse_thread_siblings_list

Parse one thread_siblings_list line into the logical CPU ids it names.

perf_core_count

Number of performance cores, which is the useful width for a decode GEMV. On macOS this is hw.perflevel0.physicalcpu (llama.cpp reads the same key). Elsewhere it is physical_core_count — one logical CPU per physical core inside this process’s affinity mask — and only a host whose sysfs topology is unreadable falls all the way back to available_parallelism.

physical_core_count

How many physical cores this process may run on.

physical_core_cpus

physical_core_cpus_in against this host’s real topology.

physical_core_cpus_in

One logical CPU per physical core, restricted to topology.

plan_cpu_pool

plan_cpu_pool_in against this host’s real topology.

plan_cpu_pool_in

Size a pinned pool, optionally reserving a core for a coordinator thread (the one that polls the device doorbell and drives the pool).

process_affinity_cpus

The logical CPUs this process may actually run on, ascending.

resolve_cpu_threads

How many rayon workers to run: FERROX_CPU_THREADS, else RAYON_NUM_THREADS, else perf_core_count.

resolve_threads_and_affinity

resolve_threads_and_affinity_in against this host’s real topology.

resolve_threads_and_affinity_in

(num_threads, core_ids) for a pinned worker pool.

should_parallelize

Prefer serial when fork-join overhead exceeds the matvec work. ~256k element-ops matches the previous prefer_serial_matvec gate.