Expand description
Pinned, bounded Rayon pools for CPU-bound work submitted from shards.
Shard cores and compute cores are disjoint on purpose. A shard core spends its time dispatching and awaiting; a compute core spends its time saturated. Running both on one core makes each worse, and the split is what keeps a long computation from adding latency to unrelated keys.
There is a pool per memory node rather than one pool for the machine. A shard
that ships a closure and its captured data to a worker on another node pays
the interconnect on every task, which is exactly the cost that pinning the
shard was meant to avoid, so OffloadPools builds the set a Plan asks
for, and each shard submits to the one local to itself.
Structsยง
- Offload
Pools - The pools a
Plancalls for: one per memory node, each bound to its own. - Offload
Stats - Rayon
Offload - A Rayon pool whose workers are bound to dedicated CPUs on one memory node, with a bound on how many tasks may be outstanding at once.