Expand description
Phase 15.4: the bounded worker pool for parallel ingest. Only meaningful with
the non-default parallel feature (which implies field); without it the
module is a zero-sized marker and every ingest runs serially.
Bounded CPU parallelism for the naturally independent ingest work
(Phase 15.4).
The pool is compiled only with the non-default parallel feature (which
implies field). Without that feature WorkerPool is a zero-sized marker
that is never constructed, and every ingest site takes its serial path; the
exact core therefore keeps no new dependency and no behavior change.
§The one architectural rule
A worker may only run a pure map over immutable inputs. It must never
touch a FieldStore, its Rc-based
IoCounters, a derived-cache memo, or the disk.
The store’s single-threaded accounting is the fuse that keeps parallelism
honest: the parallel phase computes owned canonical bytes and NodeIds,
and a serial fold merges them in physical order.
Determinism is therefore structural, not a tuning property. Indexed
par_iter().collect() preserves physical order, and every order-dependent
gate (the running MAX_TOTAL_DECODED / MAX_OBJSTM* caps, the existence
probes and counters) is replayed in that same serial fold — so the parallel
path yields byte-identical nodes, ids, index entries, and limits enforcement
to the serial path.
Structs§
- Worker
Pool - Without the
parallelfeature this is a zero-sized marker that is never constructed; the ingest sites keep their serial path (poolis alwaysNone).