Expand description
Parallel cold build of the derived read-model (ADR-107/ADR-108).
Native shape of services/parallel_build.py + parallel_merge.py: the
per-document work (parse, classify, index row, token vectors, live/scope
projection) fans out across rayon workers as compact fragments; the
parent merge runs only the cross-document steps (graph resolution,
inbound fill, portfolio) — in sorted-path order, so the store bytes are
worker-count-invariant by construction (no pickling boundary exists
in-process; ADR-114).
Correctness never depends on the parallel rung: below the file-count
threshold, on a 1–2 core box, or on ANY worker fault (a panic in a
fragment task — exercised by DECIDED_PARALLEL_BUILD_FAULT), the build
falls back to the authoritative serial floor, whose partial results are
never written.
Structs§
- Build
Stats - Per-phase cold-build timings for the
DECIDED_TIMINGscorecard line.
Constants§
- DEFAULT_
MIN_ PARALLEL_ FILES - Below this file count the fan-out’s coordination overhead outweighs the win, so the cold build stays on the serial floor (the oracle’s measured crossover, kept for contract fidelity).
Functions§
- build_
derived_ index_ parallel - Build the derived read-model with a parallel parse AND per-doc derive —
byte-identical to
derived::build_derived_indexfor any worker count. - emit_
build_ timing - Write the cold-build scorecard line to stderr when
DECIDED_TIMINGis set — env-gated, stderr-only, byte-shaped like the oracle’s (ADR-107). - parallel_
parse_ paths - The paths type alias for the freshness tracker’s explicit-list parse (INDEX-PLAN B6): parse a known path list through the one true per-file path, parallel when it pays; entries in list order.