vole-document 0.1.0-alpha.39

# Roadmap

Open work that is `PROPOSED` or not yet measured. Every item carries an explicit
prior that it may also lose, because every measured attempt so far has.

## Phase-13 open proposals

Phase 13 closes the remaining Phase-12 proposals and the last open gate. Plan:
[phase-13-plan.md](../phases/phase-13-plan.md).

| Item | Question | Prior | Outcome |
|---|---|---|---|
| PDF `/Length`/revision proceduralization as a *size* mechanism | Does persisting revision/xref structural redundancy beat the RAW/`BYTE_RANS`/generic ladders once framing is charged? (Phase 11 persists revisions as *observation* nodes only, never as a size candidate.) | Expected negative | **Measured 13.1 — recorded negative** (ADR-0036): byte-exact, 0 wins vs the ladder and 0 vs generic |
| PDF grammar/templates | Can a bounded grammar/template candidate that pays its definition cost beat a whole-file order-0 lane? | Expected negative | **Measured 13.2 — mixed** (ADR-0037): byte-exact; best VOLE lane on 4/28 (+34,505 B auto-winner) but 0 wins vs generic |
| ODT adapter | ODT is ODF packaged in ZIP, so it reuses the 12.x ZIP layer (not the OPC graph) with an OpenDocument content model. Does a fourth format enter exactly and queryably? | Unknown | **Measured 13.3 — ADOPTED** (ADR-0038): byte-exact (`len`+SHA-256+`cmp`) and queryable after source + descriptor deletion in a fresh process; missing/malformed manifest declines typed with exactness preserved |
| Byte-level partial-materialization checkpoints | The random-access `view`/seek lane was measured (ADR-0018/0019), but literal byte-level checkpoint records were never built. Do they pay their framing cost? | Unknown | **Measured 13.4 — recorded negative** (ADR-0039): byte-exact and advisory (lying/corrupt checkpoints rejected, reader falls back), but the per-op table is redundant with the observation index and *larger* than the index record it replaces (+259 to +1,159 B per byte-range query, identical op work) |
| `N5` package-index-only gate | Is the small-document win reproducible by `unzip -p` + `substr` at the same boundary? The A3/A4 ladder rungs are evidence against it; the mechanical check remains open. | Evidence suggests the field adds value | **Measured 13.5 — falsified** (gate closed): the literal mechanical control (`zipfile.read` + byte substring) answers **0/72** structural selectors while the field answers **72/72** (`2026-10-07-phase13-n5-21948bd`) |

## Phase-15 outcomes

Phase 15 is a performance programme: repair the measurement, then measure the
frontier. Plan: [phase-15-plan.md](../phases/phase-15-plan.md); results:
[phase-15-results.md](../phases/phase-15-results.md).

| Item | Question | Outcome |
|---|---|---|
| 15.1 Court repair | Re-run the frozen `real100-v1` court on the **release** binary with the storage universes split | **Repaired** (measurement): coverage/exactness identical to the debug court (deterministic); VOLE holds `pdf`/`text_repeat` + `docx`/`table`, loses the rest; persistent `2,667,668,262 B` / descriptor `1,704,524,849 B` / A1 db `2,891,784,192 B` (whole-population `du -sb` aggregates; the store figure is method-inflated — ADR-0049); 3 of 5 `>100 MiB` PDFs still fail at `encode` (`2026-10-07-real100-release-baseline-866f489`) |
| 15.2 Resident runtime | Does a resident session + `observe-batch` beat the cold lane? | **Measured — recorded negative/partial** (ADR-0042): wins only below ~1 MiB; cold 7 ms vs resident 9 ms aggregate; mechanism = lost `narrow_probe` short-circuit |
| 15.3 Packed seed store | Does a packed `fieldpack` backend beat one-file-per-node? | **Measured — adopted (seed namespace only)** (ADR-0043, corrected by ADR-0049): file/directory count 0.009× / 111× fewer at **byte parity** (file bytes 1.007×), latency parity, identity unchanged; the earlier "persistent bytes 0.719×" was a `du -sb` directory-inode artifact |
| 15.4 Parallel ingest | Does a bounded worker pool speed up ingest deterministically? | **Measured — implemented, non-default** (ADR-0044): speed-neutral (1.00× at 2/4/8); determinism positive (10/10 across every worker count) |
| 15.5 DEFLATE ablation | Which correct safe-Rust inflate is fastest? | **Measured — recommendation** (ADR-0045): `miniz-simd` enabled (1.11×); `zlib-rs` meets the bar (1.58×, RSS-neutral, byte-identical) and is recommended but **not adopted**; `zune-inflate` disqualified (255 mismatches) |
| 15.6 Adaptive promotion | Does an adaptive promotion governor beat SQLite on diversity/revision? | **Measured — recorded negative** (ADR-0046): all three pre-registered falsifiers fire; mechanism ships opt-in/default-off |
| 15.7 Durable cross-root derivations | Does canonical derived-work identity satisfy `N3`? | **Measured — recorded negative** (ADR-0047): `N3` violated again, 0 cross-member reuse; no Rust change made |
| 15.8 CUDA batch lane | Does a GPU batch inflate lane pay? | **Deferred, not measured** (ADR-0048): gate unopened; the pinned Docker lanes cannot see the GPU; `nvCOMP` proprietary; Docker-only evidence required |

## Phase-16 outcomes

Phase 16 follows Phase 15: adopt the recommended backend, finish the storage
court, fix the large-PDF pathology, continue the residency line, and test
whether SQLite loses under an equal capability contract. Results:
[phase-16-results.md](../phases/phase-16-results.md); ADRs 0049–0050.

| Item | Question | Outcome |
|---|---|---|
| 16.1 `zlib-rs` adoption | Adopt the backend 15.5 only recommended; what is the **end-to-end** effect? | **Adopted** (ADR-0045 fulfilled): one `src/field/inflate.rs` helper owns every inflate; byte-identical (**2,075 members / 17 docs**); `field-ingest` **0.917×** (~8 % faster), `encode` `0.996×` (control), peak RSS `1.000×` (`2026-10-08-phase16-zlib-ebb6636`) |
| 16.2 Full packed court | Does `--packed` beat the A1 SQLite db on the full population? | **Court complete** (95 common-success docs; `fs`/packed field id identical 97/97). Court `2026-10-08-phase16-packed-full-0b21928`; its `du -sb` byte numbers are **superseded by 16.6** |
| 16.3 Large-PDF encode pathology | Why do the `>100 MiB` PDFs fail at `encode`? | **Fixed** (ADR-0041 extended): `propose_rle` pre-allocated ~16× input before its decline; two streaming O(1)-memory passes. `nasa-pdf-0001` now completes byte-exactly; peak/input **17.7× → 8.9×**; **77/77** byte-identical (`.voldoc` SHA-256); `0002`/`0003` remain a **wall** limit, not memory (`2026-10-08-phase16-largepdf-ce8af8f`) |
| 16.4 Resident session + `narrow_probe` | Does giving the session the cold path's short-circuit make residency pay? | **Recorded negative** (ADR-0042 extended): probe works (0 descriptor bytes, ~25 µs, 90/0 equality) but no class flips — cold 6 ms vs resident 9 ms; one-time full `Field::open` dominates. Lever isolated: **lazy session open** (`2026-10-08-phase16-resident-probe-5d331f2`) |
| 16.5 Contract-equivalent court | Does SQLite lose under an **equal capability contract** (C0–C5)? | **Recorded negative for VOLE** (ADR-0050): equality at C0–C3; **VOLE declines C4/C5** (no revision surface); SQLite builds **~10×** faster, warm **~1.47×** faster, +**~3 %** bytes C0 → C4. VOLE's lone edge is storage (`2026-10-08-phase16-contract-45d2c0e`) |
| 16.6 Storage correction | Is `du -sb` the right unit for a one-file-per-node store vs a `.db`? | **Corrected** (ADR-0049): `du -sb` counted 4096 B/dir, inflating `fs` **52 %**. File-bytes-only: `fs`/SQLite **1.377× → 0.906×**, packed/SQLite **0.921× → 0.914×**, packed/`fs` **0.669× → 1.009×**; **"VOLE is 1.377× SQLite" / "packed closes the gap" refuted**; packed's win is file/directory count (`2026-10-08-phase16-storage-correction-2978e1d`) |

### Recorded open question (Phase 16)

The Phase-16.5/16.6 result — **SQLite does not lose under an equal capability
contract**, VOLE's storage edge corrects to **~0.9×**, and VOLE cannot answer
revision lineage — raises an architectural question the user asked to **record,
not resolve**: should the conceptual invention keep competing with SQLite, or
**use an embedded DB as part of its physical substrate** for materialized
observation state? **No switch is decided.** The measurements that would settle
it are enumerated in [ADR-0050](../adr/0050-sqlite-as-substrate-question.md).

## Phase-17 outcomes

Phase 17 follows Phase 16: it removes the search overhead from the runtime build
path and adds the revision-lineage surface Phase 16 recorded as missing, then
re-runs the contract court. Results:
[phase-17-results.md](../phases/phase-17-results.md); ADRs 0051–0052.

| Item | Question | Outcome |
|---|---|---|
| 17.1 Direct source → field ingestion | Can the runtime build path skip the candidate portfolio search without changing exactness or the observation surface? | **Adopted** (ADR-0051): `field-build --profile runtime` fixes `RAW` (`candidates_evaluated == 1`) and builds the authority + field in one process; build wall sum **7,928 → 3,878 ms (2.04×)**, peak RSS median **51,792 → 15,228 KB (3.4× smaller)**, authority **1.007×**, exactness **9/9**, observations **0 divergent**. Recorded caveat: the PDF metadata projection's `object_count`/`graph_ops` are encoder-dependent (`2026-10-08-phase17-direct-field-d83ddba`) |
| 17.2 Revision-lineage surface | Does adding the missing revision surface close C4/C5? | **Surface added; contract still open** (ADR-0052): `Selector::Revisions`/`Representation::Lineage` computed once at ingest, indexed, **O(depth)**, typed decline for non-PDF; the court re-run satisfies **C0–C3** but **C4/C5 still do not close** because the contract's C4 tuple is the corpus family/member/head — external metadata PDF bytes cannot derive (`different observable`) (`2026-10-08-phase17-revision-1179386`) |

### Recorded open question (Phase 17)

The C4 finding re-frames the open question: it is a **contract-definition**
question, not a missing surface. Either the contract's C4 tuple is the right
definition — in which case a single-document exact field is the wrong tool for C4
by construction — or VOLE should **ingest corpus/revision-family metadata as an
explicit external input** and answer the tuple as a derived observation. **No
decision is taken**; see [ADR-0052](../adr/0052-revision-lineage-surface.md).
This complements, and does not supersede, the Phase-16 substrate question
([ADR-0050](../adr/0050-sqlite-as-substrate-question.md)).

## Phase-18 outcomes

Phase 18 is a **build-cost programme**: starting from Phase 17's direct
`field-build`, it removes the remaining unnecessary work from the runtime build,
one mechanism at a time, and re-measures the equal-contract court. Results:
[phase-18-results.md](../phases/phase-18-results.md); ADR-0053.

| Item | Question | Outcome |
|---|---|---|
| 18.1 Contract court with the direct build | Re-run the 12-document C0–C5 court changing only VOLE's build step to `field-build --profile runtime`; what is the composable gap, and what is C4? | **Recorded** — build **10.74× → 7.24×** (the 17.1 2.04× and the 10.74× two-step gap are **not composable**); storage **0.49×**; cold sum **779 vs 772 ms (tie)**; warm **1.33×**. C4 splits: **C4a** document-native lineage is VOLE's answer (4/4 PDFs; typed unsupported for 8/8 docx/epub) and byte-derivable from the baseline's retained blob (a work-location difference); **C4b** corpus/external lineage is harness-supplied metadata (baseline 12/12, VOLE 0/12). `2026-10-08-phase18-contract-direct-f2a34a5` |
| 18.2 One-pass direct build | Can the source → authority → source round trip be removed without changing bytes? | **Adopted** — `ingest_verified` verifies against the caller's source; `ingest_pdf_direct`/`ingest_package_direct` scan the original input; `encode`/`field-ingest` unchanged. Wall **−6.1%**, peak RSS median **−28.8%**; contract gap held at **7.21×**; exactness 9/9 + 12/12. `2026-10-08-phase18-onepass-61b11350` |
| 18.3 Observation index before the single serialize | Can the redundant parse+re-serialize of the authority be removed? | **Adopted** — `Descriptor::with_observation_index` is a pure method; the index is attached before the court's single serialize; `encode` still emits no index and the one-pass authority is byte-identical to `with_observation_index(encode_with(…))`. Wall-neutral; large-doc RSS **−14–16%**. Isolation: `nist-pdf-0017` is 9425/11,605 ms (81%), 6842 files, 9.40 s on the bind mount vs 1.39 s `/tmp`. `2026-10-08-phase18-inindex-b7046f0` |
| 18.4 Packed as an ingest-write optimization | Does the packed file-count collapse also collapse the build term? | **Falsified** — packed 11,312 ms vs same-run fs-direct 11,715 ms vs SQLite 1,597 ms (7.08× vs 7.34×); files collapse but `insert` syncs **per node** (6792 `fdatasync` vs 6842 `fsync`); the term is sync **latency**, not file count. `2026-10-08-phase18-contract-packed-fb23021` |
| 18.5 Batched packed-store durability | If the term is a per-node sync, can the packed store sync once per segment — and can `observe-batch` serve it? | **Adopted** (ADR-0053) — `SyncPolicy { Batch (default), Each }`; `put_field` flushes before publishing a manifest; prefix recovery + no partial node + re-hash gate; `observe-batch --packed` now works. `nist-pdf-0017` wall **9392 → 1415 ms**, `fdatasync` **6792 → 0**; 12-doc contract build **1552 vs 1893 ms = 0.82×** (VOLE builds **~1.22× faster**); storage **0.53×**; warm **1.09×**; exactness **12/12**. **[SUPERSEDED as competitor statements by [Phase 22.1](../phases/phase-22-results.md): against the tuned equal-contract envelope storage is **0.762× `full` / 0.805× `adaptive`**, build **0.219× `full`**, cold **0.812× `full`**, warm a **loss** (**1.211× `full`**).]** `2026-10-08-phase18-batched-sync-14a7e6f`, `2026-10-08-phase18-contract-packed-14a7e6f` |

The build position recorded by Phase 16.5 as VOLE's decisive loss now reads: on the
equal-contract 12-document subset VOLE **builds ~1.22× faster** than the
source-retaining SQLite baseline, **stores ~0.53×** the bytes, **ties** on cold
queries, and is **~1.09×** slower on the warm one-session lane. **[SUPERSEDED as
competitor statements by [Phase 22.1](../phases/phase-22-results.md): those
figures are against the *historical-control* configuration, whose contract-dead
trigram index inflated the baseline (≈+44% bytes). Against a Pareto-tuned
equal-contract SQLite, storage is **0.762× `full` / 0.805× `adaptive`** (not
0.53×), build **0.219× `full`**, cold **0.812× `full`**, and warm is a **loss**
(**1.211× `full`**, 95% CI 1.006–1.483).]** **C4 is not a
closed contract:** C4a is VOLE's genuine native-lineage observable, C4b remains
open because it is external metadata.

### Recorded open question (Phase 18)

The C4b half is a **contract-definition / external-input** question, unchanged by
the build-cost programme: either the C4 tuple (corpus family/member/head) is not a
single-document derivable fact — in which case a single-document exact field is the
wrong tool for C4b by construction — or VOLE should ingest corpus/revision-family
metadata as an **explicit external input** and answer it as a derived observation.
**No decision is taken;** this remains the Phase-17 question
([ADR-0052](../adr/0052-revision-lineage-surface.md)) and composes with the
Phase-16 substrate question ([ADR-0050](../adr/0050-sqlite-as-substrate-question.md)).

## Phase-19 outcomes

Phase 19 is a **measurement-discipline phase**: it replaces Phase 18's two
single-run point estimates with paired, interleaved, repeated measurements and a
fixed-seed bootstrap interval (19.1), extends the warm lane to N=100 (19.2), and
runs the new direct build over the full frozen `real100-v1` population (19.3).
Results: [phase-19-results.md](../phases/phase-19-results.md);
[ADR-0054](../adr/0054-repeatability-and-paired-measurement.md).

| Item | Question | Outcome |
|---|---|---|
| 19.1 Repeatability court (paired, interleaved, N=10) | Is the Phase-18.5 build ratio < 1.0 and the warm ratio > 1.0, once both lanes are measured repeatedly and interleaved? | **Build established < 1.0** — paired median **0.182** (95% CI **0.102–0.228**), geometric mean 0.203 (0.128–0.395), **11 win / 0 tie / 1 loss** (`nist-pdf-0017` at 4.440×). **Correction:** the ratio-of-sums estimator reads **~0.96**, not 0.82, because one heavy document dominates VOLE's total; the move is SQLite-lane host variance (1893 → 1582 ms), VOLE's sum reproducible (1545.5 vs 1552 ms). **Warm not resolved:** paired median **1.077** (0.809–1.391, includes 1.0), 5/1/6. Best-of-3 (min) vs median-of-10: build 0.977 → 0.963, warm 1.218 → 1.194. Exactness 12/12 both. `2026-10-08-phase19-repeat-d6c8c4c` |
| 19.2 High-N warm repeat (N=100) | Is the warm lane a resolved loss, resolved parity, or still undecided at high N? | **Not resolved under the median estimator, resolved under the geometric mean** — pooled median paired ratio **1.292** (95% CI **0.994–1.658**, includes 1.0); geometric mean **1.283** (95% CI **1.069–1.535**, excludes 1.0); pooled warm medians VOLE 2.25–2.32 ms vs SQLite 1.54–1.65 ms (~0.7 ms slower). A **modest real loss (~1.29×), not parity**; the N=10 estimate was under-sampled. Variance floor CV 5.4% (p90 9.4%); median-CI half-width ±0.332; MDE(80%) ≈0.474. Exactness 12/12; 480 envelopes, 0 mismatches. `2026-10-08-phase19-warm-6b66eab` |
| 19.3 Direct build over the full `real100-v1` | Is the new direct build robust across the whole frozen population, and where is its next limit? | **100/100 built, 100/100 exact** (pdf 60/60, docx 15/15, epub 25/25; rc histogram all 0), recovering the three docs the old two-step path failed (`nasa-pdf-0001`/`0002`/`0003`); wall median **64 ms**; peak RSS median 30.9 MiB / max 2342 MiB — **memory, not wall, is the next binding constraint**. Cold coverage text 92/100, metadata 98/100 (typed declines). vs the old path (file-size-corrected) median **61 ms** vs 2,095 ms (paired 0.083×) but **+5.3%** bytes. **Storage is population-dependent: ~0.96× SQLite on the full population, not the 0.53× of the contract subset.** `2026-10-08-phase19-real100-direct-954dbc2` |

The Phase-18 **0.82×** build and **1.09×** warm headlines are superseded as point
estimates: the build win stands (paired median 0.18, CI below 1.0) but its size is
estimator-dependent (~0.96× by ratio of sums), and the warm lane is a modest
~1.29× loss that the median estimator cannot separate from 1.0 at N=100 while the
geometric mean resolves it. **[SUPERSEDED as competitor statements by [Phase
22.1](../phases/phase-22-results.md): these were all read against the
historical-control configuration; against a tuned equal-contract envelope storage
is **0.762× `full`**, build **0.219× `full`**, cold **0.812× `full`**, warm a
**loss** (**1.211× `full`**).]**

## Phase-20 outcomes

Phase 20 is a **hardening-and-economics phase** that attacks, in risk order, the
four open problems Phase 19 left: prove (or bound) the packed store's durability
under faults, bound large-source memory, diagnose the warm loss, and decide C4b.
All four plan items are now **complete and sealed**. Results:
[phase-20-results.md](../phases/phase-20-results.md); plan:
[phase-20-plan.md](../phases/phase-20-plan.md) (complete); ADR-0055 and the
Phase-20.1 amendment to ADR-0053.

| Item | Question | Outcome |
|---|---|---|
| 20.1 Crash / power-cut fault-injection court | Does the packed store hold its recovery invariants across an arbitrary crash at every durability boundary, under both sync policies, with recovery exactly a prefix? | **Complete — 1,300 PASS / 0 FAIL / 0 CRITICAL** (families A 1,024, B 236, C 40; Batch 650/650, Each 650/650); `bad_hash` **0**; prefix violations **0**; whole-node re-hash gate rejected **212,812** nodes across **110** cases; corruption fails closed typed. **Scope:** proves ordering / no-partial-node / prefix recovery / fail-closed, **not** true power loss or torn rename (page cache survives SIGKILL; `write_atomic` never dir-fsyncs). `2026-10-08-phase20-crash-47acde7` |
| 20.2 Large-source memory architecture | Can peak memory be bounded for sources ≳1 GiB without changing exactness or the wire? | **Complete — adopted:** RSS/source **5.998× → 3.997×** (−33 %); `nasa-pdf-0001` **2342 → 1563 MiB**; 1 GiB synthetic **rc 137 @6113 MiB → rc 0 @4099 MiB** (≈1 GiB now fits, ~2 GiB headroom; cap boundary ~1.0 → ~1.5 GiB; **no cap raised**). Exactness **16/16**; descriptor SHA-256 identical **15/15**. Residual floor 4 copies (3 needs a wire-type change). `2026-10-08-phase20-memory-7b897ba` |
| 20.3 Warm heterogeneous query | Is the ~1.29× warm loss a closeable constant or durable? | **Complete — no change shipped, with proof.** Dominant term `Descriptor::parse` (~1.7 ns/B) + full `Field::open`; no safe lever (lazy/partial open cannot help this contract; no redundant re-read; relaxing a validation would weaken decoder authority). Paired N=100 before/after is a **NULL** (median 1.292 → 1.322; CI half-width ±0.311; MDE ≈0.44). Loss recorded **durable**. `2026-10-08-phase20-warm-5ed5957` |
| 20.4 C4b / `ExternalContext` | Can corpus/external lineage be admitted as an explicit typed layer beside the field without contaminating it? | **Complete — C4b closes** under an equal external input. Typed, removable `ExternalContext` at `<store>/external/<FieldId>`; `Basis::ExternalMetadata` (`exact=false`); plain `--metadata` byte-identical before/after attach/clear **12/12**; `materialize --exact` **12/12**; external query declines rc 6 after clear **12/12**; both lanes answer and tuples match **12/12**. ADR-0055. `2026-10-08-phase20-c4b-5ab2e76` |

The warm loss is now **diagnosed and durable**; the packed store's durability is
**adversarially attacked with its scope stated**; large sources up to ~1 GiB
**fit**; and **C4a and C4b are both answered**. The Phase-16 substrate question
([ADR-0050](../adr/0050-sqlite-as-substrate-question.md)) is unchanged and no
option of it is chosen.

## Phase-22 programme (22.1–22.7 complete)

Phase 22 is the **economic programme** synthesised from an external technical
review: it targets a better **capability/cost frontier** against the strongest
honest competitor, in a commercially relevant workload region, with bounded
statistics and no loss of accuracy or capability. **22.1 (the competitor
envelope), 22.2 (the compact-index profiling gate) and 22.3.0 (the fused-execution
measurement gate) are complete and sealed** — the programme is **complete**; no row
measurements**. Plan:
[phase-22-plan.md](../phases/phase-22-plan.md); results:
[phase-22-results.md](../phases/phase-22-results.md),
[phase-22-3-results.md](../phases/phase-22-3-results.md); the IP/presentation
distinction it depends on is [ADR-0056](../adr/0056-runtime-vs-research-reconstruction-programs.md).
Nothing below is measured beyond **22.1–22.3.0**; each remaining row is a **gate**
and its **prior** (a VOLE win is not presumed likely against the record). Order is
by dependency and risk.

| Subphase | Review | Question | Prior / gate |
|---|---|---|---|
| 22.1 Competitor envelope | P0 | Is there a strong competitor envelope, including an adaptive and a hybrid SQLite, before any frontier claim? | **Complete (Phase 22.1).** Six purpose-tuned SQLite configurations added, the Phase-18 baseline kept **unmodified** as the `hist` control; gate met (**no omitted capability**, durability compared explicitly). Against the tuned envelope: storage **0.762× `full` / 0.805× `adaptive`**, build **0.219× `full`** (0.194× `hist`), cold **0.812× `full`**, warm a **loss** (**1.211× `full`**, 95% CI 1.006–1.483); equivalence **1920/1920**, 0 mismatches; exactness 12/12 VOLE, 6/6 per doc per config. `2026-10-08-phase22-competitors-86d9312` |
| 22.2 Compact query-native directory / hot-cold layout | P1 | Can warm CPU/I/O/allocation fall without storage growth or weaker integrity? | **Complete (negative, Phase 22.2).** Decisive profiling gate **first, nothing shipped**. Warm session **open 55.5 % (`Descriptor::parse` 44.7 %) + loop 44.5 %**; probe **0.4 %**; index read+verify **10.5 %** (redundancy: 493 opens / 20 files, one depth-0 root leaf re-read 24–117×). A perfect selector directory removes ≤10.6 % (≈**0.13** shift) vs court **MDE ≈0.399** → **cannot be resolved**; it would also add persistent bytes. Before/after warm court vs tuned `full` is a **NULL** (1.211 → 1.293, overlapping CIs); no layout byte changed; exactness 12/12, 0 mismatches. Left for a **higher-resolution** court: in-session verified-node memo (0 persistent bytes, still sub-MDE); split `dispatch` (32 %). `2026-10-08-phase22-2-d81689c` |
| 22.3 Fused heterogeneous execution | P2 | Can a known batch be one dependency closure and one schedule? | **22.3.0 complete — negative (`SESSION-ALREADY-CAPTURES`, Phase 22.3).** Measurement-only gate, nothing shipped: an independent DOCX/EPUB batch executes **3.66–4.11×** more seed nodes than the shipping resident session (memo + derived cache already remove the redundancy); the only axis the session never dedupes is the **index/selector descent** (`idx_dedup` 1.00 for all 12 docs) — the **22.2 sub-MDE** candidate. A first `|U|`-based `CLEARS` was **withdrawn** (unsound metric). 22.3.1/22.3.2 not started. `2026-10-08-phase22-3-dup-ca9f29e` |
| 22.4 Unknown-query lifetime frontier | P3 | On a hidden schedule, which system wins the cumulative lifetime frontier? | **Complete (region split, `MIXED`).** Pre-registered hidden schedule (seed **220400**, len 24, undisclosed seeded proportions); all adaptation charged. Cumulative query ratio VOLE/`full` per equivalent observation: pooled 0.99–1.08 (**unresolved**, MDE ±0.29–0.36), **pdf 0.73–0.83 (resolved WIN)**, docx 0.88–1.00 (unresolved), **epub 1.50–1.82 (resolved LOSS)**; storage **0.468× (win)**; peak RSS **1.24–1.32 (loss)**; build unresolved. Exactness 12/12; 0 value mismatches; 102 divergent PDF page-text (by design). Seed-check (220477) reproduces the split. `2026-10-08-phase22-4-lifetime-b4225fe` |
| 22.5 Large/remote selective materialization | P4 | Can remote byte-range reads cut bytes transferred at competitive p95? | **Complete — negative under a labelled model (`LOSS`, bytes 3.15×).** Explicitly-labelled `remote-selective-v1` (no S3): real local byte counts + a real loopback HTTP range server/coalescing client. VOLE selective transfers **median 3.15× more bytes** than a page-level SQLite control (0 wins / 17 losses / 7 unresolved); exact byte-range reads tie on bytes and win modelled latency (unresolved); one large PDF 0.54×. Exactness 12/12; ranges verified 55/55 + 96/96. Control-sensitive (vs whole-`.db` VOLE wins bytes). `2026-10-08-phase22-5-remote-762c81c` |
| 22.6 Compact structural representation | P5 | Is an existing typed node representable more compactly at equal capability? | **Complete — no win (measurement-only).** Full `real100-v1` footprint 2.81 GB is **97.2 % exact `descriptor/`**; typed bytes (`seed`+`index`+`field`) are only **2.765 %**. Candidate encodings on actual node bytes (round-trip 100/100): content-id interning 11.2 MB, header-fold 9.4 MB, delta+varint 6.9 MB, string dict 5.3 MB; bitmap/rank **loses**. Best combined **1.166 % of footprint** < the 5 % bar. Compaction pays only in small structure-dense docs (~7.5 % of `<1 MiB`), byte-negligible at scale. `2026-10-08-phase22-6-compact-85eee7c` |
| 22.7 Agent end-to-end economic court | P6 | Does the whole document-agent workload cost ≥2× less per correct, grounded task? | **Complete — VOLE LOSS (gate not met).** Deterministic scripted agent (no LLM), 9 docs / 27 frozen tasks, pinned offline tokenizer. Cost per correct grounded task: VOLE **$0.001501** vs baseline **$0.001059** = **1.418×** (gate ≤0.5×) → **not met**; VOLE **16/27** correct+grounded vs **27/27** (PDF **0/9** heuristic text, no span; docx 7/9 find misses tables; **EPUB a token tie**). Pooled loss is coverage/grounding, not token economy. `2026-10-08-phase22-7-agent-ee8723b` |
| — XLSX/PPTX economic validation | P7 | Is XLSX/PPTX evaluated through this economic thesis? | **Phase 21's** work (21.1), judged by frontier not format count |

The programme also fixes a **world-class lifetime cost model** (the physical cost
vector — ingest/query/update time, persistent bytes, bytes read, bytes
transferred, peak RSS, I/O request count — plus a monetary cost, both normalised
per successful contract-equivalent answer) and states that **failed gates remain
publishable negative results**. Explicitly **not** pursued: another generic Rayon
pass (ADR-0044), speculative CUDA (ADR-0048), more compression candidates
(ADR-0017), or reviving the refuted cross-root-derivation / adaptive-promotion
results (ADR-0046/0047).

## Phase-23 outcomes

Phase 23 closes the **two durability gaps Phase 20.1 named but could not measure**, by
engineering rather than by re-assertion. Both are **complete and sealed**.
Results: [phase-23-results.md](../phases/phase-23-results.md); decision:
[ADR-0057](../adr/0057-directory-fsync-and-power-loss-proxy.md); the closing
amendment to [ADR-0053](../adr/0053-batched-packed-sync.md).

| Item | Question | Outcome |
|---|---|---|
| 23.1 Directory `fsync` (GAP 1) | Does an atomic publish make the **rename** durable, so a published manifest survives a power cut? | **Complete — adopted, default `Safe`.** Every atomic writer `fsync`s the containing directory after the rename and on new segment / index creation, through one `src/store/durable.rs` seam (`field::write_atomic`, `FsSeedStore::put_node`, `FsIndexStore::put`, packed `ensure_open`/`seal`). `--dir-sync=off` is an explicit escape hatch. Cost on the host bind mount: fs **717 → 1,400 ms** (~1.95×, one dir-`fsync` per node, ~6,800 for the doc), packed **23 → 35 ms** (~1.5×); below noise on tmpfs. Load-bearing, not decorative: with `--dir-sync=off` the model loses the whole published store 4/4 |
| 23.2 Model-based power-loss proxy (GAP 2) | Can un-barriered bytes be shown to be the only thing lost, without a real power cut? | **Complete — 32 cases, 20 PASS / 0 FAIL / 0 CRITICAL on shipped arms; 12 CRITICAL on the counterfactual `model-drop` arms (sensitivity control); `bad_hash` 0.** Non-default `power-log` feature journals every barrier with its byte range; `tests/power_loss_proxy.rs` folds the log into a per-path model (a file survives only if its create/rename was followed by a parent `dirsync`; content truncated to the last completed barrier), reconstructs the post-power-loss store, and re-runs the Phase-20.1 invariants under Batch/Each × fs/packed, complete and cut. **Residual: a model from a barrier log, not a physical power cut** — no device write cache, journal, torn sectors, or real directory-entry loss |

The **`Batch` vs `Each`** distinction is now stated precisely: under the model they
are **equivalent for the published-manifest invariant** (flush-before-publish), and
differ only for the **unreferenced** open-segment tail (visible at the
`flush.before_sync` cut). The remaining honest open item is a **true hardware power
cut**, which the pinned unprivileged lane cannot reproduce.

## Phase-25 outcomes

Phase 25 closes **two correctness gaps** an external review found in the Phase-23
durability work (both verified in the code first), plus a court weakness. Results:
[phase-25-results.md](../phases/phase-25-results.md); decision:
[ADR-0058](../adr/0058-directory-ancestry-durability-and-unpublished-segment-recovery.md).

| Item | Question | Outcome |
|---|---|---|
| 25.1 Directory-ancestry durability | Does creating a directory make its **entry** durable in its parent, so an ancestor cannot be lost while its child is synced? | **Complete — adopted.** `durable::create_dir_all` now creates each missing component and `fsync`s its parent (default `Safe`); the store-open paths route through it. The proxy model folds `mkdir`/ancestry and prunes non-durable directories deepest-first (e.g. `off` fs removes **827** dirs) |
| 25.2 Unpublished-segment recovery | Does a cut leaving a segment with an incomplete header **reopen** (prefix recovery), as Phase 20 requires — not just fail closed? | **Complete — adopted.** `ensure_open` syncs the segment header **before** the directory; `scan_open_segment` treats an incomplete/unparseable header on an **unsealed** (unpublished) segment as **absent**, so the store reopens and recovers the prefix. The two cut arms that previously failed closed (`record.after_body`, `flush.before_sync`) now **open** |
| 25.3 Strict verdicts | Do the cut arms enforce manifest exactness and prefix recovery? | **Complete.** `lenient_verdict` removed; every arm judged by one rule: no manifest ⇒ **reopen**; published manifest ⇒ exact/serviceable |

Re-sealed (`2026-10-08-phase23-durability-fc63436-phase25`): **32 cases, 20 PASS /
0 FAIL / 0 shipped-arm CRITICAL** under the **stricter** rules; 12 counterfactual
`model-drop` CRITICAL; `bad_hash` 0; fs `off` 723 → `safe` 1,381 ms (~1.91×),
packed 22 → 34 ms (~1.55×). The residual is unchanged: a **model**, not a real
power cut.

## Phase-21 outcomes (format programme)

Phase 21 adds new document-format adapters over the shared package substrates and
judges them by the capability/cost frontier on the same contract, **not by format
count**. Plan: [phase-21-plan.md](../phases/phase-21-plan.md); decision:
[ADR-0059](../adr/0059-xlsx-adapter-and-analytical-comparator.md).

| Item | Question | Outcome |
|---|---|---|
| 21.1 XLSX | Does a SpreadsheetML workbook enter exactly and queryably, exposing the spreadsheet structure as distinct observations, and how does it sit on the equal-contract frontier? | **Complete — adopted.** 21.1.1 OPC/SpreadsheetML surface + exact closure (exact 2/2); 21.1.2 the semantic model — styles, merges, comments(+VML), internal/external hyperlinks, defined names, tables, drawings/charts/media, external rels, and a bounded deterministic **displayed-value** projection kept distinct from the stored formula and cached result (exact 3/3); 21.1.2b a hardening pass an adversarial audit required (checked arithmetic on A1 refs, bounded coordinates/projection, positive DOCX-vs-XLSX detection, precise decline wording); 21.1.3 the **economic court vs a source-retaining SQLite baseline *and* a DuckDB/Parquet comparator** (exactness 8/8). Paired ratios on the 8-workbook self-authored corpus (VOLE/comparator, median): build **0.637×** SQLite / **0.291×** DuckDB (8/0/0 both); storage **0.202×** SQLite (8/0/0) but **1.610×** DuckDB (Parquet carries no provenance and declines Q8/Q9/Q10); warm **0.704×** SQLite (5/0/3, ratio-of-sums 4.18 — still a loss on the large fixture). VOLE records Q3 (formula dependents) and Q7 (chart→table linkage) as **capability gaps**, never equivalences. `2026-10-09-phase21-3-xlsx-b2400f1` |
| 21.2 PPTX | The presentation counterpart from the same OPC substrate (slide/layout/media graph). | **Planned — not started.** |
| 21.2 PPTX | The presentation counterpart from the same OPC substrate (slide/layout/media graph). | **Complete — adopted.** 21.2.1 the PresentationML adapter (non-default `pptx` feature; slide order from `p:sldIdLst`, shapes/text-runs/pictures/embedded tables/groups/connectors, notes/layouts/masters/themes/media; exact 5/5); 21.2.3 the economic court vs a source-retaining SQLite baseline (8 decks, Q1–Q8): paired medians build **0.879×**, storage **0.967×**, cold **0.094×**, warm **0.447×**; exactness 8/8. The audit also fixed a latent cross-field cache-aliasing bug in the field node class ([ADR-0060](../adr/0060-source-scoped-node-identity.md)). `2026-10-09-phase21-2-pptx-8aab956`, `2026-10-09-phase21-3-pptx-054ce93` |
| 21.3 ODS | The spreadsheet counterpart from the ODF substrate (reuses ODT). | **Complete — adopted.** 21.3.1 the OpenDocument Spreadsheet adapter (non-default `ods` feature; sheets/rows/cells with typed value + displayed text + stored formula + style kept distinct, merges, bounded repeated cells/rows, named expressions, styles, comments; exact 8/8); 21.3.2 the economic court vs a source-retaining SQLite baseline **and** the mandatory **DuckDB/Parquet** comparator (8 workbooks, Q1–Q10): paired medians build **0.708×** SQLite / **0.334×** DuckDB, storage **0.175×** SQLite but **1.356×** DuckDB (no provenance), warm **0.477×**; exactness 8/8. `2026-10-09-phase21-3-ods-ef26d97`, `2026-10-09-phase21-3-2-ods-3dd5827` |
| 21.4 ODP | The presentation counterpart from the ODF substrate. | **Complete — adopted.** 21.4.1 the OpenDocument Presentation adapter (non-default `odp` feature; slides = `draw:page` in **document order**, shapes/text runs/pictures/embedded tables/notes/masters/media; exact 6/6); 21.4.2 the economic court vs a source-retaining SQLite baseline (8 decks, Q1–Q8): paired medians build **1.018×** (~parity), storage **1.448×** (VOLE larger here), cold **0.123×**, warm **0.392×**; exactness 8/8. **Completes Wave 1 — six office formats from two shared substrates.** `2026-10-09-phase21-4-1-odp-957a800`, `2026-10-09-phase21-4-odp-econ-957a800` |
| Wave 2 — JSON | The first structured-tree format. | **Complete — adopted.** 21.5 spans-preserving JSON adapter (exact token spans, member order, numeric/escape spelling, duplicate keys kept distinct; `--json-pointer` RFC 6901; exact 8/8) + economic court vs a source-retaining SQLite `json1`/`jsonb` baseline (build **1.244×**, storage **0.599×**, warm **0.570×**; the two mismatches are exactly representation-preservation). `2026-10-09-phase21-5-1-json-0d94667`, `2026-10-09-phase21-5-json-econ-0d94667` |
| Wave 2 — YAML | The second structured-tree format. | **Complete — adopted.** 21.6 a bounded, representation-preserving YAML subset (anchors/aliases as a graph, tags, scalar styles, merge keys, multiple documents, comment spans; `--yaml-path`/`--yaml-anchor`/…; exact 10/10) + economic court vs a normalizing YAML→JSON SQLite baseline (build **1.015×**, storage **0.604×**, warm **0.780×**; spans/tags/styles are pure SQLite capability gaps). `2026-10-09-phase21-6-1-yaml-ceab8ec`, `2026-10-09-phase21-6-yaml-econ-ceab8ec` |
| Wave 2 — CSV/TSV | The tabular family (a cheap high-volume court). | **Complete — adopted.** 21.7 an exact-preserving CSV/TSV adapter (record/field spans + dialect; bounded memory; conservative detection; no CSV index) + economic court vs SQLite **and** the mandatory DuckDB/Parquet comparator (build **0.927×**/0.478×, storage **0.234×** SQLite but **6.396×** DuckDB, warm **0.685×**; exact 10/10 · 6/6). DuckDB wins storage/ingest/indexed reads — recorded. `2026-10-09-phase21-7-1-csv-49f523ba`, `2026-10-09-phase21-7-csv-econ-49f523ba` |
| Wave 2 — Markdown | The prose/docs format. | **Complete — adopted.** 21.8 a bounded, representation-preserving CommonMark subset (exact block/inline spans and bytes; the source is never re-flowed) + economic court vs SQLite **and** a Markdown→HTML/text render baseline (which declines Q8 `not-native`); exact 12/12 · 9/9. `2026-10-10-phase21-8-1-markdown-0e6a97c3`, `2026-10-10-phase21-8-markdown-econ-0e6a97c3` |
| Wave 2 — XML | The structured-tree format with namespaces/DTD policy. | **Complete — adopted.** 21.9 a span-preserving scanner (entities literal); a benign DOCTYPE is accepted and never fetched, a **DTD internal subset is refused** (no XXE/billion-laughs) + economic court vs SQLite **and** `ElementTree`; exact 10/10 · 6/6. `2026-10-10-phase21-9-1-xml-4e3c3fbb`, `2026-10-10-phase21-9-xml-econ-4e3c3fbb` |
| Wave 2 — HTML | The error-recovering web format. | **Complete — adopted.** 21.10 a bounded, span-preserving, error-recovering scanner (raw `script`/`style` bytes, all attribute-quoting forms); a document-level HTML marker wins HTML over XML + economic court vs SQLite **and** `html.parser`; exact 10/10 · 6/6. `2026-10-10-phase21-10-1-html-f04d7545`, `2026-10-10-phase21-10-html-econ-f04d7545` |
| Wave 2 — TOML | The config/structured-tree format. | **Complete — adopted.** 21.11 a span-preserving parser with exact scalar spelling; duplicate keys/table redefinitions decline typed + economic court vs SQLite **and** `tomllib`; exact 10/10 · 7/7. `2026-10-10-phase21-11-1-toml-eb045c17`, `2026-10-10-phase21-11-toml-econ-eb045c17` |
| Wave 2 — JSONL | The line/event-stream format. | **Complete — adopted.** 21.12 a per-line span-preserving model reusing the shared JSON parser; JSON tried first, each non-blank line must be one JSON value + economic court vs SQLite **and** a per-line load; exact 9/9 · 6/6. `2026-10-10-phase21-12-1-jsonl-92e34118`, `2026-10-10-phase21-12-jsonl-econ-92e34118` |
| Wave 2 — EML/MIME | The messaging format. | **Complete — adopted.** 21.13 a span-preserving message model (header spans/order/duplicates/folding, the resolved `multipart/*` tree, exact decoded constituents); a leading mbox `From ` is skipped, a boundary-less multipart declines + economic court vs SQLite **and** `email`; exact 7/7 · 6/6. `2026-10-10-phase21-13-1-eml-02dc88a7`, `2026-10-10-phase21-13-eml-econ-02dc88a7` |
| Wave 2 — Parquet | The first analytical format (the most adversarial comparator). | **Complete — adopted.** 21.14 a dependency-free Thrift-Compact footer reader (schema, chunk inventory with exact spans + statistics, decoded PLAIN/RLE_DICTIONARY values, UNCOMPRESSED/GZIP); unsupported codecs/encodings/types and bombs decline typed + economic court vs SQLite **and the mandatory DuckDB/Parquet comparator**, which **wins the analytical axes** (recorded); exact 13/13 · 7/7. `2026-10-10-phase21-14-1-parquet-bfe32a33`, `2026-10-10-phase21-14-parquet-econ-bfe32a33` |
| Wave 2 — Arrow IPC | The second analytical format. | **Complete — adopted.** 21.16 a dependency-free Flatbuffers reader (schema, record batches with exact spans, decoded primitive/binary buffers, file + stream); nested/view/dictionary types and compression decline typed + economic court vs SQLite **and** a DuckDB comparator reading a recorded **Parquet projection** (the pinned wheel has no Arrow IPC file reader); DuckDB wins the analytical axes; exact 18/18 · 3/3. `2026-10-10-phase21-16-1-arrow-4fdc7ad0`, `2026-10-10-phase21-16-arrow-econ-4fdc7ad0` |
| Wave 2 — JSON5/JSONC | The structured-extra format (JSON superset). | **Complete — adopted.** 21.17 a span-preserving JSON5/JSONC superset reusing the JSON parser (comments with spans, unquoted keys, single quotes, trailing commas, hex/leading-dot/`Infinity`/`NaN` numbers, string continuations, extended whitespace, recorded `jsonc`-vs-`json5` dialect; `json5 = ["json"]`; strict JSON stays `Json`); adapter exactness 10/10. Economic court vs a source-retaining SQLite store (strict-JSON normalized) **and** a conventional JSON5→object load: build **1.049/0.877** SQLite · **1.445/1.188** conv (median/geomean), storage **0.590/0.637** · **31.548/21.606**, warm **0.074/0.100** · **1.042/0.966**; exact 9/9. The strict-JSON lane cannot represent `NaN`/`Infinity` (recorded). `2026-10-10-phase21-17-1-json5-88b30730`, `2026-10-10-phase21-17-json5-econ-10f5b898` |
| Wave 2 — CBOR | The binary structured-tree format (RFC 8949). | **Complete — adopted.** 21.18 a bounded, dependency-free CBOR reader (every major type, encoding width, byte-vs-text, tag numbers never resolved, map order + duplicate keys, float width, definite/indefinite form; `cbor = ["json"]`; no magic bytes); adapter exactness 16/16. Economic court vs SQLite (strict-JSON normalized) **and** a conventional CBOR→host load: build **1.042/1.036** · **1.433/1.427**, storage **0.570/0.601** · **35.406/27.933**, warm **0.467/0.573** · **3.750/3.996**; exact 20/20. The small-int/short-container prefix **overlaps MessagePack** (recorded; ambiguous inputs stay `Opaque`). `2026-10-10-phase21-18-1-cbor-f7093672`, `2026-10-10-phase21-18-cbor-econ-6aecd8c5` |
| Wave 2 — MessagePack | The binary structured-tree sibling of CBOR. | **Complete — adopted.** 21.19 a bounded, dependency-free MessagePack reader (exact format byte incl. signedness, `str` vs `bin`, map order + duplicate keys, float width, extension type + payload; `msgpack = ["json"]`); adapter exactness 17/17 (incl. a CBOR coexistence control). Economic court vs SQLite + a conventional host load: build **1.037/1.033** · **1.432/1.423**, storage **0.571/0.602** · **36.315/28.353**, warm **0.483/0.571** · **4.107/4.020**; exact 21/21. CBOR is tried first, so a CBOR document is never stolen. `2026-10-10-phase21-19-1-msgpack-f7093672`, `2026-10-10-phase21-19-msgpack-econ-6aecd8c5` |
| Wave 2 — config (INI/`.env`/properties) | The key/value-line family. | **Complete — adopted.** 21.20 one bounded adapter covering INI/`.env`/Java properties (section/key/separator/value/comment spans, line order, `export`, quoting, properties continuations + `\uXXXX` spelling, duplicate keys reported; `config = []`); adapter exactness 11/11. Economic court vs a source-retaining extract + a conventional decode-to-host load: build **0.953/0.886** · **0.969/0.876**, storage **0.583/0.542** · **0.537/0.471**, warm **0.083/0.079** · **0.082/0.081**; exact 14/14. The pure `KEY=VALUE` env-vs-properties overlap stays `Opaque` (recorded). `2026-10-10-phase21-20-1-config-646e812b`, `2026-10-10-phase21-20-config-econ-735d9b69` |
| Wave 2 — RSS/Atom | The syndication format (XML bytes). | **Complete — adopted.** 21.21 a bounded RSS 2.0/Atom 1.0 model reusing the XML parser (dialect, element/attribute spans, element order, attribute spelling, Atom namespace; `feed = ["xml"]`; tried before the generic XML detector); adapter exactness 7/7. Economic court vs a source-retaining extract + a conventional decode-to-host load: build **0.966/0.968** · **0.960/0.945**, storage **0.655/0.680** · **0.626/0.618**, warm **0.120/0.144** · **0.120/0.151**; exact 11/11. RSS 1.0 and Atom 0.3 stay `Xml` (recorded). `2026-10-10-phase21-21-1-feed-d426b43a`, `2026-10-10-phase21-21-feed-econ-735d9b69` |
| Wave 2 — GeoJSON | The spatial format (JSON bytes). | **Complete — adopted.** 21.22 a bounded GeoJSON (RFC 7946) model reusing the JSON parser (exact `type` token, `coordinates` nesting with exact numeric spelling, `properties` order + duplicates, `id`/`bbox`/`geometry`/`features` order, foreign members; `geojson = ["json"]`; tried before the generic JSON detector); adapter exactness 8/8. Economic court: build **1.013/0.991** · **0.943/0.960**, storage **0.613/0.655** · **0.601/0.610**, warm **0.090/0.100** · **1.097/1.012**; exact 11/11. **Found-and-fixed defect:** 7 observation projections emitted invalid JSON (stray `"`), fixed in `src/field/observe.rs` (commit `ba4df0d4`) — found by parsing the outputs in the economic court. `2026-10-10-phase21-22-1-geojson-670675f9`, `2026-10-10-phase21-22-geojson-econ-735d9b69` |
| Wave 2 — KML/GPX | The geospatial format (XML bytes). | **Complete — adopted.** 21.23 a bounded KML 2.2/GPX 1.1 model reusing the XML parser (dialect, element/attribute spans, element order, KML `coordinates`/GPX `lat`/`lon` spelling, namespace; `gis = ["xml"]`; tried before the generic XML detector); adapter exactness 8/8. Economic court: build **0.951/0.955** · **0.953/0.922**, storage **0.650/0.661** · **0.610/0.587**, warm **0.105/0.122** · **0.110/0.128**; exact 10/10. Older KML (2.0/2.1) and GPX 1.0 namespaces stay `Xml` (recorded). `2026-10-10-phase21-23-1-gis-aed361cf`, `2026-10-10-phase21-23-gis-econ-735d9b69` |
| Wave 2 — Jupyter notebook (`.ipynb`) | The document-shaped format (JSON bytes). | **Complete — adopted.** 21.24 a bounded nbformat model reusing the JSON parser (exact `nbformat`/`nbformat_minor`, exact `cell_type`/`output_type`, exact `source` form — string vs line array never re-joined, `execution_count`, cell/output order, `metadata`, `attachments`; `notebook = ["json"]`; tried before the generic JSON detector); adapter exactness 7/7. Economic court: build **0.971/0.992** · **0.947/0.962**, storage **0.641/0.676** · **0.620/0.622**, warm **0.091/0.098** · **1.008/0.990**; exact 11/11. Plain JSON / non-nbformat `cells` stay `Json` (recorded). `2026-10-10-phase21-24-1-notebook-b6860e87`, `2026-10-10-phase21-24-notebook-econ-735d9b69` |
| Wave 2 — PSV + fixed-width (tabular extra) | The third CSV delimiter and the column-position tabular format. | **Complete — adopted.** 21.25.1 the **pipe (PSV)** delimiter as the CSV/TSV adapter's **third recorded dialect** (one parser, three delimiters; tried last and **declined on a GFM delimiter row**) **and** a **distinct** fixed-width (column-position) adapter — inferred per-column start/end positions and widths, the uniform record width, every record's exact content span and every field's exact padded bytes (`fixedwidth = ["csv"]`; requires **≥ 3 uniform-width records** and **declines on any delimited/Markdown table**); adapter exactness **10/10**. Economic court **(3-lane; ADR-0059)** vs a source-retaining store **and** a conventional decode-to-host load **and** the mandatory **DuckDB/Parquet** lane: build **0.760/0.857** SQLite · **0.928/0.994** conv · **0.333/0.381** DuckDB, storage **0.583/0.590** SQLite · **21.801/8.652** conv · **6.557/7.483** DuckDB (ratio-of-sums **11.233** — **DuckDB wins storage**, e.g. `large.psv` 49,804 B vs VOLE 529,027 B), cold **0.027/0.033** · **0.028/0.034** · **0.010/0.013**, warm **0.074/0.088** · **0.776/0.591** · **0.001/0.003**; exact 11/11. DuckDB equivalence 6 equal / 6 capability-gap / 0 mismatch; an ambiguous aligned-text blob **is** claimed (recorded); cold is Python-startup-dominated. `2026-10-10-phase21-25-1-tabular-e3c77a86`, `2026-10-10-phase21-25-tabular-econ-584ee52e` (3-lane; supersedes `…-f0a3a5cf`) |
| Wave 2 — reStructuredText / AsciiDoc / MDX (docs) | The remaining prose formats, reusing the prose/line layer. | **Complete — adopted.** **21.26.1 reStructuredText** (Docutils; `rst = []`; reST-**specific** signal; exact block/inline spans, section adornments/hierarchy, directives verbatim, field/option/definition lists, literal/doctest blocks, grid/simple tables; adapter exactness **8/8**); **21.26.2 AsciiDoc** (Asciidoctor; `asciidoc = []`; AsciiDoc-**specific** mark; document title/sections with marker + level, attributes with literal references, delimited blocks verbatim, tables, admonitions, macros; adapter exactness **11/11**); **21.26.3 MDX** (`mdx = ["markdown"]`; reuses the Markdown parser/model; ESM/JSX/expression signal; exact ESM/JSX/expression spans, JSX never executed; adapter exactness **11/11**). Economic courts vs a source-retaining store + a conventional decode-to-host load — reST build **0.902/0.928** · **1.223/1.232**, storage **0.602/0.599** · **9.160/7.032**, warm **0.086/0.097** · **1.058/0.889**, exact 9/9; AsciiDoc build **0.915/0.944** · **1.254/1.243**, storage **0.602/0.608** · **7.475/6.129**, warm **0.086/0.097** · **1.069/0.907**, exact 11/11; MDX build **0.904/0.911** · **1.202/1.195**, storage **0.594/0.599** · **9.932/6.197**, warm **0.081/0.093** · **1.027/0.836**, exact 12/12. Boundaries recorded: reST claims spaced `:name:` field lists ahead of AsciiDoc; AsciiDoc needs an AsciiDoc-specific mark; MDX needs an ESM/JSX/expression signal (`Html` stays `html`; inline `{x}` alone does not promote Markdown); cold ratios are Python-startup-dominated. `2026-10-10-phase21-26-{1-rst,2-asciidoc,3-mdx}-91b02068`, `…-26-{1-rst,2-asciidoc,3-mdx}-econ-f0a3a5cf` |
| Wave 2 — MHTML (messaging + web) | The MIME web-archive format, reusing the MIME + HTML layers. | **Complete — adopted.** 21.27.1 a bounded MHTML adapter (`mhtml = ["eml", "html"]`, non-default; reuses the EML MIME layer and the HTML scanner) exposing the MIME envelope, the ordered parts, the ordered sub-resources keyed by `Content-Location`/`Content-ID`, and the root HTML part; decoded (quoted-printable/base64) bytes kept distinct from the retained encoded span; adapter exactness **9/9**. Economic court vs a source-retaining store + a conventional decode-to-host load: build **0.690/0.673** SQLite · **0.821/0.806** conv, storage **0.709/0.736** · **12.298/8.214**, cold **0.026/0.036** · **0.028/0.038**, warm **0.128/0.245** · **1.608/1.969**; exact 7/7. **Over-approximation recorded:** a plain `multipart/related` with a `From`+`Subject` envelope and a `text/html` part **is admitted as MHTML**; warm-vs-SQLite is a recorded loss. `2026-10-10-phase21-27-1-mhtml-eccb438c`, `2026-10-11-phase21-27-mhtml-econ-372a86e0` |
| Wave 2 — syslog / log streams | The line/event-stream format, reusing the line policy. | **Complete — adopted.** 21.28.1 a bounded, dependency-free `logstream = []` adapter preserving each record's recorded dialect (RFC 5424 / RFC 3164 (BSD) / generic), exact line span and terminator, decoded priority, deepest structured-data nesting, and every field's exact span; adapter exactness **9/9**. Economic court: build **0.811/0.839** SQLite · **1.040/1.043** conv, storage **0.593/0.604** · **14.432/10.036**, cold **0.025/0.029** · **0.026/0.031**, warm **0.082/0.107** · **0.566/0.618**; exact 10/10. Boundary recorded: prose whose **every** line begins with a level word **is claimed**; a pure BSD stream without a `: ` **stays `yaml`**. `2026-10-10-phase21-28-1-logstream-bd187003`, `2026-10-11-phase21-28-logstream-econ-372a86e0` |
| Wave 2 — package metadata | The manifest format (npm/Cargo/pyproject), reusing JSON + TOML. | **Complete — adopted.** 21.29.1 a bounded `pkgmeta = ["json", "toml"]` adapter recording the dialect (`npm_package`/`cargo_manifest`/`pyproject_manifest`/`cargo_lock`/`npm_lock`) and a span-preserving section/entry model; adapter exactness **8/8**. Economic court: build **0.723/0.745** · **0.915/0.911**, storage **0.606/0.650** · **15.841/11.515**, cold **0.024/0.029** · **0.025/0.031**, warm **0.109/0.148** · **1.292/1.236**; exact 9/9. A generic JSON `name`+`version` **stays `Json`**; a generic TOML `name`/`version` **stays `Toml`**. `2026-10-10-phase21-29-1-pkgmeta-867eb792`, `2026-10-11-phase21-29-pkgmeta-econ-372a86e0` |
| Wave 2 — API/spec (JSON Schema/OpenAPI/Swagger/AsyncAPI) | The API-spec format, reusing the JSON parser + a schema model. | **Complete — adopted.** 21.30.1 a bounded `apispec = ["json"]` adapter recording the dialect and exposing objects/members/`$ref`s (never resolved); adapter exactness **7/7**. Economic court: build **0.798/0.791** · **1.031/1.040**, storage **0.620/0.784** · **27.245/64.667**, cold **0.023/0.030** · **0.024/0.031**, warm **0.113/0.250** · **1.491/3.833**; exact 8/8. A JSON-Schema-shaped object with no `$schema` **stays `Json`**; storage/warm-vs-SQLite are recorded losses on the large fixture. `2026-10-10-phase21-30-1-apispec-d2f77c3b`, `2026-10-11-phase21-30-apispec-econ-372a86e0` |
| Wave 2 — bibliography (BibTeX/CSL-JSON/RIS) | The bibliography format, reusing the JSON parser for CSL-JSON. | **Complete — adopted.** 21.31.1 a bounded `bib = ["json"]` adapter recording the dialect (`bibtex`/`ris`/`csl_json`) and preserving each field's exact value span (brace/quoted/number/macro kept distinct, duplicates reported; RIS two-space separator + continuation; CSL-JSON member spans); adapter exactness **5/5**. Economic court: build **0.776/0.817** · **0.984/1.002**, storage **0.600/0.670** · **14.463/8.921**, cold **0.022/0.030** · **0.024/0.033**, warm **0.087/0.142** · **1.090/1.089**; exact 6/6. **RIS continuation ambiguity recorded** (a continuation line that begins with two uppercase alphanumerics + `  - ` reads as a new tag); a non-bibliographic `id`+`type` JSON array **is claimed as `csl_json`**. `2026-10-10-phase21-31-1-bib-51bd19a3`, `2026-10-11-phase21-31-bib-econ-372a86e0` |
| Wave 2 — RTF | The legacy word-processor markup, with a control-word grammar. | **Complete — adopted.** 21.32.1 a bounded, dependency-free `rtf = []` adapter preserving the header, every group's exact brace span/depth, control words/symbols, `\'hh` hex escapes, `\uN` with its `\ucN` fallback, destinations verbatim, and byte-exact text runs; adapter exactness **5/5**. Economic court: build **0.664/0.663** · **0.950/0.947**, storage **1.292/1.109** · **1077.372/215.200**, cold **0.047/0.040** · **0.051/0.044**, warm **1.522/0.588** · **38.762/10.792**; exact 6/6. Detection is **magic + brace balance, not a full grammar**; storage-vs-SQLite is a recorded loss. `2026-10-10-phase21-32-1-rtf-714e1009`, `2026-10-11-phase21-32-rtf-econ-372a86e0` |
| Wave 2 — ZIP / TAR containers | The standalone archive formats, reusing the package physical layer. | **Complete — adopted.** 21.33.1 a bounded `container = ["package"]` adapter reusing the byte-authoritative ZIP layer and adding a TAR parser (GNU/PAX extensions folded in), preserving raw member names (incl. non-UTF-8), method, both sizes, CRC-32, and exact spans; adapter exactness **6/6**. Economic court: build **0.706/0.656** · **0.887/0.826**, storage **0.980/1.050** · **24.449/24.342**, cold **0.025/0.025** · **0.025/0.026**, warm **0.098/0.098** · **1.246/1.296**; exact 12/12. **The plain ZIP runs after the office family** (never reclassified); storage-vs-SQLite is a recorded mixed loss. `2026-10-11-phase21-33-1-container-7362e74b`, `2026-10-11-phase21-33-container-econ-372a86e0` |
| Wave 2 — HDF5 / NetCDF | The scientific containers, with a new binary container policy. | **Complete — adopted.** 21.34.1 a bounded, dependency-free `hdf = []` adapter: **NetCDF classic (CDF-1/CDF-2) in full** and a **documented bounded HDF5 subset** (superblock v0/v1, symbol-table groups, contiguous datasets, v1 object headers, fixed-point/float/string, simple dataspace, v1 attributes); out-of-subset HDF5 declines typed; adapter exactness **4/4**. Economic court: build **0.702/0.712** · **0.974/0.988**, storage **0.766/0.782** · **29.258/24.206**, cold **0.022/0.023** · **0.023/0.024**, warm **0.097/0.096** · **1.336/1.349**; exact 8/8. **`comparator_absent`**: HDF5 has **no faithful Python-stdlib decoder**, so `conv` declines every HDF5 question typed. `2026-10-10-phase21-34-1-hdf-ea347edb`, `2026-10-11-phase21-34-hdf-econ-372a86e0` |
| Wave 2 — CFBF / OLE | The compound-document physical layer, unlocking legacy DOC/XLS/PPT/MSG. | **Complete — adopted.** 21.35.1 a bounded, dependency-free `cfbf = []` adapter parsing the 512-byte header, DIFAT/FAT/MiniFAT, the 128-byte directory entries (verbatim UTF-16LE names, reconstructed sibling/child tree), and each stream's exact span; adapter exactness **2/2**. Economic court: build **0.643/0.627** · **0.815/0.796**, storage **1.481/1.467** · **333.156/290.461**, cold **0.023/0.023** · **0.027/0.027**, warm **0.113/0.111** · **1.880/1.848**; exact 7/7. **This is the physical layer only** (the legacy DOC/XLS/PPT/MSG *semantic* models are later work); a **non-contiguous stream chain declines `ExactBytes`** (a bounding hull is recorded); storage-vs-SQLite is a recorded loss; **`comparator_absent`** (no faithful stdlib CFBF decoder). `2026-10-11-phase21-35-1-cfbf-42791bce`, `2026-10-11-phase21-35-cfbf-econ-372a86e0` |

With the **Wave-2 order now complete (21.1–21.35, offices + all families)**, the
remaining format work was the **legacy DOC/XLS/PPT/MSG *semantic* models** on top
of the CFBF physical layer — **now shipped in [Phase 26](#phase-26-outcomes)**.

### Cross-format review-driven corrections (Phase 21 Wave 2)

| Item | Outcome |
|---|---|
| JSON comparator repair (review) | The receipt claimed SQLite `jsonb` but the image had 3.40.1; added a pinned `pysqlite3-binary` (SQLite **3.51.1**, JSONB verified), fixed duplicate-key accounting, and added an **independent span-preserving baseline**. VOLE **matches that baseline on every question it answers** — its differentiators are exact closure and economics. Re-sealed & superseded. `2026-10-09-phase21-5-json-econ-3400385` |
| Permanent cross-field identity court (ADR-0060) | `tools/identity-court.sh`: interleaved multi-format stores, different reconstruction programs, edits, cache clears, SIGKILL-during-build then reopen — **PASS 12/12**. `2026-10-09-identity-3400385` |
| Real-format smoke (bounded step) | `[SUPERSEDED: evidence/campaigns/2026-10-09-realformats-smoke-3400385 — replaced by the stratified court below]` 12 pinned real samples (checksummed) fetched and byte-exact **12/12**; pooled + median reported. A smoke. |
| Stratified real-world court (Phase 21.15) | **52 pinned, checksummed real documents** across 14 Wave-2 formats plus a curated **8-sample malformed/hostile stratum**; every sample byte-exact from a deleted scratch copy in a fresh process. **60/60 exact (52 real + 8 hostile), 0 skipped/0 blocked**; pooled costs reported **alongside** medians per format/size/complexity/stratum. Supersedes the 12-document smoke (still stratified, not a random sample). `2026-10-10-realformats-stratified-3529688e` |
| Detection repair (Phase 21.15) | Five detection root causes the stratified court surfaced: case-insensitive OPC/ODF control parts, ODF `mimetype` precedence, HTML-document-marker precedence over XML, mbox-envelope skip + boundary-less multipart decline, TOML boolean structural terminators. All 60 samples remain byte-exact; legitimate declines recorded. Re-sealed `2026-10-10-realformats-stratified-3529688e` |
| CSV O(n²)→O(n) metadata fix | `csv_common_metadata` re-scanned every record per record; `observe --metadata` on the 50 MiB `large.csv` (~1.47M records) took ~16.5 min. Replaced with a single field-count histogram: ~0.5 s (482,098 us), with the metadata value, field id, source span and dependency ids **byte-identical**. `2026-10-10-phase21-7-1-csv-3006e56a`, `2026-10-10-phase21-7-csv-econ-3006e56a` |
| YAML span-preserving comparator (Phase 21.6.2) | Parity with the JSON repair: a bounded stdlib span-preserving YAML scanner **and** a hash-pinned modern SQLite (3.51.1, JSONB). VOLE's representation advantages shrink/vanish (Q2 8 equal; Q3 1 equal/7 both-decline; Q4 7 equal/1 both-decline; Q6 8 equal; Q7 1 equal/7 both-decline; **0 mismatches**); exact 8/8. Prior `2026-10-09-phase21-6-yaml-econ-ceab8ec` marked `[SUPERSEDED]`; re-sealed `2026-10-10-phase21-6-yaml-econ-92edce73` |

## Phase-26 outcomes (legacy Office semantics, real-world scale, Arrow comparator)

Phase 26 closes the three items `v0.1.0-alpha.35` recorded as *not done*: the four
legacy CFBF **semantic** models, a genuinely larger real-world population, and a
real Arrow reader for the Arrow court. Plan: [phase-26-plan.md](../phases/phase-26-plan.md);
results: [phase-26-results.md](../phases/phase-26-results.md).

| Item | Question | Outcome |
|---|---|---|
| 26.1 DOC | Does a legacy Word 97–2003 `.doc` close exactly and expose a representation-preserving FIB / piece-table / text model on top of CFBF, honestly at the detection boundary? | **Complete — shipped.** Non-default `doc = ["cfbf"]`. Bounded MS-DOC (FIB `0xA5EC`/`nFib`, `fcMin`/`fcMac`, `ccp*`, `fcClx`/`lcbClx`, the piece table incl. compressed CP1252 and uncompressed UTF-16LE pieces); whole-source exact leaf. Adapter-court H1 **2/2** exact, 4 structural declines (`InvalidDocStructure` 52), 3 CFBF + 1 opaque control. `2026-10-11-phase26-1-1-doc-966c3fd4` |
| 26.2 XLS | The legacy BIFF8 workbook, reusing CFBF. | **Complete — shipped.** Non-default `xls = ["cfbf"]`. Bounded BIFF8 record stream (`BOF`/`EOF`, `BOUNDSHEET`, `SST`+`CONTINUE`, cell records `LABELSST`/`NUMBER`/`RK`/`BOOLERR`/`FORMULA`+`STRING`/`BLANK`/`MULBLANK` with row/col and exact span); H1 **1/1** exact, 4 declines (`InvalidXlsStructure` 53), 3 CFBF + 1 opaque. `2026-10-11-phase26-2-1-xls-966c3fd4` |
| 26.3 PPT | The legacy MS-PPT record stream, reusing CFBF. | **Complete — shipped.** Non-default `ppt = ["cfbf"]`. Bounded MS-PPT (`RecordHeader`, `Document` container, `SlideListWithText` slides, `TextCharsAtom`/`TextBytesAtom` runs); H1 **1/1** exact, 5 declines (`InvalidPptStructure` 54), 2 CFBF + 1 opaque. `2026-10-11-phase26-3-1-ppt-966c3fd4` |
| 26.4 MSG | The legacy MS-OXMSG MAPI property streams, reusing CFBF. | **Complete — shipped.** Non-default `msg = ["cfbf"]`. Bounded MS-OXMSG (`__properties_version1.0` header + 16-byte fixed entries, `__substg1.0_*` sub-streams, recipient/attachment sub-storages); H1 **1/1** exact, 8 declines (`InvalidMsgStructure` 55), 2 CFBF + 1 opaque. All six real `.msg` samples in the v2 population were detected as `msg`. `2026-10-11-phase26-4-1-msg-966c3fd4` |
| 26.5 population v2 | A stratified population, not a smoke, with detection reconciliation. | **Complete — shipped.** **308 samples (284 real + 24 hostile) across 41 formats**, all **byte-exact**, 0 skipped/blocked; pinned by length + SHA-256 + license + `redistributable`; **32 findings** reconciled (2 genuine adapter bugs **fixed with regression tests** — RTF trailing NUL/whitespace padding, EML RFC 2231 extended boundary; 30 recorded typed declines / detector-precedence misclassifications); `msgpack`/`fixedwidth` have no real sample (recorded); the 60-sample v1 court is the historical control. `2026-10-11-realformats-stratified-v2-703ca5ac` |
| 26.6 pyarrow lane | Measure Arrow with a real Arrow reader, on equal contract. | **Complete — shipped.** Found and fixed a **corpus-conformance defect** (the hand-crafted IPC was not conformant: bare `Field.type` union tag, table-encoded Blocks, wrong `metaDataLength`) in the generator **and** VOLE's footer `Block` reader; added the hash-pinned **`pyarrow 25.0.1`** lane; fixtures regenerated (H1 suite `tests/arrow_adapter.rs` 13/13). Equal contract: VOLE **build ≈0.51×** and **cold**; **pyarrow wins warm ≈7.3× median** and **storage** (VOLE store ≈2.16× the Arrow file; the file ≈0.46× VOLE's store); `0 mismatches` where pyarrow answers; pyarrow **declines** exact spans (Q3) and byte authority (Q6 `not-native`). Stream fixtures without the `ARROW1` magic decline `Opaque` (recorded caveat). `2026-10-11-phase21-16-arrow-econ-74965426` |

**Scope note (closed by Phase 27.4).** The four legacy formats have **no faithful
conventional decoder** for legacy binary Office/MS-OXMSG, so a conventional
Lane-C comparison is `comparator_absent`. Phase 26 recorded that as *no separate
economic court*; **Phase 27.4 closes it** — a legacy economic court now measures
each format against a source-retaining SQLite store (Lane B, byte authority) with
Lane C recorded as a typed `comparator_absent` decline. See
[phase-27-results.md](../phases/phase-27-results.md).

## Phase-27 outcomes (legacy depth, detection hardening, legacy economics, Arrow streams)

Phase 27 closes the four gaps Phase 26's own report named. Plan:
[phase-27-plan.md](../phases/phase-27-plan.md); results:
[phase-27-results.md](../phases/phase-27-results.md).

| Item | Question | Outcome |
|---|---|---|
| 27.1 DOC depth | Are the footnote/header/annotation/endnote/textbox CP sections separate typed answers? | **Complete — shipped.** Seven `ccp*` sections via `--doc-section`, each a span-preserving answer with clipped piece spans; empty sections typed empty. H1 **3/3** exact, 7/7 sections, 10/10 clipped boundary pieces. `2026-10-12-phase27-1-1-doc-section-fc5cce39` |
| 27.2 PPT depth | Are the master/notes/handout text lists separate typed answers? | **Complete — shipped.** Every `SlideListWithText` by `recInstance` via `--ppt-list`; derived model **v2** (list arena is the source of truth). H1 **2/2** exact, 6 declines (incl. a malformed auxiliary list). `2026-10-12-phase27-2-1-ppt-lists-64626a39` |
| 27.3 Detection hardening | Attack the recorded stratified-v2 findings. | **Complete — shipped.** Findings **32 → 25** (7 fixed), agreements **257 → 264**, **308/308 byte-exact**. Real root cause: **fragmented CFBF sector chains** (PPT/XLS header-prefix read) + ASCII-case-insensitive `Workbook` + GPX 1.0 / namespace-less KML; the rest recorded with reasons. `2026-10-12-realformats-stratified-v2-6f6f84d0` |
| 27.4 Legacy economics | Measure doc/xls/ppt/msg against a source-retaining baseline. | **Complete — shipped.** Lane B source-retaining SQLite; Lane **C `comparator_absent`** (no faithful stdlib decoder — recorded, not fabricated). Q6 exactness **7/7 · 5/5 · 5/5 · 4/4**; VOLE cheaper build/cold/warm but **~1.2–1.4× the storage** of a bytes-only store; cold Python-startup-dominated; degenerate CIs at n=1–3. `2026-10-12-phase27-4-{doc,xls,ppt,msg}-econ-58f7572f` |
| 27.5 Arrow streams | Detect the magic-less Arrow IPC stream framing. | **Complete — shipped.** The bare encapsulated-message stream (continuation-marked `Schema` first, end-of-stream marker) now detects, with an exact-span message inventory; the legacy no-continuation framing declines. Findings **25 → 21** (agreements **264 → 268**). Conformant stream fixture added. `2026-10-12-phase27-5-1-arrow-stream-efb51887` |

**Scope note.** The remaining 21 stratified-v2 findings are **recorded** typed
declines / detector-precedence choices with reasons
(`tools/realcorpus/findings-v2-reconciliation.tsv`): asciidoc/rst/yaml/logstream
precedence, cbor/eml/pptx conservatism, a Word 6.0 FIB, and a fragmented root mini
stream. The legacy models remain bounded and the economic corpora are self-authored.

## Phase-28 outcomes (execution, physical selectivity, revision reuse)

Phase 28 is the **performance programme**: finish detection, repair the competitor
envelope, and run the three decision experiments before any expensive rewrite.
Plan: [phase-28-plan.md](../phases/phase-28-plan.md); results:
[phase-28-results.md](../phases/phase-28-results.md).

| Item | Question | Outcome |
|---|---|---|
| 28.1 Detection completion | Convert the remaining stratified-v2 findings where the precedence is genuinely wrong. | **Complete — shipped.** Findings **21 → 15** (6 fixed), agreements **268 → 274**, **308/308 byte-exact**: fragmented root-mini-stream `.doc` (CFBF model v2 `mini_chain`), Spark/Proxifier/BSD-without-`<PRI>` log dialects, an `xref:`-only AsciiDoc navigation list, a top-level CBOR tag. Trajectory **32 → 25 → 21 → 15**. `2026-10-12-realformats-stratified-v2-941e3d0f` |
| 28.2 Competitor envelope | Give the comparator the real data and separate steady-state from cold. | **Complete — repaired.** DuckDB reads **real Arrow** (C data interface), not a Parquet projection; in-process **steady-state** headline with **cold-process**/**cold-cache** split. VOLE **loses every steady-state analytical axis** (Arrow **2.25×** DuckDB / **27.4×** pyarrow; Parquet **1.46×** DuckDB) but **wins build** and **storage vs SQLite**; exactness 3/3 · 7/7. `2026-10-12-phase28-2-{arrow,parquet}-econ-f7e0a961` |
| 28.3 Hot-path profiling | Attribute the hot path; is the `serve_source_range` clone worth removing? | **Complete — recorded NEGATIVE (28.4 ships nothing).** Paired ablation: median **0.9999**, geomean **1.0040** (CIs span 1.0); the whole `serve_source_range` call is a median **0.09%** of wall — far below the MDE. Selection-before-materialization copies the **same bytes**. Dominant cost is derived-model decode. Exactness 21/21. `2026-10-12-phase28-3-profile-af713225` |
| 28.5 Experiment 1 Fused kernel | Does a native single-column Arrow predicate/aggregate kernel beat the shipped path, and where does it stand vs pyarrow/DuckDB? | **RESOLVED POSITIVE.** fused/current-inprocess median **0.0253** (MDE 0.0302) → resolved faster; vs DuckDB **0.0077** (**~30× faster**, reversing 28.2's 2.25× loss); vs pyarrow **loses on the aggregate** (geomean **1.99**; large fixture **6.46×**), near-tie on evidence (1.03). Exactness 3/3. Semantic finding: `observe --kind text` parses integers only (0 for non-integral floats). `2026-10-12-phase28-5-fused-arrow-e41a78fe` |
| 28.6 Experiment 2 Fragment locality | Is the current closure too broad for a narrow request, and is a fragment directory worth it? | **Decision — coarse structural index justified.** The closure is the **whole encoded document**; floor+dir is **42×–435×** smaller (5/5) but beats page-SQLite in only **3/5** formats (PDF/DOCX, marginal Parquet; loses CSV/JSON). A per-record index is **not viable** (CSV/native 23.5 MB > source). Exactness 5/5. Historical control Phase 22.5 **3.15×**. `2026-10-12-phase28-6-fragment-locality-dad05faa` |
| 28.7 Experiment 3 Revision reuse | How much verified work is reusable across revisions, and at what index cost? | **RESOLVED — positive but marginal/scoped.** A fragment-level dependency graph reuses **83.4%** of next-revision observations and saves **55.5%** of full-rebuild work at a **9.13%** index — **flips NEGATIVE** if per-observation identity nodes are persisted (**35.5%**), **0%** on whole-document redrafts. Semantic edges load-bearing (**986** unchanged-bytes invalidations). Exactness 20/20. `2026-10-12-phase28-7-revision-reuse-b06a2f76` |

**Next programme — 28.8+ (the winning mechanisms).** The three decision gates set
the **order**, not the attractiveness on paper: build the **fused observation
executor** (28.5), a **coarse structural** fragment directory (28.6), and
**incremental observation** under the minimal dependency-signature index (28.7) —
each with its own falsifiable court and the recorded negatives carried forward.

## Phase-29 outcomes (generalized fused execution, coarse structural index, incremental observation)

Phase 29 builds exactly the three mechanisms the Phase-28 gates justified, each
with its own falsifiable court. Plan: [phase-29-plan.md](../phases/phase-29-plan.md);
results: [phase-29-results.md](../phases/phase-29-results.md).

| Item | Question | Outcome |
|---|---|---|
| 29.1 Generalized fused execution | Can one bounded, validity-aware fused kernel (shared versioned interface + a physical scanner per format) beat the shipped path and stand against pyarrow/DuckDB across Arrow/Parquet/CSV/fixed-width and count/sum/min/max/avg/grouped? | **Shipped.** `src/adapter/fused.rs` (`FUSED_PLAN_VERSION = 1`) + `fused-scan`. Beats **DuckDB every cell** (median **0.0033**, 570/0/0) and the shipped in-process path (70/0, median **0.034**); vs **pyarrow** median **0.078** but split (**406/41/163**) — its vectorized C++ kernel wins large inputs, so the large-input Arrow gap narrowed **6.46× → ~2.7–3.0×** (improved, not closed). **Parquet stats-skip `count` 5.4× faster than pyarrow**; **CSV 15–51× slower**. Exactness **12/12**, oracle agreement **228/228**; out-of-subset columns decline typed. `2026-10-12-phase29-1-fused-19d2c372` |
| 29.2 Coarse structural index | Does a realized, persisted coarse index serve a narrow request with fewer real bytes than the whole-descriptor path, and at what directory overhead? | **Shipped.** `src/field/coarse.rs` + `coarse-index`/`coarse-read`. **3 real defects fixed** (the `--use-map` lane never worked; unbounded Thrift-Compact recursion → stack overflow on a hostile Parquet footer; O(n²) PDF scan). **Span** requests cut the source payload to one native unit (**PDF 456×, DOCX/ODT 5×, Parquet 2.7×**); the closure beats the whole-descriptor control **4/4** and page-SQLite **3/4**. **Honest loss:** the natural **reading** request narrows in only **1/4** formats; PDF's directory is heavy (**30 KB**). Exactness **4/4**. Control Phase 28.6. `2026-10-12-phase29-2-coarse-index-1cda34b4` |
| 29.3 Revision-aware incremental observation | Can semantic dependency edges make cross-revision reuse **correct**, and what does the index cost? | **Shipped; MARGINAL/scoped.** `src/field/incremental.rs` + `field-incremental` (versioned `VOLEINC1`; physical **and** semantic edges; full/cas/incremental lanes). **Correctness verified** — incremental answers **== full rebuild on every revision**, including the style/shared-string cases where the physical-only CAS reuses **1042 DOCX paragraphs / 24 XLSX cells** and **diverges wrong**. Reuse **87.3% CSV / 59.2% JSON**, **0%** on the real whole-document draft chain. Index **27.3%** of source (**above the gate's 10% bar**; drops the per-observation identity node and is still above it) → **MARGINAL/scoped**. Exactness **16/16**. Control Phase 28.7. `2026-10-12-phase29-3-incremental-879e6334` |

**Order carried forward.** 29.1's large-input pyarrow loss and 29.3's index cost
are the recorded boundaries: the fused kernel is a resolved steady-state win
against the shipped path and DuckDB but not against pyarrow's vectorized reader on
large inputs, and incremental observation is correct but pays more index than the
28.7 gate allowed.

## Unmeasured gates

- `N4` (decline-rate threshold): no pre-registered threshold exists, so it is
  **not evaluated**.

## Open hypotheses from the consolidated findings

These are concrete, falsifiable next steps. Finer-than-object shareable units
item 1 has since been measured (a recorded negative, ADR-0028).

1. A structural model stronger than LZ — one that removes structure without
   paying a per-site plan (e.g. canonical/parametric layout, nested content
   proceduralization). Prior: loses — four independent plain-syntax
   proceduralizations lost on framing/model overhead.
2. Producer-population corpora — the enabling condition of the one structural win
   (shared plaintext that is also large/weakly coded) has never been observed in
   genuinely transformed producer output, only in our own fixtures. Prior:
   loses/unknown.
3. A like-for-like lifetime frontier on the common answered set (removing
   declined cases from both numerator and denominator), per the Phase-12 skeptic's
   `N1`/`N2` concerns.

## Recorded negatives (do not re-attempt without new evidence)

Whole-file compression (0/27, ADR-0017); seek bytes-read versus fine-block
formats (ADR-0019); cross-document sharing at object granularity (ADR-0021) and
finer-than-object granularity (ADR-0028); cross-document durable work reuse
(ADR-0034/0035, `N3`; re-violated with canonical derivation identity in Phase
15.7, ADR-0047); encoder-only parametric search as a byte win (ADR-0022);
adaptive procedural promotion on the tested corpus (ADR-0046); residency as a
repeated-observation win above ~1 MiB (ADR-0042); the `serve_source_range`
selection-before-materialization clone as an end-to-end win (Phase 28.3 — a
no-op on the tested corpus and a median **0.09%** of wall, below the MDE). See
[Findings](findings.md).

## real100-v1 corpus (frozen, 2026-10-07)

The `real100-v1` real NASA/NIST corpus is frozen at **100 documents**
(branch `real100-v1`; bytes gitignored, manifest + `SHA256SUMS` committed).
Diversity is **66 PASS / 13 PARTIAL / 1 FAIL** of 80 pre-performance gates. See
[the corpus README](../../real100-v1/README.md) and the close-out note
[phase real100-v1 corpus](../phases/real100-v1-corpus.md).

The remaining PARTIAL/FAIL gates are **corpus-spec over-constraints given
published material**, recorded rather than fabricated:

| Quota | Status | Why it cannot be met |
|---|---|---|
| NIST DOCX `long regulatory` | FAIL | NIST publishes no long regulatory Word document; Handbooks 44/130/133/105 are PDF-only |
| NIST `PDF<->DOCX` 8-10 | PARTIAL (1) | Only 5 NIST SP families publish DOCX at all, and the SP slot count is fixed at 5; max is 5, and they compete with `PDF<->EPUB` |
| `glossary/index` DOCX | PARTIAL (1) | NIST publishes a single index-bearing Word document |
| size `<100KiB` / `100KiB-1MiB` / `1-10MiB` | PARTIAL | Their exact targets plus the three satisfied large bands sum to 95, impossible at 100 documents |
| NASA `TR` 8-10 / `handbook/ref` 3-5, `simple born-digital`, `appendix/ref-heavy` | PARTIAL | Technical-doctype minimums sum to 33 > the 30 non-e-book NASA PDFs |

This is a corpus deliverable, not a codec claim; it does not itself assert any
VOLE result.