<!--nav-->
**index** · [next](s0-s3-foundations.md) →
<!--/nav-->
# Macrame — Consolidated Architecture
**A Bitemporal Graph Ledger on libSQL · Hybrid Path A/D**
| | |
|---|---|
| Document version | 0.7.0 (consolidated — supersedes v0.1, v0.2, v0.3-Amendment-A, v0.4.0, v0.4.5, v0.5.0, v0.5.1, v0.5.2, v0.5.3, v0.5.4, v0.5.5, v0.5.6, v0.6.0, and v0.7.0) |
| Date | 2026-07-31 |
| Status | Approved for implementation. [§4](s4-schema.md#4-schema) (schema) and [Appendix A](appendices.md#appendix-a--public-api-normative) (API) are the normative surfaces; all other sections are rationale and mechanism. **This file was damaged in transport between 0.5.2 and 0.5.4** — every `#` heading marker stripped, `_` eaten out of identifiers as markdown emphasis, fenced code blocks reduced to a bare language name, and every span between a `<` and the next `>` consumed, sometimes across dozens of lines. It is now fully repaired: headings and fences restored, mangled identifiers corrected throughout, and the eaten spans rebuilt — [§4.1](s4-schema.md#41-concepts-and-per-model-embeddings)'s `retired`/`valid_to` note and its embedding DDL, [§4.3](s4-schema.md#43-the-transaction-log)'s **entire trigger set** (recovered from `schema::ddl`, which turned out to disagree with the surrounding prose in three ways, recorded in place), [§8](s6-s10-flows-to-dependencies.md#8-testing-strategy)'s test list, and the generics throughout [§5.1](s5-modules.md#51-connectionrs--the-handle-the-pragmas-and-the-write-actor)'s sketches. [§5.2](s5-modules.md#52-graphbuilderrs--traversal-valid-time-and-attribute-fidelity)–[§5.9](s5-modules.md#59-vector--embeddings-the-model-registry-and-search), [§6](s6-s10-flows-to-dependencies.md#6-data-flows) and [Appendix A](appendices.md#appendix-a--public-api-normative) had been lost outright and are restored from the 0.5.1 source, forward-ported to the canonical timestamp form ([D-029](s13-decision-register.md#d-029)), the revised archive-session marker ([D-008](s13-decision-register.md#d-008) revised), the cold-database read path ([D-026](s13-decision-register.md#d-026)), the re-derived materialization ([D-035](s13-decision-register.md#d-035)) and native analytics ([D-039](s13-decision-register.md#d-039)). [Appendix A](appendices.md#appendix-a--public-api-normative) is additionally rewritten against the implementation, with the divergences tabulated rather than erased ([D-040](s13-decision-register.md#d-040)). |
| Target platform | Windows desktop, embedded, single-file, no server process |
| Core stack | libSQL (MIT, unmodified) · Rust, async |
| Audience | Implementing engineers, reviewers, future maintainers |
## Revision history
| Version | Cycle | Substance |
|---|---|---|
| 0.1.0 | Initial draft | libSQL-as-dependency doctrine; CTE traversal; DiskANN vectors; CDC replay; checkpoints |
| 0.2.0 | Review cycle 1 | Axis separation in retirement; 5-column bitemporal PK; links_current materialization; transaction_log replaces CDC; per-model embeddings; properties; archiving |
| 0.3.0-A | Review cycle 2 | AttributeMode with log hydration; in-engine vector distance; byte-budget cost model; rebuild_current() |
| 0.4.0 | Consolidation | Single authoritative document; full prose rationale; corrected archive-guard formulation; decision register; glossary |
| 0.4.5 | Amendment A | Priority Write Actor: sole write connection in dedicated task; two-tier biased channels; cooperative chunking (500–1,000 rows); chunked-write fidelity boundary; writer-aware shutdown and snapshot coordination; WriterUnavailable / WriterDroppedResponder containment; concurrency tests; [D-009](s13-decision-register.md#d-009)…[D-016](s13-decision-register.md#d-016) |
| 0.5.0 | Review cleanup 1 | Identifier unification to snake_case; SQL repairs (json_object, temp.sqlite_master, malformed CTE); integrity additions (recorded_at monotonicity trigger, PRAGMA query_only, PRAGMA recursive_triggers); "checkpoint" → "snapshot" rename; [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets) performance table completed; [D-017](s13-decision-register.md#d-017)…[D-021](s13-decision-register.md#d-021) |
| 0.5.1 | Review cleanup 2 | seq_id "gap-free" → "strictly monotonic" (rollback gaps documented); archive scope defined (concepts never physically archived — FK/DiskANN rationale); rebuild_current() sizing note and operational guidance; retired/valid_to semantic clarification; SystemClock monotonicity floor specification; assert→retire→re-assert scenario test; INSTR cycle-detection performance note; TEMP table connection-locality documented; snapshot file cleanup noted; [D-022](s13-decision-register.md#d-022)…[D-025](s13-decision-register.md#d-025) |
| 0.5.2 | Final consolidation | Cold-database ATTACH read path for reconstruct(ts) specified ([§5.5](s5-modules.md#55-temporalreplayrs-and-temporalsnapshotrs--reconstruction-and-snapshots), [D-026](s13-decision-register.md#d-026)); strict SystemClock ISO-8601 parsing contract with graceful startup fallback ([§5.1.2](s5-modules.md#512-handle-shape-and-the-clock-contract), [D-027](s13-decision-register.md#d-027)); write-queue latency semantics and timeout-≠-cancellation rule documented ([§5.1.8](s5-modules.md#518-write-queue-latency-and-caller-timeouts-052-d-028), [D-028](s13-decision-register.md#d-028)); [R14](s11-s12-milestones-and-risks.md#r14) added |
| 0.5.3 | Implementation cycle 1 | [D-008](s13-decision-register.md#d-008) corrected: the `temp.sqlite_master` archive-guard probe is unimplementable — SQLite rejects cross-database references at CREATE TRIGGER time — and is replaced by a `main.sqlite_master` marker whose lifetime is bounded by the archive transaction; `trg_concepts_guard_delete` becomes unconditional per [D-022](s13-decision-register.md#d-022); a delete guard is added to transaction_log; archive-session behaviour under lock contention documented |
| 0.5.4 | Implementation cycle 2 | **Schema change.** [§4.1](s4-schema.md#41-concepts-and-per-model-embeddings) now mandates a single fixed timestamp *width*, not merely a `Z` suffix: every temporal column is exactly `YYYY-MM-DDTHH:MM:SS.ffffffZ`, enforced by `CHECK` on all four tables, and the open-interval sentinel widens to `9999-12-31T23:59:59.999999Z` ([D-029](s13-decision-register.md#d-029)). `audit_current()` corrected to count the symmetric difference — the 0.4.5–0.5.3 query reduced to a constant zero and certified any corruption as clean ([D-030](s13-decision-register.md#d-030)). `format_iso8601_utc` / `parse_iso8601_utc` given real implementations ([D-031](s13-decision-register.md#d-031)). Migrations become a real `user_version` ladder with a legacy-free baseline at v2 ([D-032](s13-decision-register.md#d-032)). `reconstruct()`'s ATTACH is released on all paths and snapshots are written atomically. Engine-error classification is concentrated in one boundary ([D-033](s13-decision-register.md#d-033)); the write commands carry typed payloads and both actor matches become exhaustive ([D-034](s13-decision-register.md#d-034)). Model-based property suites added for [Doctrine VI](s0-s3-foundations.md#doctrine-vi) and for the doctrine as a whole; the first of them found a live bitemporal defect in `archive()`, whose `links_current` compensation filtered on valid time while its own archivability predicate also required transaction time, so `archive()` now re-derives the materialization rather than describing the deletion's shadow ([D-035](s13-decision-register.md#d-035)). The 1.0 on-disk compatibility contract is settled — ledger tables frozen and additive-only, derivative tables explicitly rebuildable — with executable tests for both halves ([D-036](s13-decision-register.md#d-036)). Dependency moved to libSQL 0.9.30; [R15](s11-s12-milestones-and-risks.md#r15) rediagnosed as a concurrency fault rather than a volume one, mitigated by serialising libtest and gating the generated-history binaries behind a `property-tests` feature. Vector path implemented: per-model embedding tables and their DiskANN indexes are created by `register_model()`, model names are validated as SQL identifiers by a `ModelName` newtype (the search path previously spliced an unvalidated caller string into a table name), `search_vector()` goes through `vector_top_k` and `vector_distance_cos` — it had been calling `vector_distance`, which does not exist, so the path had never executed — and the tautological dimension check is replaced by the dimension declared in the schema. [§4.1](s4-schema.md#41-concepts-and-per-model-embeddings)'s claim about engine-side enforcement corrected ([D-037](s13-decision-register.md#d-037)); migration verification changed from counting `sqlite_master` to checking required objects by name ([D-038](s13-decision-register.md#d-038)). petgraph dropped for a native adjacency-list Subgraph and five in-crate algorithms, validated by brute-force oracles rather than by a dependency’s track record; the Louvain that was replaced was a stub returning one community per node. Traversal edge types become bind parameters (they were spliced into the CTE as literals on a read path nothing validated), and `TraversalBuilder::attribute_mode` is now read rather than silently ignored. `SubgraphTooLarge` is constructed for the first time, enforcing [D-007](s13-decision-register.md#d-007)’s byte budget ([D-039](s13-decision-register.md#d-039)). **Document restoration:** [§5.2](s5-modules.md#52-graphbuilderrs--traversal-valid-time-and-attribute-fidelity)–[§5.8](s5-modules.md#58-integrity--audit-and-rebuild) and [§6](s6-s10-flows-to-dependencies.md#6-data-flows) were destroyed by a transport corruption that consumed everything between each `<` and the next `>`, and are restored from the 0.5.1 source and forward-ported; [§5.3](s5-modules.md#53-graphvector_filterrs--strategies-and-the-byte-budget-cost-model), [§5.4](s5-modules.md#54-graphsubgraphrs-and-graphalgorithmsrs--native-in-memory-analytics) and [§5.9](s5-modules.md#59-vector--embeddings-the-model-registry-and-search) are recovered from the 0.4.5 document, which carried the material 0.5.1 had reduced to "unchanged from 0.4.0" stubs — the byte-budget cost formulas, the `analytics_annotations` design, and the whole vector-module section that the 0.5.x renumbering dropped without relocating (it had been [§5.6](s5-modules.md#56-temporalas_ofrs--valid-time-queries-and-attribute-hydration); it is [§5.9](s5-modules.md#59-vector--embeddings-the-model-registry-and-search) here, and `search.rs`'s stale citations are repointed). Three premises of the [§5.3](s5-modules.md#53-graphvector_filterrs--strategies-and-the-byte-budget-cost-model) design are recorded as unestablished, one of them measured false: `PRAGMA query_only` rejects `CREATE TEMP TABLE`, so `TwoPhaseTempTable` cannot run on the read connection as configured. Recovering [§5.4](s5-modules.md#54-graphsubgraphrs-and-graphalgorithmsrs--native-in-memory-analytics) surfaced a live defect and it is fixed here: analytics write-back overwrote `concepts.content` with the annotation value and routed derived labels into the ledger, so the schema gains the derivative `analytics_annotations` table ([§4.5](s4-schema.md#45-analytics-annotations--the-second-derivative-table-054-d-041)) on a `v2 → v3` rung — the [D-032](s13-decision-register.md#d-032) ladder's first step beyond the baseline — with `Annotation`, `Database::write_analytics_annotations`, and four regression tests ([D-041](s13-decision-register.md#d-041)). [Appendix A](appendices.md#appendix-a--public-api-normative) is rewritten against the crate with its divergences tabulated ([D-040](s13-decision-register.md#d-040)). **Document de-corruption pass:** the remaining sections — [§0](s0-s3-foundations.md#0-doctrine)–[§4](s4-schema.md#4-schema), [§7](s6-s10-flows-to-dependencies.md#7-errors)–[§13](s13-decision-register.md#13-decision-register) and Appendices B–C — are repaired to match, which recovered [§4.3](s4-schema.md#43-the-transaction-log)'s trigger DDL from `schema::ddl` and exposed three prose-versus-schema divergences there (trigger names, the total absence of any delete-logging trigger, and `embedding_model` missing from concept payloads); [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets) is annotated to say that its budgets are not CI gates and nothing measures them; [§11](s11-s12-milestones-and-risks.md#11-milestones)'s milestone gates are marked against what is actually delivered. **External review, three accepted and two declined:** a covering index for the traversal CTE replaces `idx_lc_src_active`, making the recursive step index-only, with the column order settled by `EXPLAIN QUERY PLAN` after the first reasoning about it proved to be an artifact of the subsumed index still being present ([D-042](s13-decision-register.md#d-042)); snapshots gain a versioned header so an incompatible file is refused rather than deserialized into wrong values ([D-043](s13-decision-register.md#d-043)); a best-effort DETACH before each ATTACH closes the panic-shaped hole that pairing cannot, the proposed RAII guard being unusable because `Drop` cannot await ([D-044](s13-decision-register.md#d-044)). Actor `catch_unwind` and single-flight coalescing of `reconstruct` are declined with reasons recorded ([D-045](s13-decision-register.md#d-045)). A second review, proposing five of `petgraph`'s internal design choices for `Subgraph`, found two already implemented and two real-but-unmeasured — and led to a defect neither it nor the original work had noticed: the byte-budget check ran per row and made loading O(E²), fixed with incremental accounting and a mutation-verified growth-rate test ([D-047](s13-decision-register.md#d-047)). Snapshot composition, recorded as specified-and-not-implemented ([D-046](s13-decision-register.md#d-046)), is then implemented: `reconstruct` composes the newest snapshot at or before `ts` with an anchored `seq_id > :anchor` fold, tombstones and all, gated by a property test that the composed and full-fold answers agree over generated histories — and the anchored fold is the first code [D-024](s13-decision-register.md#d-024)'s inequality rule has ever bound. Measuring that rule showed its stated mechanism was wrong: a rolled-back transaction does *not* leave a `seq_id` gap, because `sqlite_sequence` rolls back with it; the archive's scattered deletions are where gaps actually come from ([D-049](s13-decision-register.md#d-049)). **The vector write path is closed:** `Database::register_model` and `Database::upsert_embeddings` route through the Write Actor, so Phase 3 is reachable from the public API for the first time — it had been green and unreachable, its suite passing because every test opened its own connection ([D-048](s13-decision-register.md#d-048)) |
| 0.5.5 | Implementation cycle 3 | **Phase 5 closed and filtered vector search implemented.** The [§8](s6-s10-flows-to-dependencies.md#8-testing-strategy) test matrix is complete: [Doctrine VIII](s0-s3-foundations.md#doctrine-viii) pinned by requiring `as_of` and `reconstruct` to *differ* across a retroactive correction, in both directions — belief withdrawn and belief added, two tests because an implementation can lose one and keep the other; archive crash safety pinned by failing a session at both dangerous points and requiring the delete guards to re-arm ([D-012](s13-decision-register.md#d-012)); and [Doctrine VII](s0-s3-foundations.md#doctrine-vii) given a generated suite driven through the handle alone ([D-048](s13-decision-register.md#d-048)), including one property that runs a history twice, with and without its embedding writes, because "the derivative is not an input to the ledger" is not a claim one database can be asked. [§5.3](s5-modules.md#53-graphvector_filterrs--strategies-and-the-byte-budget-cost-model) is implemented as `FilteredVectorSearch` and **loses a strategy**: measuring the three premises 0.5.4 had recorded as unestablished showed `TwoPhaseTempTable` has neither of its two mechanisms — `CREATE TEMP TABLE` on the read connection returns `SQLITE_READONLY (8)` under [D-019](s13-decision-register.md#d-019), and there is no allow-list to push into, `vector_top_k` refusing a fourth argument and `vectorIndexSearch` rejecting `argc != 3` in the bundled amalgamation — so it is removed on [D-039](s13-decision-register.md#d-039)'s precedent rather than kept as a name with nothing behind it ([D-050](s13-decision-register.md#d-050)). `CostEstimator` now reads the `byte_budget` it had carried unused, selectivity is *measured* by a bounded counting probe that doubles as the candidate set, and the plan is returned as a `CostEstimate` rather than only logged — closing [D-007](s13-decision-register.md#d-007)'s empirical requirement with a value a test can assert on instead of log output it would have to scrape. The governing rule is that **the strategy may never change the answer**: a `PostFilter` pass that comes back short from a *saturated* index scan escalates to the exact strategy, and the acceptance gate is that the two agree across filter tightness and k. **Two process findings.** An un-reverted mutation was found in `archive.rs` — the archive-session marker created before `BEGIN`, as committed state — which broke every archive across four test binaries and was invisible because `cargo test` without `--no-fail-fast` stops at the one binary already known to fail and never reaches the eleven behind it. And two newly written tests were caught asserting nothing before they were trusted: the chunk-merge test exercised a single chunk while claiming to cross the boundary, and would have passed even sized up, because candidate ids arrive in id order while the fixture made distance monotone in id. |
| 0.5.5b | Hybrid search | **[§5.9](s5-modules.md#59-vector--embeddings-the-model-registry-and-search)'s keyword arm exists.** `concepts_fts` arrives in [§4.6](s4-schema.md#46-the-concept-text-index--the-third-derivative-table-055-d-051) on a `v4 → v5` rung as an FTS5 **external-content** index — the tokens are indexed and the text is not duplicated, so the index cannot come to disagree with the concept about what the concept says, and [D-036](s13-decision-register.md#d-036)'s rebuildability becomes the engine's own `'rebuild'` command rather than code of ours that could drift from the triggers ([D-051](s13-decision-register.md#d-051)). The rung backfills, which [D-041](s13-decision-register.md#d-041)'s could not. `HybridSearch` fuses the vector and keyword arms by RRF at k = 60, reading each arm to a depth of `max(5 × top_k, 50)` because fusing two top-k lists is not the top k of the fusion. Three things are recorded because they are the parts that fail quietly: the update trigger must **retract** old terms using the OLD values or the index goes on matching text the concept no longer contains; arbitrary text is escaped before reaching MATCH, since FTS5 syntax makes a malformed query an exception and `NOT` a *wrong answer*; and there is no delete trigger only because [D-022](s13-decision-register.md#d-022)'s guard is unconditional — a dependency running the wrong way round, where a change to the archive would break the search index. `reciprocal_rank_fusion` is corrected to break ties by id: it sorted on score alone, and ties are the common case, so the same query could answer in a different order twice ([D-047](s13-decision-register.md#d-047)'s determinism trap as a search result that will not sit still). **The suite is green for the first time in several cycles (177/177).** The last red was `a_high_priority_write_completes_while_the_backlog_is_still_queued`, and it was the test that was wrong: it `.await`ed the probe, which yields, so the actor drained all forty queued chunks before the probe's command ever reached the channel — and its `COUNT == 0` assertion was a wall-clock race besides, where [§8](s6-s10-flows-to-dependencies.md#8-testing-strategy) requires an ordering property over committed `seq_id`s. Preempting already-accepted work is not something two-tier channels can do. |
| 0.5.5c | Archive read path | **A live wrong-answer defect, found by taking [D-049](s13-decision-register.md#d-049)'s carve-out seriously ([D-052](s13-decision-register.md#d-052)).** `reconstruct(ts)` decided whether it needed the cold file by asking `MIN(recorded_at) <= ts` — how far back the hot log *reaches*, not whether it is *complete*. `LOG_ARCHIVABLE` removes **superseded** rows, scattered rather than in a prefix, so one entity archived beside one entity never superseded separates the two questions: the unarchived one keeps `MIN` pointing before the cutoff while the archived one's winning row sits in cold. Reproduced through the public API — a concept **vanished entirely** from a pre-cutoff reconstruction, with no error, which is the failure [Doctrine II](s0-s3-foundations.md#doctrine-ii) exists to prevent. The sound test rests on the one guarantee the archive makes, that the newest row per entity is never archivable: hot alone suffices exactly when `ts` is at or after the newest hot stamp, which is `reconstruct(now)` and nothing else. Two tempting widenings are recorded as unsound rather than left for someone to rediscover. Closing it also closed the composition carve-out for free — `ANCHORED_COLD_FOLD` unions both files above the anchor, so the row the delta was missing is visible again — and the test that had pinned the refusal is replaced by [D-049](s13-decision-register.md#d-049)'s own gate extended across the boundary: composed equals full-fold on both sides of the cutoff, mutation-verified by folding the delta over hot alone. **The lesson recorded with it** is about the register rather than the code: this was written down in 0.5.4 as "not a sound completeness test" and sat there, because an unsound test named as a caveat reads as a known limitation instead of as a bug. |
| 0.5.5d | Snapshot cadence | **[D-049](s13-decision-register.md#d-049)'s second carve-out closed, and with it both ([D-053](s13-decision-register.md#d-053)).** [§5.5](s5-modules.md#55-temporalreplayrs-and-temporalsnapshotrs--reconstruction-and-snapshots) had specified a read-side maintenance task writing an anchor every 10,000 log entries; only `close()` ever wrote one, so a long-running session accumulated an unbounded delta and [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets)'s "≤ 200 ms with snapshot" described a mechanism that ran at exit. The specified design is honoured — a fold plus a file write, never the write connection, so [§5.1.5](s5-modules.md#515-cooperative-chunking--the-golden-rule)'s latency bound is untouched — and the lifecycle it was left open on is settled: `Database` owns the task, a `watch` channel stops it (a dropped sender counts, so a handle dropped rather than closed leaves nothing running), and `close()` stops and joins it **before** stopping the actor and taking the final snapshot, because both it and `write_final` end by running retention and retention deletes files. The trigger is a **distance** in log entries rather than a schedule, so an idle database is never anchored; anchoring is at `MAX(recorded_at)` rather than the clock, so no snapshot claims an instant later than anything it reflects. **The stop test asserted nothing until mutation said so** — close, wait, check nothing appeared passes whether or not the task is alive, since nothing is writing after `close()`; it now keeps the log growing through a raw connection and fails under a leaked stop signal. **What this makes newly load-bearing is retention**, still newest-five-flat against a specified "last five plus one daily for thirty days": harmless when snapshots were written once per shutdown, and now the reason older instants fall back to a full fold. |
| 0.5.5e | Snapshot retention | **The daily tier arrives, one entry after the cadence made it matter ([D-054](s13-decision-register.md#d-054)).** [§5.5](s5-modules.md#55-temporalreplayrs-and-temporalsnapshotrs--reconstruction-and-snapshots) has specified "the last five plus one daily for thirty days" since 0.4.0 and the code kept newest-five-flat; the divergence was harmless while a snapshot meant a shutdown, and [D-053](s13-decision-register.md#d-053) turned it into a rule that defeated the feature it had just been given — under load five anchors span minutes, so every older instant folded the whole log. The container header goes to **format v2**, carrying the snapshot's own instant so retention can bucket by day for eighteen bytes a file instead of decompressing each one. That is deliberately a *second copy* of `MaterializedState.timestamp`, which [D-030](s13-decision-register.md#d-030), [D-035](s13-decision-register.md#d-035) and [D-047](s13-decision-register.md#d-047) are all about refusing; the exception is argued rather than assumed, and rests on both being written from one value in one statement with nothing ever rewriting a snapshot in place. The format bump costs nothing new — a schema change already invalidated every snapshot on the same rule one entry earlier, and [D-043](s13-decision-register.md#d-043)'s refusal path is graceful by construction. "Today" is the newest snapshot's own day rather than the wall clock, so retention is a function of the directory's contents and a database untouched for a year is not emptied by the first write after it wakes. **A fixture bug caught by the mutation pass**, and the third of that kind this cycle: the tests built dates with `format!("2026-01-{:02}", day + 1)`, which produces the calendar-invalid `2026-01-41` past day 39, so those snapshots carried no instant at all and two tests were measuring the dateless path while claiming to measure the daily one. |
| 0.5.5f | §9 measured | **The budgets stop being unfalsifiable ([D-055](s13-decision-register.md#d-055)).** [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets) has asserted "these budgets are CI gates" since 0.4.0 with no harness and not one row measured; `benches/budgets.rs` now covers twelve under criterion. They **measure and do not gate**, which contradicts §9's wording deliberately: the numbers are stated for named reference hardware, so an absolute `≤ 5 ms` in CI is an assertion about whichever runner took the job — the flaky red this project refuses by name in the R15 note, and the reason [D-047](s13-decision-register.md#d-047)'s test asserts a ratio ("absolute timings are a property of the machine, the growth rate is a property of the algorithm"). Regression detection is criterion baselines, machine against itself. **Eleven rows pass with room** — three-hop traversal 2.1 ms of 10, `audit_current` 13.8 ms of 200, vector top-10 294 µs of 20 ms, hybrid top-10 2.0 ms of 50, full fold 21 ms of 100, and composition 3.4 ms against the same, the first direct evidence [D-049](s13-decision-register.md#d-049)'s snapshot path earns its keep. **One misses by 20×, and it is the load-bearing one:** `Chunk commit, 500 rows` measures ≈62 ms against ≤ 3 ms, and that row is [§5.1.5](s5-modules.md#515-cooperative-chunking--the-golden-rule)'s golden-rule calibration — the figure [D-011](s13-decision-register.md#d-011) cites when promising the actor yields promptly at chunk boundaries, so an interactive assertion behind a chunk waits twenty times longer than the design claims. Diagnosed (`write_edges_atomic` re-prepares `INSERT_LINK` per row, and `links` has two triggers compiled into every preparation) and deliberately left unfixed, because instrumenting and optimising in one change leaves neither reviewable. **The chunk benchmark first measured its own fixture** — one database appended to per iteration, spread `[296 ms … 780 ms]`, 2.6× where every other row is flat to a few percent; `iter_batched` made setup untimed and the spread closed to 2.6%. The *variance* exposed it, as it did the retention fixture and the chunk-merge test before that. |
| 0.5.5g | Chunk commit | **The one missing row is half a defect and half a wrong budget ([D-056](s13-decision-register.md#d-056)).** `write_edges_atomic` called `tx.execute(INSERT_LINK, …)` per row and libsql prepares on every call, with both of `links`'s triggers compiled into each preparation; hoisting one `prepare` outside the loop (with `reset()` between rows, because libsql's `execute` binds and steps without resetting) took 500 rows from **≈62 ms to ≈37 ms, 41%**. [D-055](s13-decision-register.md#d-055) had named preparation as the suspect and this entry's first draft called it "most of the chunk" — it was 41%, and the harness is what corrected that. Dropping `trg_links_log_insert` and `trg_links_current_sync` on a scratch database and committing the identical 500 rows takes **2.96 ms**, so trigger amplification is ~92% of the residual, most of it `json_object(…)`/`json(NEW.properties)` per row. **And 2.96 ms is the ≤ 3 ms budget to within 1%** — [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets)'s preamble states amplification is included, and the measured *un-amplified* cost lands on the number, so the budget was almost certainly derived from an insert without its triggers. It is therefore unreachable by construction rather than by defect: hitting it means not writing the ledger entry, which is [Doctrine IV](s0-s3-foundations.md#doctrine-iv). The consequence falls on [§5.1.5](s5-modules.md#515-cooperative-chunking--the-golden-rule)'s golden rule — at ~74 µs per amplified row a 3 ms bound implies chunks of ~40 rows, not 500 — and is left open with the measurements attached, because a budget revised to match the code is not a budget and `CHUNK_ROWS` is a throughput-for-latency decision with [D-014](s13-decision-register.md#d-014) attached. |
| 0.5.5h | Bulk paths | **The same hoist in the other three bulk paths, and the numbers invert the reason given for the first one ([D-057](s13-decision-register.md#d-057)).** `write_concepts_atomic`, `write_annotations_atomic` and `upsert_embedding_chunk` all called `execute` per row inside a transaction with an invariant statement text; each now prepares once, resets between rows, and releases the statement before commit or rollback. Measured at 500 rows in a new `bulk_chunks` group rather than claimed by analogy: concepts **34.1 → 11.9 ms** (65%), annotations **4.60 → 2.13 ms** (54%), embeddings **73.4 → 67.4 ms** (8%), against the edge chunk's 41%. [D-056](s13-decision-register.md#d-056) had explained 41% by `links` carrying two triggers whose bodies compile into every preparation, implying the saving tracks trigger weight — it does not. `analytics_annotations` has **no** triggers and gave back 54%; the DiskANN-indexed embedding tables, by far the most expensive per row, gave 8%. Preparation is roughly a constant per row, so the saving is largest where the row is cheapest, and the edge chunk's 41% was mid-range rather than explanatory. Two by-products: the untriggered 2.13 ms is the control this project lacked and corroborates [D-056](s13-decision-register.md#d-056)'s 2.96 ms trigger-free figure; and the embedding chunk at ≈135 µs per vector is now the worst bulk path in the system, which makes [§5.1.5](s5-modules.md#515-cooperative-chunking--the-golden-rule)'s open `CHUNK_ROWS` question sharper for embeddings than for edges. The concepts chunk no longer routes through `upsert_concept` (which prepares per call by construction); both bind through a shared `UPSERT_CONCEPT` and `concept_params` so the column sets cannot drift, and `classify`/`WriteOp::Concept` classification is unchanged. |
| 0.5.5i | Golden rule | **[§5.1.5](s5-modules.md#515-cooperative-chunking--the-golden-rule) re-derived from measurement, and three of its numbers were wrong ([D-058](s13-decision-register.md#d-058)).** The rule is a bound on *duration* — the SQLite write lock is not preemptible, so an interactive write queued behind a chunk waits it out in full — and `CHUNK_ROWS = 1000` was an answer to it presented as the rule itself. Now `CHUNK_BUDGET` = 3 ms (§9's own figure, kept) with four measured row counts: **edges 90, concepts 70, annotations 600, embeddings 30**, measuring **2.39 / 2.35 / 2.36 / 2.06 ms** against the single old constant's **89 / 24 / 3.5 / 143 ms**. Corrections, from a `chunk_scaling` sweep over n ∈ {1…1000}: (1) one constant cannot express one duration across paths whose per-row costs span **60×**; (2) per-transaction overhead is **not** "noise" but a flat **~0.8 ms** — over a quarter of the bound before any row is written, and a floor below ~30 rows; (3) two of the four paths are **superlinear** in chunk size (edges ~11 → ~103 µs/row, embeddings ~35 → ~151 µs/row), so for them the old chunk was the worst latency *and* the worst throughput at once — 1,000 edges cost 88.5 ms as one chunk against ~27 ms as eleven, **3.3× faster for a 37× lower stall**. [D-056](s13-decision-register.md#d-056) had deferred this as "a throughput-for-latency trade"; on half the paths there was nothing to trade. Also corrects that entry's "~40 rows" (extrapolated from the 500-row point, assuming the linearity this disproves — it is 90) and its "unreachable by construction" (the ≤ 3 ms budget *is* reachable fully amplified; what was unreachable was the row count bundled into it). Sizes solve for ≈2.5 ms, not 3: the first pass hit 2.93 ms median with a 2.96 upper estimate, and a bound that holds only for the median chunk is not a latency bound. **Open:** the superlinearity is unexplained, and if it is WAL page spill then every one of these constants is really a function of `PRAGMA cache_size`. |
| 0.5.5j | Superlinearity | **Diagnosed, and it invalidates a claim [0.5.5i](s13-decision-register.md#d-058) made ([D-059](s13-decision-register.md#d-059)).** The `chunk_scaling` sweep measured every chunk size into a *fresh* database, so chunk size and table size were **the same variable**. Separated: a fixed 90-row chunk into a hub of 0 / 2,000 / 8,000 edges costs **4.4 / 18.4 / 47.7 ms** — the cost is in the table, not the chunk. So D-058's "1,000 edges are 3.3× faster as eleven chunks" is wrong; it was eleven copies of the first chunk. End to end, both arms finishing at the same table: **85.5 ms as one transaction, 94.7 ms as eleven** — chunking is ~11% *slower*, and smaller chunks buy latency at a throughput cost on every path, as [D-056](s13-decision-register.md#d-056) originally said. The bound and the four constants stand (measured directly, not extrapolated). **The edge path's growth is a defect:** `trg_links_single_open`'s `EXISTS` plans as `SEARCH links_current USING COVERING INDEX idx_lc_traversal_cover (source_id=?)` — one equality column, because that index *contains* every column the predicate mentions and so wins as covering, while the PK autoindex would bind three but lacks `valid_to`. Every edge insert scans its source's whole out-degree; 90 rows into a 90,000-edge hub take **1.06 s**. Proof by fix: an index on `(source_id, target_id, edge_type, valid_to, valid_from)` gives a four-column lookup and **47.7 → 8.0 ms, flat** (`valid_from` only to keep it covering — omitting it changed nothing, because the planner kept the old index). **This is not a chunking problem**: it slows every interactive `assert_edge` on a high-degree node, so §9's ≤ 5 ms single-assertion budget is met on an empty database and not on a large one. Not shipped — a [D-036](s13-decision-register.md#d-036) schema change wants its own migration rung. **The embedding path is the same shape and not a defect** (49 → 224 µs/vector as the corpus grows 0 → 8,000; DiskANN insertion rewires a graph that grows as the chunk fills it). **Still open:** with the guard gone the `links_current` upsert still runs 25 → 66 µs/row — not the commit (2.2 ms of 63), not the page cache (512 MB changes nothing), not the index count, not table size. Evidence: `examples/chunk_diag.rs`. |
| 0.5.6 | Waves 1–5: the review acted on | **A full read of the crate against this document (2026-07-30) found ten defects inside a green suite, six of them returning a wrong answer rather than an error, and the four waves that answer them are delivered.** *Wave 1 — the silent defects.* The concept log payload omitted `embedding_model`, so every temporal read lost it and `AttributeMode::AtTime` — the mode [Doctrine VIII](s0-s3-foundations.md#doctrine-viii) exists to offer — returned a *less* faithful record than the mode [§5.2](s5-modules.md#52-graphbuilderrs--traversal-valid-time-and-attribute-fidelity) documents as wrong for historical text; payload v2 fixes it, and [Doctrine VII](s0-s3-foundations.md#doctrine-vii)'s static guard had to be narrowed to permit a scalar model *name* where it had banned the substring `embedding` outright. The folds partitioned on `entity_id` alone, so a concept whose id collided with a link key **vanished from the reconstruction**. `Subgraph` let its adjacency reference nodes absent from `nodes`, and the five algorithms met that four different ways — `louvain` panicked, `scc` emitted phantom components, `k_core` inflated degrees, `dijkstra` returned distances to unlookupable nodes; the fix states a *closure invariant* on the type and none of the five needed changing. Three attribute readers held three retirement semantics. `classify_archive_violation` was **deleted** rather than wired up, because `error::classify` already did the same job and [D-033](s13-decision-register.md#d-033) requires one classifier — the defect had been marked Fixed a cycle earlier by a commit that made it *delegate* instead of *called*. *Wave 2 — the deferred decisions.* [D-059](s13-decision-register.md#d-059)'s index ships as the `v5 → v6` rung; overlapping **closed** valid-time intervals are refused in the write actor rather than in a trigger, with the honest consequence recorded in [§4.2](s4-schema.md#42-links-assertion-history-and-current-belief-materialization) that the storage layer permits what the API refuses ([D-060](s13-decision-register.md#d-060)); identifiers become opaque with `\|` and `/` reserved — *neither* offered option survived contact, since requiring ULIDs would have invalidated every id the crate has been used with and full opacity leaves `transaction_log.entity_id` ambiguous between different links ([D-061](s13-decision-register.md#d-061)); and the clock becomes injectable, with flooring lifted into the `Clock` trait because "strictly increasing across restarts" is not a property any clock can hold alone ([D-062](s13-decision-register.md#d-062)). *Wave 3 — measurement.* [D-047](s13-decision-register.md#d-047)'s deferral condition is finally met and **retires** the `Subgraph` rewrite ([D-063](s13-decision-register.md#d-063)); the wave also found a defect of its own making — the Wave 2 overlap guard carried a provably safe narrowing predicate that made a *covering* index win over the selective one, reproducing [D-059](s13-decision-register.md#d-059)'s defect one wave after fixing it, worth +9.8 ms per chunk with every correctness test passing throughout ([D-064](s13-decision-register.md#d-064)) — and **declined** **AF**'s planner cache after measuring it at under 1% of a query, the stated justification for it being false of `corpus_size` ([D-065](s13-decision-register.md#d-065)). *Wave 4 — hardening.* The cadence gets its own connection, `close()` stops discarding the writer's `Result`, and the `Drop` `debug_assert` [§7.3](../Macrame%20Implementation%20Plan%20v0.5.6.md) recommended was implemented, fired on ~30 tests, and **reduced to a warning** — what dropping costs is one snapshot, which [Doctrine VI](s0-s3-foundations.md#doctrine-vi) makes disposable ([D-066](s13-decision-register.md#d-066)); an upgraded database is re-anchored at open, gated twice after the first attempt broke two contracts the suite already pinned ([D-067](s13-decision-register.md#d-067)); `raw()` stays public with actor containment documented as a convention ([D-068](s13-decision-register.md#d-068)); three errors that named the wrong subject are corrected, including `archive_horizon.archived_at` written with the cutoff — recording that every archive ran at the instant it was archiving *up to* ([D-069](s13-decision-register.md#d-069)); and `load_subgraph`'s superlinearity is explained as an O(E log E) `DISTINCT` sort with two attempted fixes measured and rejected ([D-070](s13-decision-register.md#d-070)). *Wave 5 — the last silent paths, in progress.* `reconstruct`'s `'D'` branch handled an operation no trigger writes and now refuses it; `Delta::edges_gone` went with it, since that branch was its only writer and closing one unreachable path by opening another is not a fix ([D-072](s13-decision-register.md#d-072)). The FTS5 `VACUUM` hazard was investigated and **nothing was built**: it is not reachable, because [D-022](s13-decision-register.md#d-022)'s unconditional delete guard keeps `concepts.rowid` dense so renumbering is the identity — making that guard load-bearing for the *search index* as well as the ledger — and the `verify_fts()` that would have proved it was not shipped, because FTS5's `'integrity-check'` verifies internal consistency and **not** agreement with content, reporting healthy on an emptied index ([D-071](s13-decision-register.md#d-071)). `load_subgraph_with` closes the missing `edge_types`/`min_weight` filter — a **reachability** limit rather than a convenience one, since the byte budget bounds the *unfiltered* neighbourhood — and turned up two defects older than itself: the running byte total drifted from `estimated_bytes()` by one byte per edge, and delegating with `TraversalBuilder`'s `min_weight` default of `0.0` silently disarmed the negative-weight guard, turning a typed refusal into a graph quietly missing edges ([D-073](s13-decision-register.md#d-073)). And the three invariants enforced *above* the storage layer are stated once in [§4.7](s4-schema.md#47-what-this-schema-does-not-enforce) instead of three times in three sections — which exposed that they are not the same shape: for overlapping intervals and `raw()` the write API enforces the rule and only an outside writer opens the gap, but `assert_edge` accepts a negative weight, so a file this crate wrote alone can hold a row it declines to read back ([D-074](s13-decision-register.md#d-074)). Finally `write_annotations` — which took `Vec<ConceptUpsert>` and wrote **concepts**, having kept the name when [D-041](s13-decision-register.md#d-041) split it from `write_analytics_annotations` three releases earlier — becomes `write_concepts`, closing [Appendix A.2](appendices.md#appendix-a--public-api-normative)'s last open entry; and clearing the clippy backlog **by hand** found that one of the warnings was ours: `OverlappingInterval`'s seven `String`s made `DbError` 168 bytes, so [D-060](s13-decision-register.md#d-060)'s Wave 2 fix had silently doubled the size of every `Result` in the crate, `Ok` paths included. Boxed, and pinned by a `size_of` unit test, because the failure mode was a line in a build log rather than a red test ([D-075](s13-decision-register.md#d-075)). Suite 171 → 221 (240 with the property features), clippy clean on `--all-targets --all-features`. |
| 0.6.0 | Hardening | **Schema v7 closes the third §4.7 invariant** (D-083): `CHECK (weight >= 0.0 AND weight < 9e999 AND typeof(weight) = 'real')` on `links.weight` — negative and text weights are now refused at the engine level, not just in the loader. `idx_lc_open_interval` ships as the `v5 → v6` rung (D-059), fixing the superlinearity that made every edge insert scan its source's whole out-degree. Overlapping **closed** intervals are refused in the write actor with `OverlappingInterval` (D-060), with the honest consequence stated in [§4.7](s4-schema.md#47-what-this-schema-does-not-enforce) that the storage layer still permits what the API refuses. The chunked rebuild (`rebuild_current_chunked`) ships with shadow-swap (D-082): builds `links_current_shadow` across many small transactions and swaps it in under one, longest hold 353→47 ms (7.6×). Metrics are feature-gated and zero-cost when off (D-079); `HoldTimer` reads no clock when `metrics` is off. `AttributeModeUnstated` becomes a typed error rather than a `tracing::warn!` (D-085): `as_of(ts)` without a stated `attribute_mode` raises `AttributeModeUnstated`. `OverlappingInterval` is boxed (D-075): seven `String`s = 168 bytes pushed `DbError` over `clippy::result_large_err`'s threshold; boxing keeps the error small. NaN is **not** a schema gap (D-078): `weight REAL NOT NULL` rejects NaN at the engine — `weight.is_nan()` in the loader is unreachable. `low_chunked` deduplicates the four bulk loops (D-086): four copies of a yield-critical loop are four places for the yield to be lost. `RebuildInterrupted` ≠ `RebuildFailed` (D-082): the repair *did not run* is not *the repair did not repair*; action is to retry. `raw()` is `#[doc(hidden)]` (D-091): provoking a guard is its legitimate use. `verify_snapshot_chain` (D-092): folds from genesis by withholding the snapshot directory, compares against the composed answer — `seq_anchor` reported but never compared (they legitimately differ); edges compared as a *set*; capped at `SAMPLE_LIMIT = 32` with a `truncated` flag. Unreadable indices `idx_annotations_label` and `idx_lc_tgt_active` added in v8 (D-089), scheduled for removal in 0.7.0. Suite: 240+ tests, all green. |
| 0.7.0 | Python bindings | **A synchronous Python binding** (D-095) built on pyo3 0.29 and maturin, delivered as a wheel alongside the Rust crate. The binding is synchronous because the Write Actor serialises every write through one channel, so exposing `await` on the write path advertises concurrency the architecture does not grant. Runtime boundary: `Python::detach` around `Runtime::block_on`, releasing the GIL; single process-global multi-threaded runtime behind a `OnceLock`; `PyDatabase` is `#[pyclass(frozen)]` over `RwLock<Option<Database>>` — reads take a read lock and run concurrently, `close()` takes the write lock and waits. Error mapping: every `DbError` variant maps to its own Python exception class with its fields as attributes; completeness enforced by an exhaustive `match` with no wildcard arm. Value types: timestamps accept `str` or aware `datetime` in, always return `datetime` with `tzinfo=utc`; open interval crosses as `None` (not as a sentinel `datetime` — `datetime.max` cannot survive `.astimezone()` east of UTC); embeddings accept `bytes` (fast path, 60.8 µs for 768 dims) or any sequence of floats; `Subgraph` stays opaque. Packaging: distribution `macrame-db`, import `macrame`; wheels on `manylinux_2_28` x86_64 + aarch64, macOS universal2, Windows x86_64; `abi3-py310` (D-094): one wheel per platform; wheel ships with `metrics` on (D-093) because a feature flag does not survive into a binary artifact. R15 is transparent to the boundary: 48 concurrent opens from 48 threads fault 2/12, matching the Rust control arm. Stubs: hand-written `_macrame.pyi`, compared to the live extension both ways and to `errors.rs`, verified by five injection tests. |
Contents — [§0](s0-s3-foundations.md#0-doctrine) Doctrine · [§1](s0-s3-foundations.md#1-purpose-and-scope) Purpose · [§2](s0-s3-foundations.md#2-system-context) System context · [§3](s0-s3-foundations.md#3-crate-layout) Crate layout · [§4](s4-schema.md#4-schema) Schema · [§5](s5-modules.md#5-modules) Modules · [§6](s6-s10-flows-to-dependencies.md#6-data-flows) Data flows · [§7](s6-s10-flows-to-dependencies.md#7-errors) Errors · [§8](s6-s10-flows-to-dependencies.md#8-testing-strategy) Testing · [§9](s6-s10-flows-to-dependencies.md#9-performance-budgets) Performance · [§10](s6-s10-flows-to-dependencies.md#10-dependencies) Dependencies · [§11](s11-s12-milestones-and-risks.md#11-milestones) Milestones · [§12](s11-s12-milestones-and-risks.md#12-risks) Risks · [§13](s13-decision-register.md#13-decision-register) Decision register · [§14](s14-python-bindings.md#14-python-bindings) Python bindings · Appendices A–C
<!--index-->
---
## The document, by file
This is the original single-file architecture document, split into one file per group of sections. **The prose is unchanged**: concatenating the files below, minus the navigation header each carries and this table, reproduces the former `Macrame v0.5.4.md` — see [REJOIN.md](REJOIN.md), where the command is checked rather than asserted. The one edit since the split is markup, not text: cross-references are now links (below), and stripping that markup back out returns the document byte for byte.
| File | Sections | What it settles |
|---|---|---|
| [Foundations](s0-s3-foundations.md) | §0 Doctrine · §1 Purpose · §2 System context · §3 Crate layout | The eight invariants, the `as_of` / `reconstruct` distinction, the boundary diagram, and one concern per module |
| [Schema](s4-schema.md) | §4 | **Normative.** Tables, triggers, guards, the canonical timestamp form, the two derivative tables |
| [Modules](s5-modules.md) | §5 | The Write Actor, traversal, replay and snapshots, archive, integrity, vectors |
| [Flows to dependencies](s6-s10-flows-to-dependencies.md) | §6 Data flows · §7 Errors · §8 Testing · §9 Performance · §10 Dependencies | Five end-to-end paths, `DbError`, what each test layer can prove, the budgets, the tree |
| [Milestones and risks](s11-s12-milestones-and-risks.md) | §11 · §12 | M1–M5 with exit gates; R1–R15 with mitigations |
| [Decision register](s13-decision-register.md) | §13 | D-001…D-109: the authoritative record of intent |
| [Python bindings](s14-python-bindings.md) | §14 | The wheel: build topology, the async→sync boundary, errors and timestamps across the FFI, and what it deliberately does not expose |
| [Appendices](appendices.md) | A · B · C | **Normative (A).** The API surface and what it was once promised to be; the glossary; the deferred list |
**Start here** if you are new: the doctrine in [Foundations](s0-s3-foundations.md), then [Schema](s4-schema.md), then the [Decision register](s13-decision-register.md). The doctrine says what may not change, the schema is what encodes it, and the register is why everything else looks the way it does.
**Cross-references are links.** Every `§5.5`, `D-029`, `R15`, `Doctrine VI` and `Appendix A` in the prose resolves to where it is defined, across files as well as within them. Section links target the heading's own anchor; decisions, risks and doctrines have no headings of their own, so each definition site carries an invisible `<a id="…"></a>` — `#d-029`, `#r15`, `#doctrine-vi` — and that is the only thing added to the text besides the links themselves.
Four things are deliberately *not* linked: headings, so a section title never links to itself; fenced code and inline code spans, where `§5.5` is a quoted string rather than a reference; the definition site of each decision, risk and doctrine, which carries the anchor instead; and the table above, whose file links are already the point.