Skip to main content

Module adjacency_delta

Module adjacency_delta 

Source
Expand description

Incremental adjacency: per-commit delta segments (#765).

Each pure-append topology commit writes one tiny Parquet file indexes/adjacency/deltas/<generation>.parquet holding exactly the edges it created (possibly zero rows, for a node-only flush). The adjacency provider serves a fresh view as base CSR ⊎ a contiguous chain of segments covering (G_base, G_cur], so newly-created edges are visible without a full rebuild. Anything that breaks the chain — a DELETE, a crash between commit and segment write, an unreadable segment, or a chain longer than MAX_DELTA_CHAIN — reads as stale and falls back to the existing full-rebuild path.

The merge (apply_delta_segments) reconstructs the base entries from the loaded CSR, concatenates the (filtered) delta entries, and re-runs the builder’s csr_from_entries. Because edge_id is a unique surrogate, the result is byte-identical to a full rebuild from topology/ for the same edges — that equivalence is the correctness contract (acceptance criterion 1), pinned by the tests below.

Structs§

DeltaEdge
One created edge in a delta segment, in creation (ascending edge_id) order.
DeltaSegment
The edges created by the commit that bumped the counter to generation.

Constants§

MAX_DELTA_CHAIN
Longest delta chain served before the index reads stale and is rebuilt. Bounds both per-view merge cost and deltas/ accumulation between rebuilds.

Functions§

apply_delta_segments
Merge a contiguous delta chain onto base (a loaded CSR for direction) and return the overlaid CSR — equal to a full rebuild over the base’s edges plus the chain’s, for stem.
delta_dir
indexes/adjacency/deltas/ within project_dir.
delta_path
Path of the delta segment for generation.
discard_segment
Best-effort removal of the segment at generation — for a commit that bumps the counter but writes no segment (any DELETE / non-pure-append statement). This makes “no segment here” an affirmative invariant: the chain can never contain a file the bumping commit did not author, even across a counter reset that left a stale file at that generation. A failure is harmless — an unexpected file at generation only breaks the chain there, forcing a (correct) rebuild.
prune_delta_segments
Remove every delta segment with generation <= up_to (consumed by a rebuild or compaction at up_to). Best-effort: a failed unlink leaves dead weight a later prune removes, never incorrect data. Segments > up_to (written by a concurrent append during the build) survive, so the new base + those is immediately fresh.
read_delta_chain
Read the contiguous chain of segments covering (base, current].
read_delta_segment
Read one delta segment from disk. A missing file is an error here — callers that tolerate absence (read_delta_chain) check existence first.
write_delta_segment
Write the segment for generation (creating deltas/ if needed). A zero-edge segment is still written so a node-only commit keeps the chain contiguous. Atomic via temp-file + rename (RewriteBatch).