Skip to main content

Module generation

Module generation 

Source
Expand description

Project mutation generations — staleness signals for derived indexes.

Every committed batch that rewrites topology (topology/nodes.parquet or any file under topology/edges/) bumps a monotonically increasing counter persisted at topology/generation.json. Derived indexes record the counter they were built from (see the adjacency manifest); a mismatch against the current value marks the index stale, and a stale or missing index falls back to scan-and-build — identical results, only slower. Property-only writes (properties/, edge_properties/) never bump the topology counter because properties cannot change adjacency.

Search artifacts use the sibling search_generation. It advances for node topology and node-property commits, but not for edge-only, provenance, or knowledge-layer writes. Older projects without this key inherit the current topology generation until their first search-relevant mutation.

§Crash-safety invariant

commit_topology_aware bumps the counter strictly before the first rename of the staged batch. A crash after the bump but before (or during) the commit leaves the counter advanced over an unchanged or partially renamed topology — any existing index now merely looks stale and is rebuilt, costing one spurious rebuild. The reverse order would be unsound: a crash between commit and bump would leave new topology under the old counter, making a stale index look fresh and silently serving wrong traversals. Spurious bumps are safe; missed bumps are not.

Multi-process writers can lose a bump (read-increment-rename is not cross-process atomic); this matches the consistency envelope of every Parquet rewrite in this embedded engine (see crate::staging).

Functions§

bump_search_generation
Atomically advance and persist the graph-native search generation.
bump_topology_generation
Atomically persist current + 1 (sibling temp + rename) and return the new value. Creates topology/ if needed.
commit_topology_aware
Commit staged, bumping each affected generation first (see the module docs for why bump-before-commit is the only sound order). Edge topology advances only topology; node topology advances topology and search; node properties advance only search.
generation_path
Path of the generation counter file within project_dir: topology/generation.json.
read_search_generation
The generation of the committed node topology and node properties consumed by graph-native search.
read_topology_generation
The project’s current topology generation.
touches_search_source
Whether a staged batch changes graph-native search inputs: node identity or label membership (topology/nodes.parquet) or node properties (properties/). Edge-only and knowledge-layer writes are intentionally excluded.
touches_topology
Whether any staged destination in staged rewrites topology: topology/nodes.parquet or any file under topology/edges/ (including _exploratory.parquet). Paths elsewhere (properties/, edge_properties/, provenance/, …) do not count.