Skip to main content

Module compaction

Module compaction 

Source
Expand description

SPFresh LIRE-style topology-aware local patching.

Rather than rebuilding the entire HNSW on every update, LirePatcher stitches fresh delta vectors into the main graph one node at a time using the graph’s own insert routine (which runs heuristic neighbor selection and bidirectional edge maintenance internally). Tombstones are forwarded to the HNSW as soft-deletes so search skips them immediately; physical removal is deferred to a later background compaction sweep via HnswIndex::compact.

§Partition-quality drift estimate

After inserting each fresh node we inspect its assigned neighbors. For each neighbor n, we count how many of n’s own neighbors were already neighbors of any other newly-inserted node in this batch. A high overlap means the delta nodes clustered into an already-dense region — minimal drift. A low overlap means the new node landed in a sparse region that may require broader re-wiring.

This is an O(patch_size × M) approximation of LIRE’s full local-rebuild signal (SOSP 2023 §4.2). When the average overlap fraction across all patched nodes falls below drift_threshold, we record that subgraph in PatchStats::drift_subgraphs so the caller can schedule a deeper re-pruning pass at lower priority.

Structs§

LirePatcher
SPFresh LIRE-style topology-aware local patcher.
PatchStats
Statistics returned by a single LirePatcher::patch call.