Expand description
Partial HNSW sync with dirty region tracking.
This module implements delta-based synchronisation of HNSW graph state between peers. Rather than shipping the entire index on every gossip round, it tracks which nodes have changed since the last successful sync and ships only those deltas.
§Architecture
┌──────────────────────────────────────────────────────────────┐
│ DirtyRegionTracker │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ dirty_nodes: HashMap<layer, HashSet<node_id>> │ │
│ │ generation: AtomicU64 │ │
│ └─────────────────────────────────────────────────────┘ │
│ ▲ ▼ │
│ PartialSyncManager │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ record_change / build_delta / apply_delta │ │
│ │ pending_deltas: Vec<EmbeddingDelta> (ack tracking) │ │
│ └─────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘Structs§
- Dirty
Region Tracker - Tracks which HNSW nodes are dirty (changed since the last successful sync).
- Embedding
Delta - A delta snapshot of changed embeddings in a region.
- Embedding
Region - A region of the HNSW graph identified by layer + node ID range.
- Partial
Sync Manager - Manages partial sync of HNSW graph changes between peers.