Skip to main content

Module vector_hnsw

Module vector_hnsw 

Source
Expand description

HNSW-backed VectorBackend implementation.

Wraps the existing HnswService behind the unified VectorBackend trait so that it can be used standalone or as the hot tier inside HybridBackend.

§Hardening features (Cognitum Seed WS1)

  • Epoch-based versioning: monotonic epoch bumped on every mutation.
  • Optimistic concurrency: insert_with_epoch rejects stale writes.
  • Soft-delete + compaction: tombstone records excluded from search.
  • Capacity limits: configurable max_vectors with StoreFull error.
  • Tombstone persistence (WEFT-127): HnswBackend::save_to_file / HnswBackend::load_from_file round-trip id map, epoch, capacity, and soft-delete tombstones so deleted entries stay gone after restart.

§Soft-delete and compaction

Soft-delete (VectorBackend::soft_delete) marks a numeric id as tombstoned at the current epoch. Tombstoned ids are excluded from search, contains, and len, but the underlying HNSW entry remains until compaction.

VectorBackend::compact with older_than_epoch physically removes tombstones whose deleted_at_epoch is strictly less than that threshold: the id-map entry is dropped and the string-keyed vector is deleted from the inner HnswService. Callers that want durable soft-delete across process restarts must call HnswBackend::save_to_file after deletes (and after compact, if they want the purged state on disk).

On-disk format is a JSON superset of the core HnswStore snapshot: legacy files with only entries / ef_search / ef_construction load with an empty tombstone section (backward compatible).

Compiled only when the ecc feature is enabled.

Structs§

HnswBackend
HNSW vector backend wrapping HnswService.