Module ann_cache 

Process-wide cache for the HNSW AnnIndex used by dense semantic search.

Building an HNSW graph is O(n log n) with a wide construction beam, so doing it per query would be slower than brute force. This cache keeps one built index keyed by a content fingerprint of the embedding set: repeated queries over the same corpus reuse the graph and get sub-linear search, while a changed corpus (different fingerprint) transparently triggers a rebuild.

It is threshold-gated — corpora below ANN_MIN_VECTORS skip the cache and use exact SIMD brute-force top-k, which is both faster (no graph overhead) and exact. On any lock failure it falls back to brute force, so correctness never depends on the cache being available.

Constants

ANN_MIN_VECTORS

Minimum corpus size before an HNSW graph is worth building and caching. Below this, exact SIMD brute force is faster and exact (no recall loss). At 2500, a medium codebase (~7k chunks for lean-ctx itself) enters the HNSW path and gets sub-linear dense search; brute force remains the default for smaller projects where it is both simpler and just as fast.

Functions

clear

Drop the cached HNSW index (#685 eviction hook). The next large-corpus query transparently rebuilds it; queries in between fall back to exact brute force, so correctness is unaffected. Called by the eviction orchestrator under memory pressure — before this hook the built graph + its FlatEmbeddings corpus stayed resident forever.

memory_usage_bytes

Approximate resident bytes held by the cached HNSW index (flat embedding matrix + graph adjacency), 0 when empty. Used by the eviction orchestrator to weigh the ANN cache against the RSS budget.

topk

Returns the top-k (index, similarity) pairs for query over embeddings, sorted by descending similarity.