Struct CachedIndex 

pub struct CachedIndex<I> { /* private fields */ }

A drop-in IndexCore wrapper that memoizes search results.

CachedIndex holds any I: IndexCore and forwards every call to it, with one addition: identical search calls — same query and same SearchParams — are served from an in-memory LRU cache instead of re-running the search. Because it is an IndexCore, it slots in anywhere the wrapped index does, including behind Box<dyn IndexCore>.

Correctness

The cache never returns a stale result. Every mutation that can change the search space — insert, insert_batch, and delete — invalidates the cache, so a search after a write always recomputes against the current index. Operations that do not change the result set (flush and the read-only accessors) leave the cache intact.

Opt-in

Caching is an optimization a caller chooses by wrapping an index; the database leaves indexes unwrapped by default. Construct a cache that holds a fixed number of recent searches with new or with_capacity, or tune it through a CacheConfig with with_config. A capacity of 0 disables caching entirely: every search passes straight through, which is useful for A/B measuring the cache’s effect without changing call sites.

Time-to-live

A CacheConfig::ttl gives entries an expiry: a cached result older than the TTL is treated as a miss and recomputed. Mutations through this wrapper already invalidate exactly, so the TTL exists to bound staleness from changes the wrapper cannot see — for example, the wrapped index mutated through another handle. With no TTL (the default) the clock is never consulted.

Concurrency

CachedIndex<I> is Send + Sync whenever I is (which every IndexCore is). Reads share the cache behind a Mutex held only for the lookup and the insert — never across the wrapped search — so concurrent misses run the underlying search in parallel rather than serializing on the lock.

Examples

use std::sync::Arc;

use iqdb_cache::CachedIndex;
use iqdb_index::{Index, IndexCore, IndexStats};
use iqdb_types::{DistanceMetric, Hit, IqdbError, Metadata, Result, SearchParams, VectorId};

// A minimal index that returns one hit per search; enough to show the wrap.
struct Stub {
    dim: usize,
    metric: DistanceMetric,
    ids: Vec<VectorId>,
}

impl IndexCore for Stub {
    fn insert(&mut self, id: VectorId, _v: Arc<[f32]>, _m: Option<Metadata>) -> Result<()> {
        self.ids.push(id);
        Ok(())
    }
    fn delete(&mut self, id: &VectorId) -> Result<()> {
        match self.ids.iter().position(|x| x == id) {
            Some(pos) => { let _ = self.ids.remove(pos); Ok(()) }
            None => Err(IqdbError::NotFound),
        }
    }
    fn search(&self, _q: &[f32], params: &SearchParams) -> Result<Vec<Hit>> {
        Ok(self.ids.iter().take(params.k).map(|id| Hit::new(id.clone(), 0.0)).collect())
    }
    fn len(&self) -> usize { self.ids.len() }
    fn dim(&self) -> usize { self.dim }
    fn metric(&self) -> DistanceMetric { self.metric }
    fn flush(&mut self) -> Result<()> { Ok(()) }
    fn stats(&self) -> IndexStats {
        IndexStats { n_vectors: self.ids.len(), index_type: "stub", ..IndexStats::default() }
    }
}

let stub = Stub { dim: 3, metric: DistanceMetric::Cosine, ids: vec![VectorId::from(1u64)] };
let mut cached = CachedIndex::new(stub);

let params = SearchParams::new(1, DistanceMetric::Cosine);
let first = cached.search(&[1.0, 0.0, 0.0], &params)?;  // miss: runs the search
let again = cached.search(&[1.0, 0.0, 0.0], &params)?;  // hit: served from cache
assert_eq!(first, again);

let stats = cached.cache_stats();
assert_eq!(stats.hits, 1);
assert_eq!(stats.misses, 1);

Implementations

impl<I: IndexCore> CachedIndex<I>

pub fn new(inner: I) -> Self

Wraps inner with a result cache of the default capacity (1024 recent searches) and no TTL.

Examples

let cached = CachedIndex::new(stub_index());
assert!(cached.is_enabled());

pub fn with_capacity(inner: I, capacity: usize) -> Self

Wraps inner with a result cache that holds at most capacity recent searches and no TTL.

A capacity of 0 disables caching: searches pass straight through and nothing is stored.

Examples

let cached = CachedIndex::with_capacity(stub_index(), 256);
assert_eq!(cached.capacity(), 256);

let bypass = CachedIndex::with_capacity(stub_index(), 0);
assert!(!bypass.is_enabled());

pub fn with_config(inner: I, config: CacheConfig) -> Self

Wraps inner with a result cache built from config (the Tier-2 path).

Use CacheConfig to set the capacity and an optional TTL together.

Examples

use std::time::Duration;

use iqdb_cache::{CacheConfig, CachedIndex};
let config = CacheConfig::new().capacity(512).ttl(Duration::from_secs(30));
let cached = CachedIndex::with_config(stub_index(), config);
assert_eq!(cached.capacity(), 512);
assert_eq!(cached.ttl(), Some(Duration::from_secs(30)));

pub fn capacity(&self) -> usize

The configured cache capacity. 0 means caching is disabled.

pub fn ttl(&self) -> Option<Duration>

The configured per-entry time-to-live, or None if results expire only on mutation.

pub fn policy(&self) -> EvictionPolicy

The configured eviction policy.

pub fn is_enabled(&self) -> bool

Whether caching is active (capacity > 0).

pub fn get_ref(&self) -> &I

Borrows the wrapped index.

pub fn into_inner(self) -> I

Unwraps the cache, returning the index it held.

Examples

let cached = CachedIndex::new(stub_index());
let inner = cached.into_inner();
assert_eq!(inner.dim(), 3);

pub fn clear_cache(&mut self)

Drops every cached result, keeping the wrapped index untouched.

Mutations invalidate automatically; call this only to force a cold cache (for example, after the wrapped index was changed through a handle other than this wrapper).

pub fn cache_stats(&self) -> CacheStats

A snapshot of the cache’s hit/miss counters and occupancy.

Trait Implementations

impl<I: IndexCore> IndexCore for CachedIndex<I>

fn insert( &mut self, id: VectorId, vector: Arc<[f32]>, metadata: Option<Metadata>, ) -> Result<()>

Insert one vector into the index.

fn insert_batch( &mut self, items: Vec<(VectorId, Arc<[f32]>, Option<Metadata>)>, ) -> Result<()>

Insert many vectors in a single call.

fn delete(&mut self, id: &VectorId) -> Result<()>

Remove the vector identified by id from the search space.

fn search(&self, query: &[f32], params: &SearchParams) -> Result<Vec<Hit>>

Run a top-k similarity search.

fn len(&self) -> usize

The number of vectors currently searchable in the index.

fn is_empty(&self) -> bool

Returns true when the index holds no searchable vectors.

fn dim(&self) -> usize

The dimensionality of vectors this index was configured for.

fn metric(&self) -> DistanceMetric

The distance metric this index was configured for.

fn flush(&mut self) -> Result<()>

Flush any pending state to durable storage.

fn stats(&self) -> IndexStats

A runtime snapshot of the index’s state.

fn search_batch( &self, queries: &[&[f32]], params: &SearchParams, ) -> Result<Vec<Vec<Hit>>, IqdbError>

Run a batch of top-k searches with shared params.