Module semantic_cache 

Semantic Cache Layer

A vector-similarity-based cache that returns cached results for queries whose embeddings are semantically close to previously seen queries, avoiding redundant computation.

Overview

The SemanticCacheLayer maintains a collection of CacheEntry items, each associated with a query text and its embedding vector. On each lookup the cache computes the cosine similarity between the incoming query embedding and every non-expired stored embedding. If the best match exceeds config.similarity_threshold the cached result is returned as a CacheLookupResult::Hit; otherwise CacheLookupResult::Miss is returned and the caller should compute the result and insert it.

Example

use ipfrs_semantic::semantic_cache::{
    CacheConfig, CacheEvictionPolicy, CacheKey, CacheLookupResult, SemanticCacheLayer,
};

let config = CacheConfig {
    max_entries: 128,
    similarity_threshold: 0.92,
    ttl_ms: None,
    eviction_policy: CacheEvictionPolicy::Lru,
};

let mut cache = SemanticCacheLayer::new(config);
let now: u64 = 0;

let key = CacheKey {
    query_text: "hello world".to_string(),
    embedding: vec![1.0, 0.0, 0.0],
};
cache.insert(key, "result text".to_string(), now);

match cache.lookup(&[0.999, 0.0447, 0.0], now) {
    CacheLookupResult::Hit { similarity, result, .. } => {
        println!("Cache hit (similarity={:.3}): {}", similarity, result);
    }
    CacheLookupResult::Miss => println!("Cache miss"),
}

Structs

CacheConfig

Configuration for the SemanticCacheLayer.

CacheEntry

A single entry stored in the SemanticCacheLayer.

CacheKey

The key used to store a query in the cache.

ScCacheStats

Aggregate statistics returned by SemanticCacheLayer::stats.

SemanticCacheLayer

A vector-similarity-based cache.

Enums

CacheEvictionPolicy

Policy used to select which entry to evict when the cache is full.

CacheLookupResult

The result returned by SemanticCacheLayer::lookup.