Module semantic_cache_manager 

Semantic Cache Manager

A production-quality semantic-aware cache that returns cached results for queries whose embeddings are semantically similar to previously cached queries — even when the exact query text differs.

Unlike a regular cache (exact key match), a SemanticCacheManager can return a cached result for a semantically similar query, reducing redundant computation when two different phrasings ask for the same information.

Features

• Exact-match fast-path via FNV-1a hashed query text
• Semantic-match slow-path via cosine similarity scan
• TTL expiry on every lookup (lazy) and explicit batch expiry
• Five eviction strategies: LRU, LFU, SemanticCluster, TTLFirst, HybridScore
• Byte-budget enforcement (max_bytes)
• Semantic neighbors query for introspection
• Greedy cluster statistics for monitoring embedding distribution
• Comprehensive statistics (hit-rates, avg similarity on semantic hit, etc.)

Example

use ipfrs_semantic::semantic_cache_manager::{
    ScmCacheConfig, ScmCacheKey, ScmCacheHit, ScmEvictionStrategy, SemanticCacheManager,
};

let config = ScmCacheConfig {
    max_entries: 256,
    max_bytes: 64 * 1024 * 1024,
    semantic_threshold: 0.90,
    ttl_us: None,
    strategy: ScmEvictionStrategy::LRU,
    cluster_radius: 0.15,
};

let mut mgr = SemanticCacheManager::new(config);

let key = ScmCacheKey::new("What is IPFS?".to_string(), vec![0.1, 0.2, 0.3]);
let id = mgr.insert(key, b"answer bytes".to_vec(), None).expect("insert ok");

let hit = mgr
    .lookup("What is IPFS?", &[0.1, 0.2, 0.3], 0)
    .expect("lookup ok");
assert!(matches!(hit, ScmCacheHit::Exact { entry_id } if entry_id == id));

Structs

ScmCacheConfig

Configuration for super::SemanticCacheManager.

ScmCacheEntry

A single cached result, keyed by a ScmCacheKey.

ScmCacheKey

Cache key: combines the raw query text, its embedding, and a fast FNV-1a hash of the query text used for exact-match lookups.

ScmCacheStats

Runtime statistics snapshot from super::SemanticCacheManager::stats.

SemanticCacheManager

A semantic-aware cache backed by cosine similarity search.

Enums

ScmCacheError

Errors returned by super::SemanticCacheManager operations.

ScmCacheHit

Result returned by super::SemanticCacheManager::lookup.

ScmEvictionStrategy

Determines which entry to remove when the cache is full.

Type Aliases

ScmEntryAlias

Alias: the cache entry type used by SemanticCacheManager.

ScmErrorAlias

Alias: the error type used by SemanticCacheManager.

ScmHitAlias

Alias: the cache hit/miss result type used by SemanticCacheManager.

ScmKeyAlias

Alias: the cache key type used by SemanticCacheManager.

ScmStatsAlias

Alias: the cache statistics type used by SemanticCacheManager.