Expand description
LLM cache.
LlmCache is the trait. Stock backends:
InMemoryLlmCache(always available)SemanticLlmCache<E>— embeds the prompt; returns a cached answer if a previous prompt was withinthresholdcosine distance. Useful for “near-duplicate” cache hits.SqliteLlmCache(featuresqlite) andRedisLlmCache(featureredis): backend stubs whose real wire-up lives in a deployment patch.