Struct LLMCache 

pub struct LLMCache { /* private fields */ }

LLM call cache

Caches LLM call inputs/outputs, avoiding duplicate API calls for identical requests.

Example

use langchainrust::core::cache::LLMCache;

let cache = LLMCache::new();
cache.put("key", llm_result).await;

if let Some(cached) = cache.get("key").await {
    println!("缓存命中: {}", cached.result.content);
}

Implementations

impl LLMCache

pub fn new() -> Self

Creates a cache with default configuration.

pub fn with_config(config: CacheConfig) -> Self

Creates a cache with the given configuration.

pub fn build_key( messages: &[Message], model: &str, ) -> Result<String, CacheError>

Builds a cache key from a message list.

Serializes the message list to a JSON string as the key. Includes the model name so calls to different models do not affect each other. On serialization failure, returns an error instead of falling back to an empty string (M34).

pub async fn get(&self, key: &str) -> Option<CachedLLMResult>

Fetches a cached result.

An expired entry is removed immediately (H36). A hit refreshes cached_at, keeping the cache in true LRU semantics (Q7: a hit makes the entry the most-recently used).

pub async fn put(&self, key: impl Into<String>, result: LLMResult)

Stores a cached result

pub async fn clear(&self)

Clears the cache

pub async fn len(&self) -> usize

Returns the cache size

pub async fn is_empty(&self) -> bool

Whether the cache is empty

pub async fn evict_expired(&self) -> usize

Removes expired entries

Trait Implementations

impl Default for LLMCache

fn default() -> Self

Returns the “default value” for a type.