Struct ProviderCache 

pub struct ProviderCache { /* private fields */ }

Provider response cache

Caches AI provider responses to improve performance and reduce API calls. Uses SHA256 hash of request to create cache keys.

Implementations

impl ProviderCache

pub fn new( cache_dir: impl AsRef<Path>, ttl_seconds: u64, ) -> Result<Self, ProviderError>

Create a new provider cache

Arguments

• cache_dir - Directory to store cache files
• ttl_seconds - Time-to-live for cache entries (default: 86400 = 24 hours)

Errors

Returns error if cache directory cannot be created

pub fn get( &self, provider: &str, model: &str, request: &ChatRequest, ) -> Result<Option<ChatResponse>, ProviderError>

Get a cached response

Arguments

• provider - Provider name (e.g., “openai”, “anthropic”)
• model - Model name (e.g., “gpt-4”, “claude-3”)
• request - Chat request

Returns

Returns cached response if found and not expired, None otherwise

pub fn set( &self, provider: &str, model: &str, request: &ChatRequest, response: &ChatResponse, ) -> Result<(), ProviderError>

Cache a response

Arguments

• provider - Provider name
• model - Model name
• request - Chat request
• response - Chat response to cache

Errors

Returns error if response cannot be cached

pub fn invalidate( &self, provider: &str, model: &str, request: &ChatRequest, ) -> Result<bool, ProviderError>

Invalidate a cached response

Arguments

• provider - Provider name
• model - Model name
• request - Chat request

Returns

Returns Ok(true) if entry was deleted, Ok(false) if entry didn’t exist

pub fn clear(&self) -> Result<(), ProviderError>

Clear all cached responses

Errors

Returns error if cache cannot be cleared

pub fn cleanup_expired(&self) -> Result<usize, ProviderError>

Clean up expired cache entries

Returns

Returns the number of entries cleaned up