Trait Cache 

pub trait Cache: Send + Sync {
    // Required methods
    fn get(&self, prompt: &str) -> Option<String>;
    fn set(&self, prompt: &str, response: String);
}

Trait for prompt caching strategies.

Required Methods

fn get(&self, prompt: &str) -> Option<String>

Try to retrieve a cached response for a prompt.

fn set(&self, prompt: &str, response: String)

Store a response in the cache.

Implementors

impl Cache for ExactCache

impl Cache for SemanticCache

impl Cache for TieredCache