Trait CacheExt 

pub trait CacheExt: Llm + Sized {
    // Provided method
    fn with_cache(self, capacity: usize) -> CachedLlm<Self> { ... }
}

Extension trait for adding caching to any Llm.

Provided Methods

fn with_cache(self, capacity: usize) -> CachedLlm<Self>

Wrap this LLM with an LRU cache of the given capacity.

Identical prompts will return cached results without calling the underlying LLM again.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl<L: Llm> CacheExt for L