Struct CachedChatModel 

pub struct CachedChatModel { /* private fields */ }

A ChatModel wrapper that caches responses using an LlmCache.

Cache keys are generated by serializing the ChatRequest to JSON. Streaming (stream_chat) delegates directly to the inner model without caching.

Implementations

impl CachedChatModel

pub fn new(inner: Arc<dyn ChatModel>, cache: Arc<dyn LlmCache>) -> Self

Create a new cached model wrapping the given model and cache.

Trait Implementations

impl ChatModel for CachedChatModel

fn chat<'life0, 'async_trait>( &'life0 self, request: ChatRequest, ) -> Pin<Box<dyn Future<Output = Result<ChatResponse, SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn stream_chat(&self, request: ChatRequest) -> ChatStream<'_>

fn profile(&self) -> Option<ModelProfile>

Return the model’s capability profile, if known.