Module cache 

Response cache wrapper for LLMClient over a KvStore.

Response cache wrapper for LLMClient backed by a KvStore.

CacheClient is a dedicated LLMClient wrapper — not an observation-only middleware — because serving a cached response must short-circuit the upstream call, which the LLMClientMiddleware hooks (immutable, observe-only) deliberately cannot do. It composes like the other wrappers: CacheClient<RetryClient<OpenAIClient>>.

Only LLMClient::complete is cached; LLMClient::stream_complete is always pass-through, since a streamed response cannot be replayed from a stored buffer without changing its semantics.

The cache key incorporates the client identity (model_name) so that two different providers sharing one KvStore never cross-contaminate an identical request. An optional TTL (CacheClient::with_ttl) expires stale entries; without it entries live until removed.

Structs

CacheClient

An LLMClient wrapper that caches complete responses in a KvStore.