Struct Cache 

pub struct Cache { /* private fields */ }

Implementations

impl Cache

pub fn new(config: &Value, storage: Arc<dyn Storage>) -> Result<Self, String>

pub fn admin_routes(&self) -> Router

Trait Implementations

impl Extension for Cache

fn name(&self) -> &str

Human-readable name for this extension, used in logs and diagnostics.

fn prefix(&self) -> Prefix

Fixed 4-byte prefix that namespaces this extension’s storage keys.

fn on_cache_lookup( &self, request: &ChatCompletionRequest, ) -> BoxFuture<'_, Option<ChatCompletionResponse>>

Check for a cached response before provider dispatch. Return Some to skip the provider call entirely. Called for non-streaming requests only.

fn on_response( &self, ctx: &RequestContext, request: &ChatCompletionRequest, response: &ChatCompletionResponse, ) -> BoxFuture<'_, ()>

Called after a non-streaming chat completion response arrives from the provider.

fn storage_key(&self, suffix: &[u8]) -> Vec<u8>

Build a full storage key by prepending this extension’s prefix to suffix.

fn on_request( &self, _ctx: &RequestContext, ) -> Pin<Box<dyn Future<Output = Result<(), ExtensionError>> + Send + '_>>

Called post-auth, pre-dispatch. Return Err to short-circuit the pipeline (no provider call, no further extensions run).

fn on_chunk( &self, _ctx: &RequestContext, _chunk: &ChatCompletionChunk, ) -> Pin<Box<dyn Future<Output = ()> + Send + '_>>

Called once per SSE chunk during a streaming response, before serialization.

fn on_error( &self, _ctx: &RequestContext, _error: &Error, ) -> Pin<Box<dyn Future<Output = ()> + Send + '_>>

Called when the provider returns an error.