Struct OclaCacheBridge 

pub struct OclaCacheBridge { /* private fields */ }

Shared proxy adapter for the OCLA response cache.

Implementations

impl OclaCacheBridge

pub fn new(cache: Arc<ResponseCache>) -> Self

Creates a bridge backed by a shared OCLA response cache.

pub fn try_cache_hit( &self, model: &str, prompt_hash: &str, temp: f32, max_tokens: u32, ) -> Option<CachedProviderResponse>

Returns a cached response body for matching request fields.

pub fn record_response( &self, model: &str, prompt_hash: &str, temp: f32, max_tokens: u32, status: StatusCode, body: &[u8], tokens: u64, )

Records a response for matching request fields.

Trait Implementations

impl Clone for OclaCacheBridge

fn clone(&self) -> OclaCacheBridge

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OclaCacheBridge

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.