Type Alias CachedBackend 

pub type CachedBackend<T> = Arc<Mutex<T>>;

Handle to a cached backend. Callers lock the inner mutex for the duration of an inference call to serialize with concurrent requests for the same model.

Aliased Type

pub struct CachedBackend<T> { /* private fields */ }