Struct ModelCache 

pub struct ModelCache { /* private fields */ }

LRU cache of loaded OnnxModel handles.

See the module-level docs for an end-to-end example. Keys are canonicalised via canonical_path so equivalent relative/absolute paths resolve to the same slot.

Implementations

impl ModelCache

pub fn new() -> Self

Create a new cache with DEFAULT_CAPACITY slots.

pub fn with_capacity(capacity: usize) -> MlResult<Self>

Create a new cache with the given positive capacity.

Errors

Returns MlError::CacheCapacityZero if capacity == 0.

pub fn capacity(&self) -> usize

Capacity reported by this cache.

pub fn len(&self) -> MlResult<usize>

Current number of cached models.

pub fn is_empty(&self) -> MlResult<bool>

Return whether the cache is empty.

pub fn get_or_load( &self, path: impl AsRef<Path>, device: DeviceType, ) -> MlResult<Arc<OnnxModel>>

Load path or return the cached handle.

Under the hood the entire operation is serialised by the internal mutex, which avoids the classic double-checked-locking race where two threads both miss and both load the same model.

Errors

Returns MlError::Pipeline with stage "cache" if the internal mutex is poisoned, or any error produced by OnnxModel::load on a cache miss.

pub fn remove(&self, path: impl AsRef<Path>) -> MlResult<Option<Arc<OnnxModel>>>

Remove the cached entry for path if present and return it.

pub fn clear(&self) -> MlResult<()>

Drop all cached entries.

Trait Implementations

impl Debug for ModelCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ModelCache

fn default() -> Self

Returns the “default value” for a type.