Skip to main content

Module model_cache

Module model_cache 

Source

Structs§

CachedEngine
A model entry in the cache.
ModelCache
Multi-model cache with LRU eviction under VRAM pressure.

Enums§

ModelResidency
Where a cached model’s weights currently reside.