Skip to main content

Module model_lifecycle

Module model_lifecycle 

Source
Expand description

Model lifecycle management — unload/reload for memory efficiency.

When managing multiple large models on resource-constrained devices, unloading idle models reclaims GPU/ANE memory without losing the ability to quickly reload them.

§Design

ModelHandle is a move-based state machine wrapping Model. State transitions (unload, reload) consume self and return a new handle, so the Rust type system prevents use-after-unload at compile time.

  load()       unload()       reload()
 --------> Loaded -------> Unloaded -------> Loaded
               <-----------          <-----------
                 reload()              unload()

Enums§

ModelHandle
A model handle that supports unloading and reloading.