Struct StreamingLoader 

pub struct StreamingLoader { /* private fields */ }

Streaming layer loader for memory-efficient loading of large models.

Instead of loading all weights at once, this loader loads one layer at a time, allowing models larger than available RAM to be processed.

Implementations

impl StreamingLoader

pub fn new(path: &Path, config: LoadConfig) -> Result<Self>

Create a new streaming loader.

pub fn model_config(&self) -> GgufConfig

Get model configuration.

pub fn total_layers(&self) -> usize

Get total number of layers.

pub fn current_layer(&self) -> usize

Get current layer index.

pub fn has_more_layers(&self) -> bool

Check if there are more layers to load.

pub fn load_embeddings(&self) -> Result<Vec<LoadedTensor>>

Load embedding and pre-layer normalization tensors.

pub fn load_next_layer(&mut self) -> Result<Option<Vec<LoadedTensor>>>

Load the next layer’s tensors.

pub fn load_output_head(&self) -> Result<Vec<LoadedTensor>>

Load final normalization and output head tensors.

pub fn reset(&mut self)

Reset to beginning for another pass.