Struct Loader

pub struct Loader<Hp: Hyperparameters, F: FnMut(LoadProgress)> {
    pub container_type: ContainerType,
    pub hyperparameters: Hp,
    pub vocabulary: Vocabulary,
    pub tensors: HashMap<String, TensorLoadInfo>,
    /* private fields */
}

A GGML format loader for LLMs.

Fields

container_type: ContainerType

The container type of the model.

hyperparameters: Hp

The hyperparameters of the model.

vocabulary: Vocabulary

The vocabulary of the model.

tensors: HashMap<String, TensorLoadInfo>

The tensors of the model.

Implementations

impl<Hp: Hyperparameters, F: FnMut(LoadProgress)> Loader<Hp, F>

pub fn new(load_progress_callback: F) -> Self

Creates a new loader.

Trait Implementations

impl<Hp: Hyperparameters, F: FnMut(LoadProgress)> LoadHandler<LoadError> for Loader<Hp, F>

fn container_type( &mut self, container_type: ContainerType, ) -> Result<(), LoadError>

Called when the ContainerType is read.

fn vocabulary_token( &mut self, i: usize, token: Vec<u8>, score: f32, ) -> Result<(), LoadError>

Called when a token is read so it can be added to the model’s vocabulary.

fn read_hyperparameters( &mut self, reader: &mut dyn BufRead, ) -> Result<PartialHyperparameters, LoadError>

Called when the model’s hyperparameters need to be read.

fn tensor_buffer(&mut self, info: TensorLoadInfo) -> Result<(), LoadError>

Called when a new crate::Tensor is read for the model.