Struct GgufLoader 

pub struct GgufLoader<B: Backend + BackendQuantGguf + BackendQuantMarlin> { /* private fields */ }

Backend-generic weight loader for GGUF files.

Build with GgufLoader::open. The underlying file stays mmap’d for the lifetime of the loader so per-tensor reads only do byte slicing, not file I/O.

Implementations

impl<B: Backend + BackendQuantGguf + BackendQuantMarlin> GgufLoader<B>

pub fn open(path: impl AsRef<Path>) -> Result<Self>

Open and parse a .gguf file. Tensor payloads stay on disk (mmap’d) until each load_tensor / load_linear call.

pub fn from_file(gguf: Arc<GgufFile>) -> Self

Build from an already-opened GgufFile (test helper, also useful when several loaders share the same mmap).

pub fn gguf(&self) -> &GgufFile

Direct access to the underlying file — exposes metadata + tensor descriptor lookups for callers that need them (e.g. a config helper that reads general.architecture and <arch>.block_count).

Trait Implementations

impl<B: Backend + BackendQuantGguf + BackendQuantMarlin> WeightLoader<B> for GgufLoader<B>

fn load_tensor(&self, name: &str) -> Result<B::Buffer>

Load a single tensor by fully qualified name (e.g. "model.embed_tokens.weight").

fn load_linear(&self, name: &str) -> Result<Box<dyn Linear<B>>>

Load a projection as a Linear<B>. The concrete implementation (DenseLinear / GptqLinear / AwqLinear / GgufLinear) depends on the loader’s file format and quant config.

fn has_tensor(&self, name: &str) -> bool

Whether a tensor with this name exists in the source.

fn quant_config(&self) -> Option<&QuantConfig>

Quantization metadata (parsed from quantize_config.json or a GGUF header). None means the source is dense.