Struct GgufFile 

pub struct GgufFile { /* private fields */ }

Read-only handle to a memory-mapped GGUF file.

Implementations

impl GgufFile

pub fn open(path: impl AsRef<Path>) -> CandleResult<Self>

Open and parse the header of a GGUF file.

Returns immediately after the descriptor table is read — no tensor data is materialised. read_tensor lazy-loads individual tensors.

pub fn content(&self) -> &Content

Raw access to candle’s parsed header — for callers that need the full metadata / tensor_infos maps. Prefer the typed accessors below.

pub fn architecture(&self) -> CandleResult<&str>

Architecture string, e.g. "qwen3", "llama". Read from general.architecture. Errors if the key is missing or non-string.

pub fn metadata(&self, key: &str) -> Option<&Value>

Raw metadata value lookup. Returns None if the key is absent.

pub fn metadata_string(&self, key: &str) -> CandleResult<&str>

Read a string-typed metadata field. Errors if missing or wrong type.

pub fn metadata_u32(&self, key: &str) -> CandleResult<u32>

Read a u32-typed metadata field. Errors if missing or wrong type.

pub fn metadata_u64(&self, key: &str) -> CandleResult<u64>

Read a u64-typed metadata field. Errors if missing or wrong type.

pub fn metadata_f32(&self, key: &str) -> CandleResult<f32>

Read an f32-typed metadata field. Errors if missing or wrong type.

pub fn metadata_bool(&self, key: &str) -> CandleResult<bool>

Read a bool-typed metadata field. Errors if missing or wrong type.

pub fn tensor_count(&self) -> usize

Total number of tensors declared in the header.

pub fn tensor_names(&self) -> impl Iterator<Item = &str>

Iterate over every tensor name in the file. Order is whatever the underlying HashMap yields — do not rely on it being deterministic.

pub fn tensor_info(&self, name: &str) -> Option<&TensorInfo>

Look up a tensor descriptor (shape, dtype, byte offset) without touching the payload. None if the tensor isn’t in the file.

pub fn has_tensor(&self, name: &str) -> bool

Whether a tensor with name is declared in the header.

pub fn read_tensor(&self, name: &str, device: &Device) -> CandleResult<QTensor>

Materialise a tensor as a candle QTensor on the target device.

The returned tensor is still quantized — no dequant happens here. Wrap it in QMatMul::from_qtensor for inference, or call QTensor::dequantize(device) to get a fp32 Tensor.

Beware: candle copies the bytes into an owned Vec<u8> (see TensorInfo::read). For the steady-state weight upload path use Self::tensor_byte_slice instead — it returns a slice directly into the mmap with no allocation.

pub fn mmap_bytes(&self) -> &[u8]

Whole mmap region as a byte slice. Used to wrap the file as a single zero-copy MTLBuffer on Metal — the lifetime of the slice is tied to &self, so the caller is expected to keep an Arc<GgufFile> alive for as long as anything references the mmap.

pub fn tensor_byte_slice(&self, name: &str) -> Option<&[u8]>

Byte slice covering exactly tensor name inside the mmap. The slice points into the file mapping, so reads are demand-paged and there is no heap allocation. Returns None if the tensor isn’t declared.

The byte length is computed from the tensor’s (elem_count, ggml_dtype) using candle’s block_size() / type_size(). For raw quant tensors (Q4K / Q6K / etc.), these bytes are exactly what QTensor::data() would return — but with no copy.

pub fn tensor_byte_range(&self, name: &str) -> Option<(usize, usize)>

(byte_offset_in_mmap, byte_length) for tensor name. Same computation as Self::tensor_byte_slice but returns the indices rather than the slice — useful when the caller already has the mmap base pointer (e.g. when binding a region of a shared buffer at a given offset).

Trait Implementations

impl Debug for GgufFile

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.