Skip to main content

Crate gguf_rs

Crate gguf_rs 

Source
Expand description

§GGUF File Parser

A Rust library for parsing and reading GGUF (GGML Universal Format) files.

GGUF files are binary files that contain key-value metadata and tensors, commonly used for storing quantized machine learning models like LLaMA, Phi, etc.

§Features

  • Decode GGUF files (v1, v2, v3)
  • Access key-value metadata
  • Access tensor information
  • Support for little-endian and big-endian files
  • CLI tool for quick inspection
  • Optional memory-mapped file support (enable mmap feature)

§Example

use gguf_rs::get_gguf_container;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Open a GGUF file
    let mut container = get_gguf_container("model.gguf")?;
    let model = container.decode()?;

    // Print model info
    println!("Version: {}", model.get_version());
    println!("Architecture: {}", model.model_family());
    println!("Parameters: {}", model.model_parameters());
    println!("File type: {}", model.file_type());
    println!("Tensors: {}", model.num_tensor());

    // List tensors
    for tensor in model.tensors() {
        println!("  {}: {:?} {:?}", tensor.name, tensor.kind, tensor.shape);
    }

    Ok(())
}

§CLI Usage

Install the CLI tool:

cargo install gguf-rs

Show model info:

gguf model.gguf

Show tensors:

gguf model.gguf --tensors

§Memory-Mapped Files

For large files, enable the mmap feature for more efficient access:

[dependencies]
gguf-rs = { version = "0.1", features = ["mmap"] }
ⓘ
use gguf_rs::mmap::MmapGGUF;

let mmap = MmapGGUF::open("large_model.gguf")?;
let model = mmap.model();

§Async I/O

For async applications, enable the async feature:

[dependencies]
gguf-rs = { version = "0.1", features = ["async"] }
ⓘ
use gguf_rs::async_io::AsyncGGUF;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut container = AsyncGGUF::open("model.gguf").await?;
    let model = container.decode().await?;

    println!("Architecture: {}", model.model_family());
    Ok(())
}

Modules§

writer
GGUF file writing support GGUF file writing support

Structs§

GGUFContainer
GGUF file container for reading GGUF binary files.
GGUFModel
Decoded GGUF model containing metadata and tensors.
Tensor
Tensor in the GGUF file.
V1
Version 1 of the GGUF file.
V2
Version 2 of the GGUF file.
V3
Version 3 of the GGUF file.

Enums§

ByteOrder
Byte order of the GGUF file.
GGMLType
GGML type of a tensor in the GGUF file.
MetadataValueType
Metadata value type in GGUF files.
Version
Version of the GGUF file.

Constants§

FILE_MAGIC_GGJT
Magic constant for ggml files (versioned, ggjt).
FILE_MAGIC_GGLA
Magic constant for ggla files (LoRA adapter).
FILE_MAGIC_GGMF
Magic constant for ggml files (versioned, ggmf).
FILE_MAGIC_GGML
Magic constant for ggml files (unversioned).
FILE_MAGIC_GGUF_BE
FILE_MAGIC_GGUF_LE
Magic constant for gguf files (versioned, gguf)
GGUF_DEFAULT_ALIGNMENT
Default tensor-data alignment per GGUF spec.
GGUF_MAX_DIMS
Maximum number of tensor dimensions supported by GGUF, matching GGML_MAX_DIMS.
GGUF_MAX_STRING_LENGTH
Maximum allowed string length in bytes (1 GiB), matching llama.cpp’s GGUF_MAX_STRING_LENGTH.
GGUF_VERSION_V1
GGUF_VERSION_V2
GGUF_VERSION_V3

Functions§

get_gguf_container
Get a GGUFContainer from a file, truncating tokenizer arrays to length 3.
get_gguf_container_array_size
Get a GGUFContainer from a file with the provided max array size.