Skip to main content

Module api

Module api 

Source
Expand description

GGUF high-level API (load, tokenizer, model config)

Structs§

GgufLoadResult
Result of loading a GGUF file with full tokenizer data and model config
GgufModelConfig
Transformer model configuration extracted from GGUF metadata CRITICAL for APR inference - must match realizar::apr::AprMetadata
GgufRawLoadResult
Result of loading GGUF with raw quantized tensors
GgufRawTensor
Raw tensor data with quantization preserved
GgufTokenizer
Tokenizer data extracted from GGUF file

Functions§

load_gguf_raw
Load GGUF with raw quantized tensors (preserves Q4K for GPU inference)
load_gguf_tensors
Load GGUF file and extract all tensors as F32
load_gguf_with_tokenizer
Load GGUF file and extract tensors, tokenizer, AND model config