Expand description
GGUF high-level API (load, tokenizer, model config)
Structs§
- Gguf
Load Result - Result of loading a GGUF file with full tokenizer data and model config
- Gguf
Model Config - Transformer model configuration extracted from GGUF metadata CRITICAL for APR inference - must match realizar::apr::AprMetadata
- Gguf
RawLoad Result - Result of loading GGUF with raw quantized tensors
- Gguf
RawTensor - Raw tensor data with quantization preserved
- Gguf
Tokenizer - Tokenizer data extracted from GGUF file
Functions§
- load_
gguf_ raw - Load GGUF with raw quantized tensors (preserves Q4K for GPU inference)
- load_
gguf_ tensors - Load GGUF file and extract all tensors as F32
- load_
gguf_ with_ tokenizer - Load GGUF file and extract tensors, tokenizer, AND model config