Skip to main content

load_gguf_raw

Function load_gguf_raw 

Source
pub fn load_gguf_raw<P>(path: P) -> Result<GgufRawLoadResult, AprenderError>
where P: AsRef<Path>,
Expand description

Load GGUF with raw quantized tensors (preserves Q4K for GPU inference)

This is essential for APR format to achieve 2x Ollama performance. The Q4K bytes are stored directly in APR and used by GPU kernels.