Skip to main content

Crate onnx_runtime_quantization

Crate onnx_runtime_quantization 

Source
Expand description

Audited llama.cpp IQ quantization grids shared by CPU and GPU decoders.

Constantsยง

IQ1S_GRID
IQ2S_GRID
IQ2XS_GRID
IQ2XS_SIGNS
IQ2XXS_GRID
IQ3S_GRID
IQ3XXS_GRID