quantize

a simple, fast quantization library usable as a rust crate or python package.

to learn more, feel free to peruse the chapters, which show how the repo's algorithms progressed over time.

use as a library

use quantize::{adaptive, asymmetric, quantize, Scheme};

let weights = [0.42_f32, -0.10, 0.70, -0.50];

let q = quantize::<f32, 8, 32>(&weights).unwrap();
let _ = asymmetric::quantize::<f32, 8, 32>(&weights).unwrap();
let _ = adaptive::quantize::<f32, 32>(&weights, 0.001).unwrap();
let _ = Scheme::Q4_32.quantize::<f32>(&weights).unwrap();

let back = q.dequantize();
let _ = q.dot(&weights);

comparison

1024x1024 matrix, 50 iterations.

quality

cargo run --release --example compare

reconstruct, then matmul.

speed

cargo run --release --example throughput

pack and unpack. apple M4.

ns = nanosecond, a billionth of a second. smaller is faster.