ruvector-turboquant
Turbo4: a 4-bit Lloyd-Max quantized vector datatype (ADR-296) — Qdrant-style primary-storage quantization, not a search-time cache:
- deterministic randomized Hadamard rotation (sign ⊙ permute ⊙ block-FWHT
rounds, seeded SplitMix64 — bit-stable across platforms and versions,
no
randdependency); - precomputed 16-level Lloyd-Max tables for the rotated (≈ Gaussian) coordinates — no training pass, online ingest;
- packed nibble codes:
D/2 + 8bytes per vector (≈ 7.9× vs f32 at 1536-D) — the original float vector is never stored; - direct scoring on packed codes: symmetric (code×code, for graph construction), asymmetric (int8 query×code, for traversal), and exact f32 rescoring — with runtime-dispatched AVX2 kernels and a scalar oracle they are tested bit-exact against.
use ;
let dim = 128;
let codec = new.unwrap;
let a: = .map.collect;
let b: = .map.collect;
let code_a = codec.encode.unwrap; // 64 + 8 bytes, floats discarded
let code_b = codec.encode.unwrap;
let query = codec.encode_query.unwrap;
let d_sym = symmetric_distance;
let d_asym = asymmetric_distance;
let d_exact = rescore;
assert!;
assert!;