Struct PqCodec 

pub struct PqCodec {
    pub dim: usize,
    pub m: usize,
    pub k: usize,
    pub sub_dim: usize,
    /* private fields */
}

PQ codec with trained codebooks.

Fields

dim: usize

Original vector dimensionality.

m: usize

Number of subvectors (subspaces).

k: usize

Centroids per subvector (fixed at 256 for u8 encoding).

sub_dim: usize

Dimensions per subvector: dim / m.

Implementations

impl PqCodec

pub fn train( vectors: &[&[f32]], dim: usize, m: usize, k: usize, max_iter: usize, memory: ScopedMemory, ) -> Result<Self, VectorError>

Train PQ codebooks from a set of training vectors via k-means.

m = number of subvectors (must divide dim evenly). k = centroids per subvector (typically 256). max_iter = k-means iterations (20 is usually sufficient).

pub fn encode(&self, vector: &[f32]) -> Vec<u8>

Encode a vector: for each subvector, find the nearest centroid index.

This is a per-vector hot-path operation. Governor charging is intentionally skipped here to avoid atomic overhead on every candidate during search; use [encode_batch] for bulk encoding with budget enforcement.

Precondition: vector.len() == self.dim. This per-candidate hot path does not re-check it; every caller checks the dimension first (encode_batch, the IVF-PQ add, and the codec-index entry points).

pub fn encode_batch(&self, vectors: &[&[f32]]) -> Result<Vec<u8>, VectorError>

Batch encode all vectors into a contiguous byte array.

Charges m * vectors.len() bytes to the bound budget (if set) before allocating the output buffer. The guard is released at the end of this call — the buffer itself remains alive.

pub fn build_distance_table( &self, query: &[f32], ) -> Result<Vec<Vec<f32>>, VectorError>

Build an asymmetric distance table for a query vector.

Returns table[sub][centroid] = distance from query’s sub-vector to each centroid. Pre-computing this table makes distance evaluation O(M) per candidate instead of O(D).

Charges m * k * size_of::<f32>() bytes to the bound budget (if set) before allocating the table.

pub fn asymmetric_distance(&self, table: &[Vec<f32>], code: &[u8]) -> f32

Compute asymmetric distance using a precomputed distance table.

O(M) per candidate — just M table lookups and additions.

pub fn decode(&self, code: &[u8]) -> Result<Vec<f32>, VectorError>

Decode a PQ code back to an approximate FP32 vector.

Charges dim * size_of::<f32>() bytes to the bound budget (if set) before allocating the output buffer.

pub fn to_bytes(&self) -> Result<Vec<u8>, VectorError>

Serialize the codec to bytes with a versioned magic header.

Format: [NDPQ\0\0 (6 bytes)][version: u8 = 1][msgpack payload]

Charges the estimated serialized size to the bound budget (if set) before allocating the output buffer. The estimate is conservative: m * k * sub_dim * size_of::<f32>() + 64 (header + framing overhead).

pub fn from_bytes( bytes: &[u8], memory: ScopedMemory, ) -> Result<Self, VectorError>

Deserialize the codec from bytes produced by Self::to_bytes.

Returns VectorError::InvalidMagic if the header does not match NDPQ\0\0, and VectorError::UnsupportedVersion for unknown versions.

Trait Implementations

impl Clone for PqCodec

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PqCodec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl VectorCodec for PqCodec

type Query = PqQuery

Prepared query: precomputed distance table + original FP32 query.

fn encode(&self, v: &[f32]) -> Self::Quantized

Encode an FP32 vector: one centroid index byte per subspace.

Panics

UnifiedQuantizedVector::new fails only when the outlier bitmask does not match the provided outlier slice. With outlier_bitmask = 0 and an empty slice this can never happen. The expect is therefore unreachable in practice.

fn prepare_query(&self, q: &[f32]) -> Self::Query

Prepare the query by precomputing the M×K asymmetric distance table.

The VectorCodec trait does not propagate errors. A PqCodec used via this trait path is created by PqCodec::train which sets no governor; build_distance_table therefore always returns Ok here. If a governor is attached and its budget is exhausted the caller that constructed the codec is responsible for handling the error — this impl panics with a descriptive message so the budget violation is never silently ignored.

fn adc_lut(&self, q: &Self::Query) -> Option<AdcLut>

Build the AdcLut from the precomputed distance table for use by SIMD rerank kernels (pshufb / vpermb).

fn fast_symmetric_distance( &self, q: &Self::Quantized, v: &Self::Quantized, ) -> f32

Symmetric distance between two PQ-encoded vectors.

Both codes are decoded to approximate FP32 vectors via the codebook, then the squared L2 difference is accumulated. This is the correct definition of symmetric PQ distance: each vector is approximated by its nearest centroids, and the distance is computed in FP32.

fn exact_asymmetric_distance(&self, q: &Self::Query, v: &Self::Quantized) -> f32

Asymmetric ADC distance: precomputed distance table vs stored code.

O(M) per candidate — delegates to PqCodec::asymmetric_distance.

type Quantized = PqQuantized

The packed quantized form. Must be convertible to a UnifiedQuantizedVector reference via AsRef.

fn train(&mut self, samples: &[&[f32]]) -> Result<(), CodecError>

Optional: fit the codec’s learned parameters on a set of training vectors.