Struct PqCodec 

pub struct PqCodec {
    pub dim: usize,
    pub m: usize,
    pub k: usize,
    pub sub_dim: usize,
    /* private fields */
}

PQ codec with trained codebooks.

Fields

dim: usize

Original vector dimensionality.

m: usize

Number of subvectors (subspaces).

k: usize

Centroids per subvector (fixed at 256 for u8 encoding).

sub_dim: usize

Dimensions per subvector: dim / m.

Implementations

impl PqCodec

pub fn with_governor(self, governor: Arc<MemoryGovernor>) -> Self

Attach a memory governor to this codec.

Once set, heap-significant operations (train, encode_batch, build_distance_table, decode, to_bytes) will charge the EngineId::Vector budget before allocating and release the reservation when the returned value is dropped (RAII). When no governor is set those operations proceed unconditionally, preserving backward compatibility with callers that do not use the memory governor.

The governor is a runtime concern only — it is not serialized.

pub fn train( vectors: &[&[f32]], dim: usize, m: usize, k: usize, max_iter: usize, ) -> Self

Train PQ codebooks from a set of training vectors via k-means.

m = number of subvectors (must divide dim evenly). k = centroids per subvector (typically 256). max_iter = k-means iterations (20 is usually sufficient).

pub fn encode(&self, vector: &[f32]) -> Vec<u8>

Encode a vector: for each subvector, find the nearest centroid index.

This is a per-vector hot-path operation. Governor charging is intentionally skipped here to avoid atomic overhead on every candidate during search; use [encode_batch] for bulk encoding with budget enforcement.

pub fn encode_batch(&self, vectors: &[&[f32]]) -> Result<Vec<u8>, VectorError>

Batch encode all vectors into a contiguous byte array.

Charges m * vectors.len() bytes to the governor budget (if set) before allocating the output buffer. The guard is released at the end of this call — the buffer itself remains alive.

pub fn build_distance_table( &self, query: &[f32], ) -> Result<Vec<Vec<f32>>, VectorError>

Build an asymmetric distance table for a query vector.

Returns table[sub][centroid] = distance from query’s sub-vector to each centroid. Pre-computing this table makes distance evaluation O(M) per candidate instead of O(D).

Charges m * k * size_of::<f32>() bytes to the governor (if set) before allocating the table.

pub fn asymmetric_distance(&self, table: &[Vec<f32>], code: &[u8]) -> f32

Compute asymmetric distance using a precomputed distance table.

O(M) per candidate — just M table lookups and additions.

pub fn decode(&self, code: &[u8]) -> Result<Vec<f32>, VectorError>

Decode a PQ code back to an approximate FP32 vector.

Charges dim * size_of::<f32>() bytes to the governor (if set) before allocating the output buffer.

pub fn to_bytes(&self) -> Result<Vec<u8>, VectorError>

Serialize the codec to bytes with a versioned magic header.

Format: [NDPQ\0\0 (6 bytes)][version: u8 = 1][msgpack payload]

Charges the estimated serialized size to the governor (if set) before allocating the output buffer. The estimate is conservative: m * k * sub_dim * size_of::<f32>() + 64 (header + framing overhead).

pub fn from_bytes(bytes: &[u8]) -> Result<Self, VectorError>

Deserialize the codec from bytes produced by Self::to_bytes.

Returns VectorError::InvalidMagic if the header does not match NDPQ\0\0, and VectorError::UnsupportedVersion for unknown versions.

Trait Implementations

impl Clone for PqCodec

fn clone(&self) -> PqCodec

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PqCodec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for PqCodec

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<'__msgpack_de> FromMessagePack<'__msgpack_de> for PqCodec

fn read<R: Read<'__msgpack_de>>(reader: &mut R) -> Result<Self, Error>

where Self: Sized,

Reads the MessagePack representation of this value from the provided reader.

impl Serialize for PqCodec

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl ToMessagePack for PqCodec

fn write<W: Write>(&self, writer: &mut W) -> Result<(), Error>

Writes the MessagePack representation of this value into the provided writer.

impl VectorCodec for PqCodec

type Query = PqQuery

Prepared query: precomputed distance table + original FP32 query.

fn encode(&self, v: &[f32]) -> Self::Quantized

Encode an FP32 vector: one centroid index byte per subspace.

Panics

UnifiedQuantizedVector::new fails only when the outlier bitmask does not match the provided outlier slice. With outlier_bitmask = 0 and an empty slice this can never happen. The expect is therefore unreachable in practice.

fn prepare_query(&self, q: &[f32]) -> Self::Query

Prepare the query by precomputing the M×K asymmetric distance table.

The VectorCodec trait does not propagate errors. A PqCodec used via this trait path is created by PqCodec::train which sets no governor; build_distance_table therefore always returns Ok here. If a governor is attached and its budget is exhausted the caller that constructed the codec is responsible for handling the error — this impl panics with a descriptive message so the budget violation is never silently ignored.

fn adc_lut(&self, q: &Self::Query) -> Option<AdcLut>

Build the AdcLut from the precomputed distance table for use by SIMD rerank kernels (pshufb / vpermb).

fn fast_symmetric_distance( &self, q: &Self::Quantized, v: &Self::Quantized, ) -> f32

Symmetric distance between two PQ-encoded vectors.

Both codes are decoded to approximate FP32 vectors via the codebook, then the squared L2 difference is accumulated. This is the correct definition of symmetric PQ distance: each vector is approximated by its nearest centroids, and the distance is computed in FP32.

fn exact_asymmetric_distance(&self, q: &Self::Query, v: &Self::Quantized) -> f32

Asymmetric ADC distance: precomputed distance table vs stored code.

O(M) per candidate — delegates to PqCodec::asymmetric_distance.

type Quantized = PqQuantized

The packed quantized form. Must be convertible to a UnifiedQuantizedVector reference via AsRef.

fn train(&mut self, samples: &[&[f32]]) -> Result<(), CodecError>

Optional: fit the codec’s learned parameters on a set of training vectors.