Struct IvfPqIndex 

pub struct IvfPqIndex { /* private fields */ }

IVF-PQ index: inverted file with product quantization.

Implementations

impl IvfPqIndex

pub fn to_bytes(&self) -> Result<Vec<u8>, VectorError>

Encode the whole index as MessagePack.

pub fn from_bytes( bytes: &[u8], memory: ScopedMemory, ) -> Result<Self, VectorError>

Decode an index written by Self::to_bytes, charging the PQ codec to memory.

Fails with VectorError::CheckpointDeserializationError when the bytes do not decode or the decoded cells disagree with the dimension, the PQ code width, or the centroid count.

impl IvfPqIndex

pub fn new(dim: usize, params: IvfPqParams) -> Self

Create an empty, untrained IVF-PQ index.

pub fn train( &mut self, vectors: &[&[f32]], memory: ScopedMemory, ) -> Result<(), VectorError>

Train the coarse centroids and the PQ codebooks on vectors, tracking PQ codebook allocations against memory.

Fails with VectorError::InvalidInput when the set is empty, the dimension is zero or not divisible by pq_m, the set is smaller than pq_k, or the index already holds vectors (their codes would not match new codebooks). A vector without the index dimension fails with VectorError::DimensionMismatch. A failed training changes nothing.

pub fn insert_with_id( &mut self, id: u32, vector: Vec<f32>, ) -> Result<(), VectorError>

Add vector under the caller-assigned id.

Fails with VectorError::DimensionMismatch for a vector without the index dimension, and with VectorError::InvalidInput when the index is untrained or already holds id. A failed add changes nothing.

pub fn add(&mut self, vector: &[f32]) -> Result<u32, VectorError>

Add a vector under the next free id. Returns the id.

Fails as Self::insert_with_id does.

pub fn add_batch(&mut self, vectors: &[&[f32]]) -> Result<(), VectorError>

Add vectors under consecutive free ids. Stops at the first vector that fails to add.

pub fn is_trained(&self) -> bool

Whether the index holds a trained codebook.

pub fn contains(&self, id: u32) -> bool

Whether the index holds id, live or soft-deleted.

pub fn is_deleted(&self, id: u32) -> bool

Whether id is held and soft-deleted.

pub fn delete(&mut self, id: u32) -> bool

Soft-delete id. false when the index does not hold it live.

pub fn undelete(&mut self, id: u32) -> bool

Reverse a soft delete of id. false when id was not deleted.

pub fn get_vector(&self, id: u32) -> Option<&[f32]>

The FP32 vector of a live id.

pub fn live_vectors(&self) -> Vec<(u32, Vec<f32>)>

Every live entry as (id, vector), ordered by id.

pub fn roll_back_to(&mut self, next_id: u32)

Drop every entry with id next_id or later, so the next Self::add takes next_id again. The training stays. A rollback uses it to withdraw the newest adds.

pub fn compact(&mut self) -> usize

Remove every soft-deleted entry. Returns the number removed. Ids of the remaining entries do not change.

pub fn len(&self) -> usize

Entries held, live or soft-deleted.

pub fn live_count(&self) -> usize

Entries held and not soft-deleted.

pub fn tombstone_count(&self) -> usize

Soft-deleted entries held.

pub fn is_empty(&self) -> bool

pub fn dim(&self) -> usize

pub fn n_cells(&self) -> usize

pub fn params(&self) -> &IvfPqParams

pub fn trained_on(&self) -> usize

Vectors the codebooks were trained on. 0 when untrained.

pub fn trained_at_ms(&self) -> u64

Unix milliseconds the caller stamped the training with.

pub fn set_trained_at_ms(&mut self, ms: u64)

Stamp the training time, in Unix milliseconds.

pub fn memory_bytes(&self) -> usize

Approximate heap bytes of the centroids, codes and vectors.

impl IvfPqIndex

pub fn search( &self, query: &[f32], top_k: usize, ) -> Result<Vec<SearchResult>, VectorError>

The top_k live entries nearest to query under the index metric.

A query without the index dimension fails with VectorError::DimensionMismatch. A distance table over the memory budget fails the search: skipping its cell would drop results.

pub fn search_with( &self, query: &[f32], top_k: usize, metric: DistanceMetric, filter: Option<&RoaringBitmap>, ) -> Result<Vec<SearchResult>, VectorError>

The top_k live entries nearest to query under metric, keeping only ids in filter when one is given. Fails as Self::search.