Trait VectorStore 

pub trait VectorStore:
    Send
    + Sync
    + Sized
    + Clone {
    type DistanceCalculator<'a>: DistCalculator
       where Self: 'a;

    // Required methods
    fn as_any(&self) -> &dyn Any;
    fn schema(&self) -> &SchemaRef;
    fn to_batches(&self) -> Result<impl Iterator<Item = RecordBatch> + Send>;
    fn len(&self) -> usize;
    fn distance_type(&self) -> DistanceType;
    fn row_id(&self, id: u32) -> u64;
    fn row_ids(&self) -> impl Iterator<Item = &u64>;
    fn append_batch(
        &self,
        batch: RecordBatch,
        vector_column: &str,
    ) -> Result<Self>;
    fn dist_calculator(
        &self,
        query: ArrayRef,
        dist_q_c: f32,
    ) -> Self::DistanceCalculator<'_>;
    fn dist_calculator_from_id(&self, id: u32) -> Self::DistanceCalculator<'_>;

    // Provided methods
    fn is_empty(&self) -> bool { ... }
    fn dist_between(&self, u: u32, v: u32) -> f32 { ... }
}

Vector Storage is the abstraction to store the vectors.

It can be in-memory or on-disk, raw vector or quantized vectors.

It abstracts away the logic to compute the distance between vectors.

TODO: should we rename this to “VectorDistance”?;

Internal API

API stability is not guaranteed

Required Associated Types

type DistanceCalculator<'a>: DistCalculator where Self: 'a

Required Methods

fn as_any(&self) -> &dyn Any

fn schema(&self) -> &SchemaRef

fn to_batches(&self) -> Result<impl Iterator<Item = RecordBatch> + Send>

fn len(&self) -> usize

fn distance_type(&self) -> DistanceType

Return DistanceType.

fn row_id(&self, id: u32) -> u64

Get the lance ROW ID from one vector.

fn row_ids(&self) -> impl Iterator<Item = &u64>

fn append_batch(&self, batch: RecordBatch, vector_column: &str) -> Result<Self>

Append Raw RecordBatch into the Storage. The storage implement will perform quantization if necessary.

fn dist_calculator( &self, query: ArrayRef, dist_q_c: f32, ) -> Self::DistanceCalculator<'_>

Create a DistCalculator to compute the distance between the query.

Using dist calculator can be more efficient as it can pre-compute some values.

fn dist_calculator_from_id(&self, id: u32) -> Self::DistanceCalculator<'_>

Provided Methods

fn is_empty(&self) -> bool

Returns true if this graph is empty.

fn dist_between(&self, u: u32, v: u32) -> f32

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl VectorStore for RabitQuantizationStorage

type DistanceCalculator<'a> = RabitDistCalculator<'a>

impl VectorStore for FlatBinStorage

type DistanceCalculator<'a> = FlatDistanceCal<'a, UInt8Type>

impl VectorStore for FlatFloatStorage

type DistanceCalculator<'a> = FlatDistanceCal<'a, Float32Type>

impl VectorStore for ProductQuantizationStorage

type DistanceCalculator<'a> = PQDistCalculator

impl VectorStore for ScalarQuantizationStorage

type DistanceCalculator<'a> = SQDistCalculator<'a>