Skip to main content

Codec

Trait Codec 

Source
pub trait Codec<S: EmbeddingSpace> {
    type Encoded: Clone;
    type EncodeRef<'a>: OpRef
       where Self: 'a;
    type DecodeRef<'a>: OpRef
       where Self: 'a;
    type TrainRef<'a>: OpRef
       where Self: 'a;
    type ObserveRef<'a>: OpRef
       where Self: 'a;

    // Required methods
    fn encode(&mut self, embedding: &S::EmbeddingData) -> Self::EncodeRef<'_>;
    fn encode_batch(
        &mut self,
        embeddings: &[S::EmbeddingData],
    ) -> Vec<Self::EncodeRef<'_>>;
    fn decode(&self, encoded: &Self::Encoded) -> Self::DecodeRef<'_>;
    fn decode_batch(
        &self,
        encoded: &[Self::Encoded],
    ) -> Vec<Self::DecodeRef<'_>>;
    fn code_size(&self) -> Option<usize>;
    fn train(&mut self, embeddings: &[S::EmbeddingData]) -> Self::TrainRef<'_>;
    fn observe(&mut self, embedding: &S::EmbeddingData) -> Self::ObserveRef<'_>;
    fn observe_batch(
        &mut self,
        embeddings: &[S::EmbeddingData],
    ) -> Vec<Self::ObserveRef<'_>>;
    fn is_trained(&self) -> bool;
}
Expand description

Encode, decode, and optionally train a compressed embedding representation.

Codec is the data-plane counterpart to Index: an index stores and retrieves vectors, while a codec compresses them. If a codec also participates in gossip, the gossip protocol wraps the codec and implements OverlayProtocol separately — separation of data-plane from control-plane.

Every operation returns an OpRef handle rather than a bare result, mirroring the pattern used by Index. This lets the same trait describe both in-process codecs (where the handle completes synchronously) and networked codecs (where the handle tracks a remote RPC).

Like Index, Codec includes training and observation methods directly. Codecs that don’t need training should make train and observe no-ops and return true from is_trained.

§Associated type families

Required Associated Types§

Source

type Encoded: Clone

The compressed representation produced by encode.

Source

type EncodeRef<'a>: OpRef where Self: 'a

Handle to an in-flight encode operation.

Source

type DecodeRef<'a>: OpRef where Self: 'a

Handle to an in-flight decode operation.

Source

type TrainRef<'a>: OpRef where Self: 'a

Handle to an in-flight train operation.

Source

type ObserveRef<'a>: OpRef where Self: 'a

Handle to an in-flight observe operation.

Required Methods§

Source

fn encode(&mut self, embedding: &S::EmbeddingData) -> Self::EncodeRef<'_>

Compress a single embedding.

Source

fn encode_batch( &mut self, embeddings: &[S::EmbeddingData], ) -> Vec<Self::EncodeRef<'_>>

Compress a batch of embeddings.

Source

fn decode(&self, encoded: &Self::Encoded) -> Self::DecodeRef<'_>

Reconstruct an embedding from its compressed form.

Source

fn decode_batch(&self, encoded: &[Self::Encoded]) -> Vec<Self::DecodeRef<'_>>

Reconstruct a batch of embeddings from their compressed forms.

Source

fn code_size(&self) -> Option<usize>

Returns the fixed byte-size of an encoded vector, if the encoding is fixed-size. Variable-length encodings return None.

Source

fn train(&mut self, embeddings: &[S::EmbeddingData]) -> Self::TrainRef<'_>

Batch training from a set of embeddings.

Codecs that don’t learn from data should make this a no-op and return true from is_trained.

Source

fn observe(&mut self, embedding: &S::EmbeddingData) -> Self::ObserveRef<'_>

Online/incremental update from a single observation.

Source

fn observe_batch( &mut self, embeddings: &[S::EmbeddingData], ) -> Vec<Self::ObserveRef<'_>>

Online/incremental update from a batch of observations.

Source

fn is_trained(&self) -> bool

Whether the codec has been trained and is ready to encode/decode.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§