Struct Embed

pub struct Embed { /* private fields */ }

A transformer that can generate embeddings for an Node

This file defines the Embed struct and its implementation of the BatchableTransformer trait.

Implementations

impl Embed

pub fn new(model: impl EmbeddingModel + 'static) -> Self

Creates a new instance of the Embed transformer.

Parameters

• model - An embedding model that implements the EmbeddingModel trait.

Returns

A new instance of Embed.

pub fn with_concurrency(self, concurrency: usize) -> Self

pub fn with_batch_size(self, batch_size: usize) -> Self

Sets the batch size for the transformer. If the batch size is not set, the transformer will use the default batch size set by the pipeline

Parameters

• batch_size - The batch size to use for the transformer.

Returns

A new instance of Embed.

Trait Implementations

impl BatchableTransformer for Embed

fn batch_transform<'life0, 'async_trait>( &'life0 self, nodes: Vec<Node>, ) -> Pin<Box<dyn Future<Output = IndexingStream> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Transforms a batch of Node objects by generating embeddings for them.

Parameters

• nodes - A vector of Node objects to be transformed.

Returns

An IndexingStream containing the transformed Node objects with their embeddings.

Errors

If the embedding process fails, the function returns a stream with the error.

fn concurrency(&self) -> Option<usize>

Overrides the default concurrency of the pipeline

fn batch_size(&self) -> Option<usize>

Overrides the default batch size of the pipeline

fn name(&self) -> &'static str

impl Clone for Embed

fn clone(&self) -> Embed

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Embed

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl WithBatchIndexingDefaults for Embed

fn with_indexing_defaults(&mut self, _indexing_defaults: IndexingDefaults)

impl WithIndexingDefaults for Embed

fn with_indexing_defaults(&mut self, _indexing_defaults: IndexingDefaults)