Struct BatchProcessor 

pub struct BatchProcessor { /* private fields */ }

Represents a batch of texts to be processed.

Implementations

impl BatchProcessor

pub fn new(max_batch_size: usize) -> Self

Creates a new batch processor with the specified maximum batch size.

Arguments

• max_batch_size - Maximum number of texts to process in a single batch

Returns

Returns a new BatchProcessor instance.

pub fn builder() -> BatchProcessorBuilder

Creates a batch processor with custom configuration.

pub fn process_batch( &self, model: &mut EmbeddingModel, texts: &[&str], ) -> Result<Vec<Vec<f32>>>

Processes a batch of texts to generate embeddings.

This function implements the following pipeline:

1. Parallel tokenization of all texts (using rayon)
2. True batch model inference (multiple sequences in single pass)
3. Parallel post-processing and normalization (handled by model)

Arguments

• model - The embedding model to use
• texts - Vector of texts to process

Returns

Returns a vector of embedding vectors, maintaining the input order.

Errors

This function will return an error if:

• Any text fails tokenization
• Model inference fails
• Memory allocation fails

pub fn set_progress_callback<F>(&mut self, callback: F)

where F: Fn(usize, usize) + Send + Sync + 'static,

Sets a progress callback for batch processing.

The callback will be called with (current_index, total_count) during processing.

Arguments

• callback - A function that receives progress updates