Struct EmbedderService 

pub struct EmbedderService<M> { /* private fields */ }

Background batching service for semantic embedding requests.

BatchItem values preserve their embedding role and optional title metadata. Dropping the returned result receiver cancels background processing.

Implementations

impl<M: Send + 'static> EmbedderService<M>

pub fn new( embedder: Arc<dyn EmbeddingProvider>, max_tokens: NonZeroUsize, max_batch_size: NonZeroUsize, workers: NonZeroUsize, ) -> Result<(Self, Receiver<Result<EmbeddingResult<M>>>)>

Creates a new embedding service and result stream.

Errors

Returns an error if no Tokio runtime is active.

This constructor spawns background tasks immediately.

pub async fn enqueue(&mut self, item: BatchItem<M>) -> Result<bool>

Adds one item into the service and flushes a batch if needed.

Errors

Returns Error::BatchChannelClosed after Self::flush has closed the service. Returns Error::BatchItemTooLarge when one item exceeds the configured hard token limit.

pub async fn flush(&mut self) -> Result<bool>

Flushes pending batched items and closes the batch input channel.

After this returns successfully, subsequent Self::enqueue calls return Error::BatchChannelClosed.