Struct AiBatchClient 

pub struct AiBatchClient<S = AiTransportSender> { /* private fields */ }

Batch embedding client.

Generic over the backend so tests can inject mocks without HTTP. The default type parameter is AiTransportSender (production).

Implementations

impl AiBatchClient<AiTransportSender>

pub fn new(transport: AiTransport) -> Self

pub fn from_runtime(runtime: &RedDBRuntime) -> Self

impl<S: SubBatchSender> AiBatchClient<S>

pub fn with_sender(sender: S) -> Self

Create with a custom backend (useful in tests).

pub fn with_max_batch_size(self, size: usize) -> Self

Override the max sub-batch size (defaults per provider if not set).

pub fn with_dedup_cache(self, cache: Arc<EmbeddingDedupCache>) -> Self

Enable dedup cache.

pub fn with_chunk_mode(self, mode: ChunkMode) -> Self

Set chunk mode (Single or Multi).

pub fn with_max_tokens(self, max: usize) -> Self

Set max tokens per chunk.

pub fn with_audit_log(self, audit_log: Arc<AuditLogger>) -> Self

pub async fn embed_batch( &self, provider: &AiProvider, model: &str, api_key: &str, texts: Vec<String>, ) -> Result<Vec<Vec<f32>>, AiTransportError>

Embed texts in batch. Returns one Vec<f32> per input in order. Empty/whitespace-only inputs yield an empty Vec<f32> at their position without consuming a provider request slot.

When dedup is enabled, previously-seen texts are served from cache and only unseen texts are sent to the provider. Duplicate texts within a single call are also deduplicated — the provider receives each unique text only once.

When chunking is enabled, texts exceeding max_tokens are chunked; in Single mode the first chunk is sent to the provider.