Struct AiBatchClient 

pub struct AiBatchClient<S = AiTransportSender> { /* private fields */ }

Batch embedding client.

Generic over the backend so tests can inject mocks without HTTP. The default type parameter is AiTransportSender (production).

Implementations

impl AiBatchClient

pub fn new(transport: AiTransport) -> AiBatchClient

pub fn from_runtime(runtime: &RedDBRuntime) -> AiBatchClient

impl<S> AiBatchClient<S>

where S: SubBatchSender,

pub fn with_sender(sender: S) -> AiBatchClient<S>

Create with a custom backend (useful in tests).

pub fn with_max_batch_size(self, size: usize) -> AiBatchClient<S>

Override the max sub-batch size (defaults per provider if not set).

pub fn with_dedup_cache( self, cache: Arc<EmbeddingDedupCache>, ) -> AiBatchClient<S>

Enable dedup cache.

pub fn with_chunk_mode(self, mode: ChunkMode) -> AiBatchClient<S>

Set chunk mode (Single or Multi).

pub fn with_max_tokens(self, max: usize) -> AiBatchClient<S>

Set max tokens per chunk.

pub fn with_audit_log(self, audit_log: Arc<AuditLogger>) -> AiBatchClient<S>

pub async fn embed_batch( &self, provider: &AiProvider, model: &str, api_key: &str, texts: Vec<String>, ) -> Result<Vec<Vec<f32>>, AiTransportError>

Embed texts in batch. Returns one Vec<f32> per input in order. Empty/whitespace-only inputs yield an empty Vec<f32> at their position without consuming a provider request slot.

When dedup is enabled, previously-seen texts are served from cache and only unseen texts are sent to the provider. Duplicate texts within a single call are also deduplicated — the provider receives each unique text only once.

When chunking is enabled, texts exceeding max_tokens are chunked; in Single mode the first chunk is sent to the provider.