Module builder 

EmbeddingsBuilder — batch embedding with automatic chunking.

The builder collects documents, extracts their texts via Embed, chunks them to respect MAX_DOCUMENTS, calls the model in sequential batches, then reassembles the results back to the originating document.

Why sequential batching?

Rig uses buffer_unordered(10) from the futures crate to parallelise batch requests. irig deliberately avoids the futures dependency and runs batches sequentially instead. On ICP, HTTP outcalls are already individually async — if you need parallelism at the application level, issue multiple agent/embedding calls from your canister code.

Example

let results: Vec<(Article, Vec<Embedding>)> =
    EmbeddingsBuilder::new(model)
        .documents(articles)?
        .build()
        .await?;

for (article, embeddings) in results {
    // embeddings[0] = title vector, embeddings[1] = body vector
    store.upsert(article.id, embeddings);
}

Structs

EmbeddingsBuilder

Accumulates documents, embeds them in efficient batches, and returns each document paired with its embedding(s).