Function embed_passages_controlled 

pub fn embed_passages_controlled(
    embedder: &Mutex<TextEmbedding>,
    texts: &[&str],
    token_counts: &[usize],
) -> Result<Vec<Vec<f32>>, AppError>

Embeds passages grouped into token-budget-aware batches to avoid OOM on variable-length inputs.

texts and token_counts must have the same length. Batches are planned using an internal budget algorithm and single-item batches fall back to embed_passage.

Errors

Returns Err when lengths differ, the mutex is poisoned, or inference fails.