Skip to main content

embed_texts_parallel

Function embed_texts_parallel 

Source
pub fn embed_texts_parallel(
    embedder: &Mutex<LlmEmbedding>,
    texts: &[String],
    parallelism: usize,
    batch_size: usize,
) -> Result<Vec<Vec<f32>>, AppError>
Expand description

G42/S3 core: bounded parallel batch embedding.

  • texts are grouped into batches of batch_size (one LLM call per batch, G42/S2);
  • at most effective_permits(parallelism) LLM subprocesses run simultaneously (Arc<Semaphore> + acquire_owned, BLOCO 2);
  • results stream through a BOUNDED mpsc channel so the caller-side collector applies backpressure and can persist incrementally (BLOCO 5);
  • the global CancellationToken aborts in-flight work on the first signal; subprocesses die with their futures via kill_on_drop (BLOCO 6).