Skip to main content

embed_passages_controlled

sqlite_graphrag::embedder

Function embed_passages_controlled 

Source 
pub fn embed_passages_controlled(
    embedder: &Mutex<LlmEmbedding>,
    texts: &[&str],
    token_counts: &[usize],
) -> Result<Vec<Vec<f32>>, AppError>
Expand description

Embeds a batch of passages with token-count-aware batching. The token_counts are still used to keep the LLM invocation under the per-call context budget, but the count is now an approximation (whitespace-split words) since the tokenizers crate was removed.