Struct Embedder 

pub struct Embedder { /* private fields */ }

Text embedding generator using nomic-embed-text-v1.5

Automatically downloads the model from HuggingFace Hub on first use. Detects GPU availability and uses CUDA/TensorRT when available.

Example

use cqs::Embedder;

let mut embedder = Embedder::new()?;
let embedding = embedder.embed_query("parse configuration file")?;
println!("Embedding dimension: {}", embedding.len()); // 768

Implementations

impl Embedder

pub fn new() -> Result<Self, EmbedderError>

Create a new embedder, downloading the model if necessary

Automatically detects GPU and uses CUDA/TensorRT when available. Falls back to CPU if no GPU is found.

Note: ONNX session is lazy-loaded on first embedding request (~500ms).

pub fn new_cpu() -> Result<Self, EmbedderError>

Create a CPU-only embedder

Use this for single-query embedding where CPU is faster than GPU due to CUDA context setup overhead. GPU only helps for batch embedding.

pub fn token_count(&self, text: &str) -> Result<usize, EmbedderError>

Count tokens in a text

pub fn split_into_windows( &self, text: &str, max_tokens: usize, overlap: usize, ) -> Result<Vec<(String, u32)>, EmbedderError>

Split text into overlapping windows of max_tokens with overlap tokens of context. Returns Vec of (window_content, window_index). If text fits in max_tokens, returns single window with index 0.

pub fn embed_documents( &mut self, texts: &[&str], ) -> Result<Vec<Embedding>, EmbedderError>

Embed documents (code chunks). Adds “passage: “ prefix for E5.

pub fn embed_query(&mut self, text: &str) -> Result<Embedding, EmbedderError>

Embed a query. Adds “query: “ prefix for E5. Uses LRU cache for repeated queries.

pub fn provider(&self) -> ExecutionProvider

Get the execution provider being used

pub fn batch_size(&self) -> usize

Get the batch size

pub fn warm(&mut self) -> Result<(), EmbedderError>

Warm up the model with a dummy inference