Struct EmbeddingService 

pub struct EmbeddingService { /* private fields */ }

Embedding service with Candle inference and LRU caching.

Implementations

impl EmbeddingService

pub fn new(model_dir: &Path) -> Result<Self, CodememError>

Create a new embedding service, loading model from the given directory. Expects model.safetensors, config.json, and tokenizer.json in the directory.

pub fn default_model_dir() -> PathBuf

Get the default model directory path (~/.codemem/models/{MODEL_NAME}).

pub fn download_model(dest_dir: &Path) -> Result<PathBuf, CodememError>

Download the model from HuggingFace Hub to the given directory. Returns the directory path. No-ops if model already exists.

pub fn embed(&self, text: &str) -> Result<Vec<f32>, CodememError>

Embed a single text string. Returns a 768-dim L2-normalized vector. Uses LRU cache for repeated queries.

pub fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, CodememError>

Embed a batch of texts with cache-aware batching.

Checks the LRU cache first and only runs the model on uncached texts, using a true batched forward pass (single GPU/CPU kernel launch per chunk).

pub fn cache_stats(&self) -> (usize, usize)

Get cache statistics: (current_size, capacity).

Trait Implementations

impl EmbeddingProvider for EmbeddingService

fn dimensions(&self) -> usize

Embedding vector dimensions.

fn embed(&self, text: &str) -> Result<Vec<f32>, CodememError>

Embed a single text string.

fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, CodememError>

Embed a batch of texts (default: sequential).

fn name(&self) -> &str

Provider name for display.

fn cache_stats(&self) -> (usize, usize)

Cache statistics: (current_size, capacity). Returns (0, 0) if no cache.