Trait Embedder

Source

pub trait Embedder: Send + Sync {
    // Required methods
    fn dim(&self) -> usize;
    fn embed(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, HostError>;
}

Expand description

Turns texts into embedding vectors. Batched by design — providers price and perform far better on batches.

embed takes &self, and the trait requires Sync, because an embedder is a client of a remote service, not a piece of mutable state. Every caller in this workspace shares one instance across threads (a database’s writer, the napi binding’s libuv workers, the MCP worker pool), and a &mut self signature forced every one of them to put a Mutex in front of it. That mutex serialized the HTTP round trips: four concurrent recalls against a 300 ms provider took 1200 ms, with the provider seeing one request at a time. With &self they take 300 ms and the provider sees four.

An implementation that genuinely needs mutable state (a local cache, a rate-limit budget) brings its own interior mutability, which is the right place for it: only that implementation knows what may overlap and what may not. OpenAiCompatEmbedder needs none — a ureq::Agent is a connection-pool handle built for concurrent use.