Struct CandleEmbeddingEngine 

pub struct CandleEmbeddingEngine { /* private fields */ }

Pure-Rust embedding engine using candle for BERT inference.

Uses candle-transformers’ BERT implementation with tokenizers for tokenisation. No C/C++ FFI dependencies — compiles on all platforms.

A dedicated OS thread owns the model exclusively. Async callers send work via a channel and await a oneshot reply. If a call times out, the caller gets an error immediately; the worker finishes its current task, discards the stale reply channel, and picks up the next request — no restart needed.

Implementations

impl CandleEmbeddingEngine

pub fn new( embed_timeout: Duration, queue_size: usize, reporter: SubsystemReporter, ) -> Result<Self, MemoryError>

Initialise the candle embedding engine.

Downloads model weights from HuggingFace Hub on first use (cached in the standard HF cache directory, respects HF_HOME).

embed_timeout caps how long a single embed call may block. If the worker is still running when the timeout fires, the caller gets an error but the engine recovers automatically — no restart needed.

queue_size sets the bounded channel capacity — how many requests can queue behind the one being processed. Extra callers get an immediate “busy” error.

reporter receives report_ok/report_err after each embed call so the /readyz handler can reflect the engine’s operational state without active probing.

Trait Implementations

impl Drop for CandleEmbeddingEngine

fn drop(&mut self)

Executes the destructor for this type.

impl EmbeddingBackend for CandleEmbeddingEngine

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Embed a batch of texts, returning one vector per input.

fn dimensions(&self) -> usize

Number of dimensions produced by the model.

fn embed_one<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>, MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Convenience: embed a single text.