Struct OnnxEmbedder 

pub struct OnnxEmbedder { /* private fields */ }

Implementations

impl OnnxEmbedder

pub fn new() -> Result<Self>

pub fn with_model( name: &str, model_url: &str, tokenizer_url: &str, dimension: usize, output_tensor_name: &str, ) -> Result<Self>

pub fn with_model_and_data( name: &str, model_url: &str, model_data_url: Option<&str>, tokenizer_url: &str, dimension: usize, output_tensor_name: &str, use_token_type_ids: bool, ) -> Result<Self>

pub async fn warmup(self: &Arc<Self>) -> Result<()>

Eagerly load the ONNX session so the first embed() call doesn’t pay the cold-start penalty. Must be called from an async context (uses spawn_blocking internally since ONNX init creates a mini runtime).

pub fn try_unload_if_idle(&self) -> bool

Drops the ONNX session if it has been idle for longer than the timeout, freeing ~240 MB RSS. The LRU embedding cache is preserved. The session is re-initialised transparently on the next embed() call.

pub async fn maintenance_tick(self: &Arc<Self>)

Periodic maintenance entry-point. Call from a tokio interval timer.

Trait Implementations

impl Debug for OnnxEmbedder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Embedder for OnnxEmbedder

Available on crate feature real-embeddings only.

fn dimension(&self) -> usize

Returns the embedding dimension.

fn embed(&self, text: &str) -> Result<Vec<f32>>

Generates an embedding vector for the given text.

fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>

Generates embedding vectors for multiple texts in a single call. The default implementation calls embed() in a loop; backends that support true batched inference (e.g. ONNX) override this for better throughput.