Struct OnnxEmbeddingEngine 

pub struct OnnxEmbeddingEngine { /* private fields */ }

ONNX-based embedding engine for local inference

Wraps ONNX Runtime session and HuggingFace tokenizer. Based on examples/embeddings.rs with proper tokenization for Python parity.

Implementations

impl OnnxEmbeddingEngine

pub fn new( config: OnnxEmbeddingConfig, ) -> Result<OnnxEmbeddingEngine, EmbeddingError>

Create a new ONNX embedding engine

Initializes ONNX Runtime, loads the model, and downloads/caches the tokenizer.

Arguments

• config - Engine configuration with model path and tokenizer model ID

Returns

• Initialized engine or error

Errors

• Returns error if model file not found, ONNX Runtime init fails, or tokenizer download fails

Example

let config = OnnxEmbeddingConfig::bge_small("./target/models");
let engine = OnnxEmbeddingEngine::new(config)?;

pub async fn with_auto_download( config: OnnxEmbeddingConfig, ) -> Result<OnnxEmbeddingEngine, EmbeddingError>

Create a new ONNX embedding engine with automatic model downloading

Downloads model and tokenizer from HuggingFace Hub if not found locally. This is the recommended constructor for most use cases.

Arguments

• config - Engine configuration with model path and tokenizer model ID

Returns

• Initialized engine or error

Errors

• Returns error if download fails, ONNX Runtime init fails, or tokenizer load fails

Example

let config = OnnxEmbeddingConfig::bge_small("./target/models");
let engine = OnnxEmbeddingEngine::with_auto_download(config).await?;

Trait Implementations

impl EmbeddingEngine for OnnxEmbeddingEngine

fn embed<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, texts: &'life1 [&'life2 str], ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>, EmbeddingError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, OnnxEmbeddingEngine: 'async_trait,

Embed texts, splitting into config.batch_size sub-batches so the ONNX session never receives an unbounded batch. A transformer’s activation memory scales with batch × seq_len²; embedding a whole corpus in one call (several thousand chunks) would allocate tens of GB and OOM. Sub-batching keeps peak memory flat regardless of how many texts are passed.

fn dimension(&self) -> usize

Get the dimensionality of embeddings produced by this engine

fn batch_size(&self) -> usize

Get the optimal batch size for this engine

fn max_sequence_length(&self) -> usize

Get the maximum sequence length (in tokens) supported