Struct NativeEmbeddingService 

pub struct NativeEmbeddingService { /* private fields */ }

Unstable: model-loading API still evolving; signature may change as lattice-inference matures.

Pure Rust embedding service backed by lattice-inference.

Uses SIMD-accelerated matrix multiplication and safetensors weight loading. No ONNX Runtime, no C++ FFI, no fastembed dependency.

Supports both encoder (BERT/BGE) and decoder (Qwen3) architectures.

Cancellation Safety

Model loading uses std::sync::OnceLock + spawn_blocking instead of tokio::sync::OnceCell. This is critical because tokio::sync::OnceCell:: get_or_try_init resets when the calling future is dropped (e.g., client disconnect during MCP timeout). With OnceLock, the blocking task runs to completion and stores the result regardless of async cancellation, so the model only loads once per process lifetime.

Implementations

impl NativeEmbeddingService

pub fn new() -> Self

Unstable: constructor signature may change; use EmbeddingService trait for stable API.

pub fn with_model(model_type: EmbeddingModel) -> Self

Unstable: constructor signature may change; use EmbeddingService trait for stable API.

pub fn with_model_config(model_config: ModelConfig) -> Result<Self>

Unstable: create with explicit model config (model + optional MRL truncation dim).

pub fn with_model_from_env(model_type: EmbeddingModel) -> Result<Self>

Unstable: create with model config read from LATTICE_EMBED_DIM env var.

pub fn save_cache(&self) -> Result<usize>

Unstable: persistence API may be moved to a separate manager type.

pub fn cache_size(&self) -> usize

Unstable: internal diagnostic; may be removed or moved to metrics.

Trait Implementations

impl Default for NativeEmbeddingService

fn default() -> Self

Returns the “default value” for a type.

impl EmbeddingService for NativeEmbeddingService

fn embed<'life0, 'life1, 'async_trait>( &'life0 self, texts: &'life1 [String], model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<Vec<f32>>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate embeddings for multiple texts.

fn model_config(&self, model: EmbeddingModel) -> ModelConfig

Unstable: returns the effective ModelConfig for a given model on this service.

fn supports_model(&self, model: EmbeddingModel) -> bool

Stable: check if the service supports a given model.

fn name(&self) -> &'static str

Stable: get the name/identifier of this service.

fn embed_one<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, model: EmbeddingModel, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stable: generate an embedding for a single text.