Struct LocalEmbeddingProvider 

pub struct LocalEmbeddingProvider { /* private fields */ }

Local embedding provider for offline and privacy-focused deployments

Supports local inference with Hugging Face models or custom implementations. Ideal for:

• Privacy-sensitive applications
• High-volume processing without API costs
• Offline or air-gapped environments
• Custom fine-tuned models

Popular Models

• sentence-transformers/all-MiniLM-L6-v2: 384 dims, fast and lightweight
• sentence-transformers/all-mpnet-base-v2: 768 dims, high quality
• sentence-transformers/all-distilroberta-v1: 768 dims, balanced

Example

use rrag::prelude::*;
use std::sync::Arc;

let provider = Arc::new(LocalEmbeddingProvider::new(
    "sentence-transformers/all-MiniLM-L6-v2",
    384
));

let service = EmbeddingService::new(provider);
let embeddings = service.embed_documents(&[
    Document::new("First document"),
    Document::new("Second document")
]).await?;

assert_eq!(embeddings.len(), 2);

Performance Notes

• Uses parallel processing for batch operations
• Smaller batch sizes recommended (16-32) for memory efficiency
• CPU-intensive; consider GPU acceleration for large workloads

Implementations

impl LocalEmbeddingProvider

pub fn new(model_path: impl Into<String>, dimensions: usize) -> Self

Create a new local embedding provider

Trait Implementations

impl EmbeddingProvider for LocalEmbeddingProvider

fn name(&self) -> &str

Provider name (e.g., “openai”, “huggingface”)

fn supported_models(&self) -> Vec<&str>

Supported models for this provider

fn max_batch_size(&self) -> usize

Maximum batch size supported

fn embedding_dimensions(&self) -> usize

Embedding dimensions for the current model

fn embed_text<'life0, 'life1, 'async_trait>( &'life0 self, text: &'life1 str, ) -> Pin<Box<dyn Future<Output = RragResult<Embedding>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generate embedding for a single text

fn embed_batch<'life0, 'async_trait>( &'life0 self, requests: Vec<EmbeddingRequest>, ) -> Pin<Box<dyn Future<Output = RragResult<EmbeddingBatch>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate embeddings for multiple texts (more efficient)

fn health_check<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = RragResult<bool>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Health check for the provider