Module embeddings 

Semantic Embeddings

Vector embeddings for semantic similarity search and enhanced context retrieval.

This module provides:

• Text embedding generation (local and remote)
• Vector similarity calculations
• Semantic-enhanced episode and pattern retrieval
• Configurable embedding providers

Architecture

The embedding system supports multiple providers:

• Local: sentence-transformers via candle-transformers (offline)
• OpenAI: text-embedding-ada-002 and text-embedding-3.x (cloud)
• Mistral: mistral-embed and codestral-embed (cloud)
• Azure OpenAI: Azure-hosted OpenAI embeddings
• Custom: User-provided embedding functions

Usage

use do_memory_core::embeddings::{EmbeddingProvider, LocalEmbeddingProvider, ProviderConfig, LocalConfig};

// Local embedding provider (offline)
let config = LocalConfig::default();
let provider = LocalEmbeddingProvider::new(config).await?;

// Generate embedding for text
let embedding = provider.embed_text("implement REST API").await?;

// Calculate similarity between two texts
let similarity = provider.similarity("REST API", "web service API").await?;

Re-exports

pub use config::AzureOpenAIConfig;

pub use config::CustomConfig;

pub use config::EmbeddingConfig;

pub use config::LocalConfig;

pub use config::MistralConfig;

pub use config::OpenAIConfig;

pub use config::OptimizationConfig;

pub use config::ProviderConfig;

Modules

config

Configuration for embedding providers

Structs

CircuitBreaker

Circuit breaker to prevent cascading failures

CircuitBreakerConfig

Circuit breaker configuration

EmbeddingResult

Result from embedding generation

EmbeddingStorage

Wrapper around existing storage backends to add embedding support

InMemoryEmbeddingStorage

In-memory embedding storage for testing and fallback

LatencyTimer

Helper to measure latency

LocalEmbeddingProvider

Local embedding provider using sentence transformers

MetricsSnapshot

Snapshot of metrics at a point in time

MockLocalModel

Mock implementation for local embedding model Intended for testing only - in production should not be used

ProviderMetrics

Performance metrics for an embedding provider

SemanticService

Main semantic embedding service for the memory system

SimilarityMetadata

Metadata about a similarity match

SimilaritySearchResult

Result from similarity search containing the item and similarity score

Enums

CircuitBreakerState

Circuit breaker result

LocalModelUseCase

Re-export utilities from the utils module Use cases for local model selection

Constants

DEFAULT_EMBEDDING_DIM

Default embedding dimension for sentence transformers

Traits

EmbeddingProvider

Trait for embedding providers that convert text to vectors

EmbeddingStorageBackend

Trait for embedding storage backends

Functions

cosine_similarity

Calculate cosine similarity between two vectors

get_recommended_model

Re-export utilities from the utils module Get recommended model configuration for different use cases

list_available_models

Re-export utilities from the utils module List available local models

normalize_vector

Normalize a vector to unit length