Struct SemanticDeduplicationEngine 

pub struct SemanticDeduplicationEngine { /* private fields */ }

Main semantic deduplication engine with production-ready safety and concurrency controls

Implementations

impl SemanticDeduplicationEngine

pub fn new( config: SemanticDeduplicationConfig, repository: Arc<MemoryRepository>, embedding_service: Arc<dyn EmbeddingService>, ) -> Self

pub async fn deduplicate_batch( &self, memory_ids: &[Uuid], ) -> Result<DeduplicationResult>

Perform semantic deduplication on a batch of memories with distributed locking and transaction safety

pub fn calculate_cosine_similarity(&self, a: &Vector, b: &Vector) -> Result<f32>

Calculate cosine similarity between two embeddings

pub async fn run_auto_pruning(&self) -> Result<PruningResult>

Run auto-pruning based on recall probability thresholds

pub async fn maintain_memory_headroom( &self, ) -> Result<HeadroomMaintenanceResult>

Check and maintain memory headroom

pub async fn get_memory_statistics(&self) -> Result<MemoryStatistics>

Get comprehensive memory statistics

pub async fn get_metrics(&self) -> DeduplicationMetrics

Get current deduplication metrics

pub async fn deduplicate_large_batch( &self, memory_ids: &[Uuid], ) -> Result<DeduplicationResult>

Process large memory batches with intelligent batching and concurrency control