Crate do_memory_core 

Memory Core

Core data structures and types for the self-learning memory system with episodic learning.

This crate provides the fundamental building blocks for AI agents to learn from execution:

Core Concepts

• Episodes: Complete task execution records with steps, outcomes, and metadata
• Patterns: Reusable patterns extracted from episodes using statistical analysis
• Heuristics: Learned condition-action rules for future decision-making
• Reflections: Generated insights after episode completion
• Rewards: Quantitative scoring of episode success

Module Organization

Primary APIs

• memory: Main orchestrator for the learning cycle
• episode: Episode creation, logging, and management
• patterns: Pattern extraction, clustering, and validation
• embeddings: Semantic embedding generation and similarity search

Support Modules

• types: Common types used across the system
• storage: Storage backend abstractions (persistence layer)
• pre_storage: Pre-storage ingest layer (quality, normalization, validation)
• retrieval: Memory retrieval and caching
• search: Search and ranking functionality
• security: Audit logging and security
• monitoring: Agent performance monitoring

Data Pipeline

Episode → [pre_storage: ingest] → [storage: persistence] → Backend (Turso/redb)

• pre_storage: Sanitize, enrich, normalize, validate episodes
• storage: Backend traits, repository interfaces, query construction

Quick Start

Local Development (No Credentials)

Use do-memory-storage-turso for durable local storage:

use do_memory_core::SelfLearningMemory;

// In-memory storage for quick testing (no dependencies)
let memory = SelfLearningMemory::new();

For local SQLite storage, see the do_memory_storage_turso::SelfLearningMemoryExt trait.

Basic Episode Recording

use do_memory_core::memory::SelfLearningMemory;
use do_memory_core::{TaskContext, TaskType, TaskOutcome, ExecutionStep, ExecutionResult};

#[tokio::main]
async fn main() {
    let memory = SelfLearningMemory::new();

    // 1. Start an episode for a task
    let context = TaskContext::default();
    let episode_id = memory.start_episode(
        "Implement authentication".to_string(),
        context,
        TaskType::CodeGeneration,
    ).await;

    // 2. Log execution steps
    let mut step = ExecutionStep::new(
        1,
        "analyzer".to_string(),
        "Analyze requirements".to_string()
    );
    step.result = Some(ExecutionResult::Success {
        output: "Requirements clear".to_string()
    });
    memory.log_step(episode_id, step).await;

    // 3. Complete episode with learning
    memory.complete_episode(episode_id, TaskOutcome::Success {
        verdict: "Auth implemented successfully".to_string(),
        artifacts: vec!["auth.rs".to_string()],
    }).await.unwrap();

    // 4. Retrieve relevant context for future tasks
    let relevant = memory.retrieve_relevant_context(
        "Add authorization".to_string(),
        TaskContext::default(),
        5,
    ).await;

    println!("Found {} relevant episodes", relevant.len());
}

Pattern-Based Learning

The system automatically extracts reusable patterns from completed episodes:

use do_memory_core::memory::SelfLearningMemory;
use do_memory_core::patterns::HybridPatternExtractor;
use do_memory_core::episode::Episode;

// Configure hybrid pattern extraction
let extractor = HybridPatternExtractor::new();

// Patterns are automatically extracted during episode completion
// The system tracks pattern effectiveness over time

Semantic Search with Embeddings

use do_memory_core::embeddings::{SemanticService, EmbeddingConfig, InMemoryEmbeddingStorage};
use do_memory_core::episode::Episode;
use do_memory_core::TaskContext;

// Use semantic similarity to find related episodes
let semantic = SemanticService::default(storage).await.unwrap();

let related_episodes = semantic
    .find_similar_episodes(
        "fix authentication bug",
        &TaskContext::default(),
        10
    )
    .await
    .unwrap();

Learning Cycle

1. Start Episode: Create a new episode with task context
2. Log Steps: Record each execution step with outcomes
3. Complete Episode: Mark episode as success/failure
4. Extract Patterns: Identify reusable patterns
5. Generate Reflection: Create insights and lessons learned
6. Calculate Reward: Score episode success
7. Retrieve: Use learned patterns for future tasks

Error Handling

Most functions return Result<T> for proper error handling:

use do_memory_core::{Error, Result};

async fn example() -> Result<()> {
    // Operations that can fail
    // .await?
    Ok(())
}

Feature Flags

• openai: Enable OpenAI embeddings
• local-embeddings: Enable local ONNX-based embeddings
• turso: Enable Turso/libSQL storage backend
• redb: Enable redb cache storage
• full: Enable all features

Re-exports

pub use sync::StorageSynchronizer;

pub use episode::CapacityManager;

pub use episode::Episode;

pub use episode::EvictionPolicy;

pub use episode::ExecutionStep;

pub use episode::PatternId;

pub use error::CacheError;

pub use error::Error;

pub use error::RelationshipError;

pub use error::Result;

pub use extraction::PatternExtractor;

pub use indexing::BenchmarkResult;

pub use indexing::IndexMetrics;

pub use indexing::IndexableMemory;

pub use indexing::QueryPerformance;

pub use indexing::hierarchical::HierarchicalIndex;

pub use indexing::hierarchical::HierarchicalIndexStats;

pub use indexing::hierarchical::HierarchicalQuery;

pub use indexing::spatiotemporal::IndexStats;

pub use indexing::spatiotemporal::QueryOptions;

pub use indexing::spatiotemporal::SpatiotemporalIndex;

pub use indexing::spatiotemporal::TimeBucket;

pub use learning::queue::PatternExtractionQueue;

pub use learning::queue::QueueConfig;

pub use learning::queue::QueueStats;

pub use memory::SelfLearningMemory;

pub use memory::checkpoint::CheckpointMeta;

pub use memory::checkpoint::HandoffPack;

pub use memory::filters::EpisodeFilter;

pub use memory::filters::EpisodeFilterBuilder;

pub use memory::filters::OutcomeType;

pub use memory::step_buffer::BatchConfig;

pub use monitoring::AgentMetrics;

pub use monitoring::AgentMonitor;

pub use monitoring::AgentType;

pub use monitoring::MonitoringConfig;

pub use monitoring::TaskMetrics;

pub use patterns::Anomaly;

pub use patterns::AnomalyReason;

pub use patterns::ChangeDirection;

pub use patterns::ChangeType;

pub use patterns::Changepoint;

pub use patterns::ChangepointConfig;

pub use patterns::ChangepointDetector;

pub use patterns::ClusterCentroid;

pub use patterns::ClusteringConfig;

pub use patterns::DBSCANAnomalyDetector;

pub use patterns::DBSCANClusterResult;

pub use patterns::DBSCANConfig;

pub use patterns::DBSCANStats;

pub use patterns::EffectivenessTracker;

pub use patterns::EpisodeCluster;

pub use patterns::FeatureWeights;

pub use patterns::Heuristic;

pub use patterns::Pattern;

pub use patterns::PatternClusterer;

pub use patterns::PatternEffectiveness;

pub use patterns::PatternMetrics;

pub use patterns::PatternUsage;

pub use patterns::PatternValidator;

pub use patterns::SegmentComparison;

pub use patterns::SegmentComparisonConfig;

pub use patterns::SegmentStats;

pub use patterns::UsageStats;

pub use patterns::ValidationConfig;

pub use procedural::ProceduralMemory;

pub use reflection::ReflectionGenerator;

pub use retrieval::CacheKey;

pub use retrieval::CacheMetrics;

pub use retrieval::QueryCache;

pub use reward::AdaptiveRewardCalculator;

pub use reward::DomainStatistics;

pub use reward::DomainStatisticsCache;

pub use reward::RewardCalculator;

pub use security::audit::ActorType;

pub use security::audit::AuditConfig;

pub use security::audit::AuditContext;

pub use security::audit::AuditEntry;

pub use security::audit::AuditEventType;

pub use security::audit::AuditLogLevel;

pub use security::audit::AuditLogger;

pub use security::audit::AuditOutput;

pub use security::audit::AuditResult;

pub use storage::DEFAULT_QUERY_LIMIT;

pub use storage::MAX_QUERY_LIMIT;

pub use storage::StorageBackend;

pub use storage::apply_query_limit;

pub use types::CloudEvent;

pub use types::ComplexityLevel;

pub use types::ConcurrencyConfig;

pub use types::DualRewardScore;

pub use types::EventEmitter;

pub use types::EventEmitterMode;

pub use types::Evidence;

pub use types::ExecutionResult;

pub use types::LogEmitter;

pub use types::MemoryConfig;

pub use types::MemoryEventMapping;

pub use types::NoOpEmitter;

pub use types::OutcomeStats;

pub use types::Reflection;

pub use types::RewardScore;

pub use types::StorageConfig;

pub use types::TaskContext;

pub use types::TaskOutcome;

pub use types::TaskType;

pub use memory::attribution::RecommendationFeedback;

pub use memory::attribution::RecommendationSession;

pub use memory::attribution::RecommendationStats;

pub use memory::attribution::RecommendationTracker;

pub use context::AddResult;

pub use context::BundleAccumulator;

pub use context::BundleConfig;

pub use context::BundleStats;

pub use context::ContextItem;

pub use context::ContextItemType;

pub use retry::ConcurrencyLimiter;

pub use retry::RetryBudget;

pub use retry::RetryConfig;

pub use retry::RetryMetrics;

pub use retry::RetryPolicy;

Modules

constants

Global constants for memory-core

context

Bounded context assembly for retrieval results.

embeddings

Semantic Embeddings

episode

Episode management for tracking task execution.

error

extraction

Pattern Extractor

indexing

Hierarchical indexing module for spatiotemporal episode organization.

learning

Learning Coordination

memory

Self Learning Memory

monitoring

Agent Monitoring

patterns

Pattern Validation and Effectiveness Tracking

pre_storage

Pre-storage ingest layer for episodic memory quality enhancement.

procedural

Procedural memory management

reflection

Reflection Generator

retrieval

Episodic memory retrieval with caching and hybrid search.

retry

reward

Reward Calculator

search

Search capabilities for memory retrieval

security

Security-related functionality for the memory system.

semantic

Semantic summarization and embedding capabilities.

spatiotemporal

Spatiotemporal memory organization for efficient episodic retrieval

storage

Persistence Layer

sync

Storage Synchronization

types