Struct MemoryStorage 

pub struct MemoryStorage { /* private fields */ }

Storage engine for long-term memory persistence

Uses a single RocksDB instance with 2 column families:

• default: main memory data (also shared by LearningHistoryStore, TemporalFactStore, etc. via key prefixes)
• memory_index: secondary indices for tag/type/timestamp queries

Implementations

impl MemoryStorage

pub fn new(path: &Path, shared_cache: Option<&Cache>) -> Result<Self>

Create a new memory storage.

If shared_cache is provided, all block-cache reads are charged against the shared LRU cache (recommended for multi-tenant server mode). When None, a small per-instance cache is created (standalone / test use).

pub fn path(&self) -> &Path

Get the base storage path

pub fn store(&self, memory: &Memory) -> Result<()>

Store a memory with configurable write durability

ROBOTICS OPTIMIZATION: Write mode is configurable via SHODH_WRITE_MODE env var.

• Async (default): <1ms per write, data survives process crashes
• Sync: 2-10ms per write, data survives power loss

For robotics/edge: Use async mode + periodic flush() calls for best latency. For compliance/critical: Set SHODH_WRITE_MODE=sync for full durability.

pub fn get_by_content_hash(&self, content: &str) -> Option<MemoryId>

Look up an existing memory by content hash (idempotency dedup, issue #109). Returns the MemoryId if identical content was already stored.

pub fn get(&self, id: &MemoryId) -> Result<Memory>

Retrieve a memory by ID

Performs lazy migration: if memory is in legacy format, re-writes it in current format for faster future reads.

pub fn find_by_external_id(&self, external_id: &str) -> Result<Option<Memory>>

Find a memory by its external_id (e.g., “linear:SHO-39”, “github:pr-123”)

Returns the memory if found, None if no memory with this external_id exists. Used for upsert operations when syncing from external sources.

pub fn update(&self, memory: &Memory) -> Result<()>

Update an existing memory

ALGO-004 FIX: Re-indexes memory after update to handle importance drift. When Hebbian feedback changes importance, the old bucket index becomes stale. We remove old indices before storing to ensure index consistency.

pub fn delete(&self, id: &MemoryId) -> Result<()>

Delete a memory with configurable durability

pub fn search(&self, criteria: SearchCriteria) -> Result<Vec<Memory>>

Search memories by various criteria

pub fn get_children(&self, parent_id: &MemoryId) -> Result<Vec<Memory>>

Get children of a memory (public API)

pub fn get_ancestors(&self, memory_id: &MemoryId) -> Result<Vec<Memory>>

Get the parent chain (ancestors) of a memory

pub fn get_hierarchy_context( &self, memory_id: &MemoryId, ) -> Result<(Vec<Memory>, Memory, Vec<Memory>)>

Get the full hierarchy context for a memory Returns (ancestors, memory, children)

pub fn get_subtree( &self, root_id: &MemoryId, max_depth: usize, ) -> Result<Vec<Memory>>

Get all memories in a subtree rooted at the given memory

pub fn get_all_ids(&self) -> Result<Vec<MemoryId>>

Get all memory IDs without loading full Memory objects.

Returns only 16-byte UUID keys (lightweight). Use with get() to load individual memories in batches to avoid OOM on large datasets.

pub fn get_all(&self) -> Result<Vec<Memory>>

Get all memories from long-term storage

Only returns entries with valid 16-byte UUID keys (consistent with get_stats)

pub fn get_uncompressed_older_than( &self, cutoff: DateTime<Utc>, ) -> Result<Vec<Memory>>

pub fn mark_forgotten_by_age( &self, cutoff: DateTime<Utc>, ) -> Result<Vec<MemoryId>>

Mark memories as forgotten (soft delete) with atomic batch write. Returns the IDs of memories that were flagged, so callers can clean up secondary indices (vector, BM25, graph).

pub fn mark_forgotten_by_importance( &self, threshold: f32, ) -> Result<Vec<MemoryId>>

Mark memories with low importance as forgotten with atomic batch write. Returns the IDs of memories that were flagged, so callers can clean up secondary indices (vector, BM25, graph).

pub fn remove_matching(&self, regex: &Regex) -> Result<usize>

Remove memories matching a pattern with durable writes

pub fn update_access(&self, id: &MemoryId) -> Result<()>

Update access count for a memory

pub fn get_stats(&self) -> Result<StorageStats>

Get statistics about stored memories

pub fn get_retrieval_count(&self) -> Result<usize>

Get the persisted retrieval counter

pub fn increment_retrieval_count(&self) -> Result<usize>

Increment and persist the retrieval counter, returns new value

pub fn cleanup_corrupted(&self) -> Result<usize>

Remove corrupted memories that fail to deserialize even with legacy fallbacks Returns the number of entries deleted

This function safely cleans up:

1. Entries with keys that are not valid 16-byte UUIDs (corrupted/misplaced)
2. Entries with valid UUID keys but corrupted values that fail ALL format fallbacks

It preserves:

• Valid Memory entries (any format - current or legacy)
• Stats entries (keys starting with “stats:”)

pub fn migrate_legacy(&self) -> Result<(usize, usize, usize)>

Migrate legacy memories to current format for improved performance Returns (migrated_count, already_current_count, failed_count)

This function:

1. Iterates all memories in storage
2. Attempts to deserialize with format fallback
3. Re-saves successfully deserialized legacy memories in current format
4. Reports migration statistics

pub fn flush(&self) -> Result<()>

Flush all column families to ensure data is persisted (critical for graceful shutdown)

pub fn db(&self) -> Arc<DB>

Get a reference to the underlying RocksDB instance

Used by SemanticFactStore to share the same database for fact storage. Facts use a different key prefix (“facts:”) to avoid collisions.

impl MemoryStorage

pub fn store_with_vectors( &self, memory: &Memory, vector_ids: Vec<u32>, ) -> Result<()>

Store memory and its text vector mapping atomically

Uses WriteBatch to ensure both operations succeed or both fail. This is the ONLY way orphaned memories can be prevented.

For text-only memories (current implementation). Use store_with_multimodal_vectors for memories with image/audio/video content.

pub fn store_with_multimodal_vectors( &self, memory: &Memory, modality: Modality, vector_ids: Vec<u32>, ) -> Result<()>

Store memory with vectors for a specific modality

MULTIMODALITY READY: Supports text, image, audio, video modalities. Each modality is stored separately, allowing cross-modal search.

pub fn get_vector_mapping( &self, memory_id: &MemoryId, ) -> Result<Option<VectorMappingEntry>>

Get vector mapping for a memory

pub fn get_all_vector_mappings( &self, ) -> Result<Vec<(MemoryId, VectorMappingEntry)>>

Get all vector mappings (for rebuilding Vamana index on startup)

Returns iterator-style results to avoid loading everything into memory at once. Sorted by memory_id for deterministic Vamana rebuilding.

pub fn delete_vector_mapping(&self, memory_id: &MemoryId) -> Result<()>

Delete vector mapping for a memory (called when deleting memory)

pub fn update_vector_mapping( &self, memory_id: &MemoryId, vector_ids: Vec<u32>, ) -> Result<()>

Update text vector mapping for a memory (for reindex operations)

Convenience method for text-only reindexing.

pub fn update_modality_vectors( &self, memory_id: &MemoryId, modality: Modality, vector_ids: Vec<u32>, ) -> Result<()>

Update vector mapping for a specific modality

MULTIMODALITY READY: Preserves vectors for other modalities while updating one.

pub fn delete_with_vectors(&self, id: &MemoryId) -> Result<()>

Delete memory and its vector mapping atomically

pub fn count_vector_mappings(&self) -> usize

Count memories with vector mappings (for health checks)

pub fn find_memories_without_mappings(&self) -> Result<Vec<MemoryId>>

Check integrity: find memories without vector mappings

Returns memories that have embeddings but no corresponding vector mapping. These need to be reindexed.

pub fn get_all_text_vector_ids(&self) -> Result<Vec<u32>>

Get all text vector IDs from mappings (for Vamana statistics)

pub fn get_modality_stats(&self) -> Result<HashMap<Modality, usize>>

Get vector count per modality (for health monitoring)

pub fn save_interference_records( &self, memory_id: &str, records: &[InterferenceRecord], ) -> Result<()>

Persist interference records for a single memory

Key format: interference:{memory_id} → JSON Vec<InterferenceRecord>

pub fn load_all_interference_records( &self, ) -> Result<(HashMap<String, Vec<InterferenceRecord>>, usize)>

Load all interference records from storage on startup

Scans all interference: prefixed keys and deserializes records. Returns (history_map, total_event_count) for bulk-loading into InterferenceDetector.

pub fn delete_interference_records(&self, memory_id: &str) -> Result<()>

Delete interference records for a single memory (called on forget/delete)

pub fn save_interference_event_count(&self, count: usize) -> Result<()>

Persist the total interference event count

Key: interference_meta:total → 8-byte little-endian u64

pub fn get_fact_watermark(&self, user_id: &str) -> Option<i64>

Load the fact extraction watermark for a user.

Key: _watermark:fact_extraction:{user_id} → 8-byte little-endian i64 (unix millis) Returns None if no watermark has been persisted yet.

pub fn set_fact_watermark(&self, user_id: &str, timestamp_millis: i64)

Persist the fact extraction watermark for a user.

Key: _watermark:fact_extraction:{user_id} → 8-byte little-endian i64 (unix millis)

pub fn clear_all_interference_records(&self) -> Result<usize>

Delete ALL interference records (GDPR forget_all)

Batch-deletes all interference: and interference_meta: keys.