Struct CodememEngine 

pub struct CodememEngine { /* private fields */ }

Core domain engine holding all backends and domain state.

This struct contains all the business logic for the Codemem memory system. Transport layers (MCP, REST API, CLI) hold a CodememEngine and delegate domain operations to it, keeping transport concerns separate.

Concrete types are intentional: CodememEngine uses concrete backend types (Storage, HnswIndex, GraphEngine) rather than trait objects (dyn StorageBackend, dyn VectorBackend, dyn GraphBackend) for performance. This enables monomorphization (the compiler generates specialized code for each concrete type), eliminates vtable indirection overhead on every call, and provides predictable memory layout for cache-friendly access patterns. The trait abstractions exist for testing and alternative implementations, but the engine itself benefits from static dispatch.

Implementations

impl CodememEngine

pub fn recall_with_impact( &self, query: &str, k: usize, namespace: Option<&str>, ) -> Result<Vec<ImpactResult>, CodememError>

Recall memories enriched with graph impact data (PageRank, centrality, connected decisions, dependent files).

pub fn get_decision_chain( &self, file_path: Option<&str>, topic: Option<&str>, ) -> Result<DecisionChain, CodememError>

Find Decision-type memories matching a file_path or topic, then follow EvolvedInto/LeadsTo/DerivedFrom edges via BFS to build a chronological chain.

pub fn session_checkpoint( &self, session_id: &str, namespace: Option<&str>, ) -> Result<SessionCheckpointReport, CodememError>

Build a mid-session progress report: activity summary, pattern detection (session-scoped + cross-session), stores new pattern insights, hot directories, markdown report.

pub fn node_coverage( &self, node_ids: &[&str], ) -> Result<Vec<NodeCoverageEntry>, CodememError>

Check which graph nodes have attached memories (depth-1 only).

impl CodememEngine

pub fn consolidate_cluster( &self, similarity_threshold: Option<f64>, ) -> Result<ConsolidationResult, CodememError>

Consolidate cluster: semantic deduplication using vector KNN + cosine similarity.

Groups memories by namespace and memory_type for candidate grouping. Within each group, uses vector KNN search to find candidate duplicates (avoiding O(n^2) pairwise comparison), then verifies with cosine similarity + union-find to cluster transitively-similar memories. Keeps the highest-importance memory per cluster.

For small groups (<=50 members), falls back to pairwise comparison since the overhead of KNN setup is not worth it.

impl CodememEngine

pub fn consolidate_creative(&self) -> Result<ConsolidationResult, CodememError>

Consolidate creative: O(n log n) semantic creative consolidation. Uses vector KNN search per memory to find cross-type neighbors and creates SHARES_THEME edges.

Memory usage: O(K768) per query instead of O(N768) for all embeddings, where K is the number of nearest neighbors searched (7). Only the memory metadata (IDs + types) is kept in RAM, not the full embedding vectors.

impl CodememEngine

pub fn consolidate_decay( &self, threshold_days: Option<i64>, ) -> Result<ConsolidationResult, CodememError>

Consolidate decay: power-law decay that rewards access frequency.

impl CodememEngine

pub fn consolidate_forget( &self, importance_threshold: Option<f64>, target_tags: Option<&[String]>, max_access_count: Option<u32>, ) -> Result<ConsolidationResult, CodememError>

Consolidate forget: delete low-importance, never-accessed memories.

impl CodememEngine

pub fn consolidate_summarize( &self, min_cluster_size: Option<usize>, ) -> Result<ConsolidationResult, CodememError>

Consolidate summarize: LLM-powered consolidation that finds connected components, summarizes large clusters into Insight memories linked via SUMMARIZES edges.

impl CodememEngine

pub fn consolidation_status( &self, ) -> Result<Vec<ConsolidationStatusEntry>, CodememError>

Get the status of all consolidation cycles.

pub fn find_forgettable_by_tags( &self, importance_threshold: f64, target_tags: &[String], max_access_count: u32, ) -> Result<Vec<String>, CodememError>

Find memories matching any of the target tags below importance threshold and with access_count <= max_access_count.

M14: Note — this loads all memories and filters in Rust. For large databases, a storage method with SQL WHERE clauses for importance/access_count/tags would be more efficient, but that requires adding a new StorageBackend trait method.

pub fn rebuild_vector_index_internal(&self, vector: &mut HnswIndex)

Internal helper: rebuild vector index from all stored embeddings.

impl CodememEngine

pub fn enrich_api_surface( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Analyze the public API surface of each module/package.

Counts public vs private symbols per file and stores Insight memories summarizing the public API.

impl CodememEngine

pub fn enrich_architecture( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Infer architectural layers and patterns from the module dependency graph.

Analyzes IMPORTS/CALLS/DEPENDS_ON edges between modules to detect layering (e.g., api -> service -> storage) and recognizes common directory patterns (controllers/, models/, views/, handlers/).

impl CodememEngine

pub fn enrich_blame( &self, path: &str, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Enrich file nodes with primary owner and contributors from git blame.

impl CodememEngine

pub fn enrich_change_impact( &self, file_path: &str, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Predict the impact of changes to a given file by combining co-change edges, call graph edges, and test file associations.

impl CodememEngine

pub fn enrich_code_smells( &self, namespace: Option<&str>, project_root: Option<&Path>, ) -> Result<EnrichResult, CodememError>

Detect common code smells: long functions (>50 lines), too many parameters (>5), deep nesting (>4 levels), and long files (>500 lines).

Stores findings as Pattern memories with importance 0.5.

impl CodememEngine

pub fn enrich_complexity( &self, namespace: Option<&str>, project_root: Option<&Path>, ) -> Result<EnrichResult, CodememError>

Enrich the graph with cyclomatic and cognitive complexity metrics for functions/methods.

For each Function/Method node, reads the source file, counts decision points (if/else/match/for/while/loop/&&/||) as cyclomatic complexity and measures max nesting depth as a cognitive complexity proxy. High-complexity functions (cyclomatic > 10) produce Insight memories.

impl CodememEngine

pub fn enrich_doc_coverage( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Analyze documentation coverage for public symbols.

Checks if each public symbol has a non-empty doc_comment in its payload. Stores Insight memories for files with low documentation coverage.

impl CodememEngine

pub fn enrich_git_history( &self, path: &str, days: u64, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Enrich the graph with git history analysis: file activity, co-changes, contributors.

impl CodememEngine

pub fn enrich_hot_complex( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Cross-reference git churn with complexity to find high-risk files.

Files that are BOTH high-churn AND high-complexity represent the highest maintenance risk. Requires E1 (complexity) and git enrichment to have run first.

impl CodememEngine

pub fn enrich_performance( &self, top: usize, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Enrich the graph with performance analysis: coupling, dependency depth, PageRank, complexity.

impl CodememEngine

pub fn enrich_quality_stratification( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Categorize existing enrichment insights by signal strength and adjust importance.

• Noise (< 0.3): basic counts, minor observations
• Signal (0.5-0.7): moderate complexity, useful patterns
• Critical (0.8-1.0): high-risk findings, security issues

impl CodememEngine

pub fn enrich_security( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Enrich the graph with security analysis: sensitive files, endpoints, security functions.

impl CodememEngine

pub fn enrich_security_scan( &self, namespace: Option<&str>, project_root: Option<&Path>, ) -> Result<EnrichResult, CodememError>

Scan actual file contents for security issues: hardcoded credentials, SQL concatenation, unsafe blocks, etc.

impl CodememEngine

pub fn enrich_test_mapping( &self, namespace: Option<&str>, ) -> Result<EnrichResult, CodememError>

Map test functions to the code they test and identify untested public functions.

For Test-kind nodes, infers tested symbols by naming convention (test_foo -> foo) and by CALLS edges. Creates RELATES_TO edges between test and tested symbols. Produces Insight memories for files with untested public functions.

impl CodememEngine

pub fn store_insight( &self, content: &str, track: &str, tags: &[&str], importance: f64, namespace: Option<&str>, links: &[String], ) -> Option<String>

Store an Insight memory through a 3-phase pipeline:

1. Semantic dedup check (reject near-duplicates before persisting)
2. Core persist via persist_memory_no_save (storage, BM25, graph node, embedding)
3. Post-step: RELATES_TO edges to linked nodes + auto-link to code nodes

Returns the memory ID if inserted, or None if it was a duplicate. Does NOT call save_index() – callers should batch that at the end.

pub fn run_enrichments( &self, path: &str, analyses: &[String], days: u64, namespace: Option<&str>, file_path: Option<&str>, ) -> EnrichmentPipelineResult

Run selected enrichment analyses (or all 14 if analyses is empty).

Parameters:

• path: project root (needed for git, blame, change_impact, complexity, code_smells, security_scan)
• analyses: which analyses to run; empty = all (except change_impact which needs file_path)
• days: git history lookback days
• namespace: optional namespace filter
• file_path: optional, needed only for change_impact

impl CodememEngine

pub fn enrich_memory_text( &self, content: &str, memory_type: MemoryType, tags: &[String], namespace: Option<&str>, node_id: Option<&str>, ) -> String

Build contextual text for a memory node.

NOTE: Acquires the graph lock on each call. For batch operations, consider passing a pre-acquired guard or caching results.

pub fn enrich_symbol_text(&self, sym: &Symbol, edges: &[ResolvedEdge]) -> String

Build contextual text for a code symbol.

pub fn enrich_chunk_text(&self, chunk: &CodeChunk) -> String

Build contextual text for a code chunk before embedding.

impl CodememEngine

pub fn save_index(&self)

Save the vector and BM25 indexes to disk if a db_path is configured. Compacts the HNSW index if ghost entries exceed 20% of live entries. Always clears the dirty flag so flush_if_dirty() won’t double-save.

pub fn reload_graph(&self) -> Result<(), CodememError>

Reload the in-memory graph from the database.

pub fn process_watch_event( &self, event: &WatchEvent, namespace: Option<&str>, project_root: Option<&Path>, ) -> Result<(), CodememError>

Process a single file watcher event by re-indexing changed/created files or cleaning up deleted file nodes.

Call this from a watcher event loop:

while let Ok(event) = watcher.receiver().recv() {
    engine.process_watch_event(&event, namespace, Some(root));
}

pub fn cleanup_deleted_files( &self, dir_path: &str, ) -> Result<usize, CodememError>

Compare files on disk vs file nodes in the graph and clean up stale entries. Call this after indexing or on watcher delete events.

pub fn index_and_enrich( &self, path: &str, namespace: Option<&str>, git_days: u64, ) -> Result<IndexEnrichResult, CodememError>

Combined result from index_and_enrich. Index a codebase and run all enrichment passes in one call.

pub fn session_context( &self, namespace: Option<&str>, ) -> Result<SessionContext, CodememError>

Synthesize context for a new session: recent memories, pending analyses, active patterns, and last session summary.

impl CodememEngine

pub fn auto_link_to_code_nodes( &self, memory_id: &str, content: &str, existing_links: &[String], ) -> usize

Scan memory content for file paths and qualified symbol names that exist as graph nodes, and create RELATES_TO edges.

pub fn auto_link_by_tags(&self, memory: &MemoryNode)

Create edges between this memory and other memories that share tags.

• session:* tags → PRECEDED_BY edges (temporal ordering within a session)
• Other shared tags → SHARES_THEME edges (topical overlap)

This runs during persist_memory so the graph builds connectivity at ingestion time, rather than relying solely on creative consolidation.

pub fn get_node_memories( &self, node_id: &str, max_depth: usize, include_relationships: Option<&[RelationshipType]>, ) -> Result<Vec<NodeMemoryResult>, CodememError>

Retrieve all memories connected to a graph node via BFS traversal.

Performs level-by-level BFS to track actual hop distance. For each Memory node found, reports the relationship type from the edge that connected it (or the edge leading into the path toward it).

impl CodememEngine

pub fn persist_memory(&self, memory: &MemoryNode) -> Result<(), CodememError>

Persist a memory through the full pipeline: storage → BM25 → graph → embedding → vector.

pub fn add_edge(&self, edge: Edge) -> Result<(), CodememError>

Add an edge to both storage and in-memory graph.

pub fn refine_memory( &self, old_id: &str, content: Option<&str>, tags: Option<Vec<String>>, importance: Option<f64>, ) -> Result<(MemoryNode, String), CodememError>

Refine a memory: create a new version with an EVOLVED_INTO edge from old to new.

pub fn split_memory( &self, source_id: &str, parts: &[SplitPart], ) -> Result<Vec<String>, CodememError>

Split a memory into multiple parts, each linked via PART_OF edges.

pub fn merge_memories( &self, source_ids: &[String], content: &str, memory_type: MemoryType, importance: f64, tags: Vec<String>, ) -> Result<String, CodememError>

Merge multiple memories into one, linked via SUMMARIZES edges.

pub fn update_memory( &self, id: &str, content: &str, importance: Option<f64>, ) -> Result<(), CodememError>

Update a memory’s content and/or importance, re-embedding if needed.

pub fn update_importance( &self, id: &str, importance: f64, ) -> Result<(), CodememError>

Update only the importance of a memory. Routes through the engine to maintain the transport → engine → storage boundary.

pub fn delete_memory(&self, id: &str) -> Result<bool, CodememError>

Delete a memory from all subsystems.

M1: Uses delete_memory_cascade on the storage backend to wrap all SQLite deletes (memory + graph nodes/edges + embedding) in a single transaction when the backend supports it. In-memory structures (vector, graph, BM25) are cleaned up separately with proper lock ordering.

impl CodememEngine

pub fn compact_graph(&self, seen_files: &HashSet<String>) -> (usize, usize)

Compact chunk and symbol graph-nodes after indexing. Returns (chunks_pruned, symbols_pruned).

impl CodememEngine

pub fn persist_index_results( &self, results: &IndexAndResolveResult, namespace: Option<&str>, ) -> Result<IndexPersistResult, CodememError>

Persist all indexing results (file nodes, package tree, symbol nodes, chunk nodes, edges, embeddings, compaction) into storage and the in-memory graph.

This is the full persistence pipeline called after Indexer::index_and_resolve().

pub fn persist_index_results_with_progress( &self, results: &IndexAndResolveResult, namespace: Option<&str>, on_progress: impl Fn(usize, usize), ) -> Result<IndexPersistResult, CodememError>

Like persist_index_results, but calls on_progress(done, total) during the embedding phase so callers can display progress.

impl CodememEngine

pub fn recall( &self, query: &str, k: usize, memory_type_filter: Option<MemoryType>, namespace_filter: Option<&str>, exclude_tags: &[String], min_importance: Option<f64>, min_confidence: Option<f64>, ) -> Result<Vec<SearchResult>, CodememError>

Core recall logic: search storage with hybrid scoring and return ranked results.

Combines vector search (if embeddings available), BM25, graph strength, temporal, tag matching, importance, confidence, and recency into a 9-component hybrid score. Supports filtering by memory type, namespace, tag exclusion, and minimum importance/confidence thresholds.

pub fn recall_with_expansion( &self, query: &str, k: usize, expansion_depth: usize, namespace_filter: Option<&str>, ) -> Result<Vec<ExpandedResult>, CodememError>

Recall with graph expansion: vector search (or BM25 fallback) for seed memories, then BFS expansion from each seed through the graph, scoring all candidates with the 9-component hybrid scorer.

pub fn namespace_stats( &self, namespace: &str, ) -> Result<NamespaceStats, CodememError>

Compute detailed stats for a single namespace: count, averages, type distribution, tag frequency, and date range.

pub fn delete_namespace(&self, namespace: &str) -> Result<usize, CodememError>

Delete all memories in a namespace from all subsystems (storage, vector, graph, BM25). Returns the number of memories deleted.

impl CodememEngine

pub fn graph_memory_estimate(&self) -> usize

Estimate the approximate RAM usage of the in-memory graph.

The graph engine (GraphEngine) keeps all nodes and edges in memory via HashMaps plus a petgraph::DiGraph. Each node carries a GraphNode struct (~200 bytes: id, kind, label, payload HashMap, centrality, optional memory_id and namespace) plus a petgraph NodeIndex. Each edge carries an Edge struct (~150 bytes: id, src, dst, relationship, weight, properties HashMap, timestamps) plus petgraph edge weight storage and adjacency list entries.

Returns an estimate in bytes. Actual usage may be higher due to HashMap overhead, string heap allocations, and payload contents.

pub fn search_code( &self, query: &str, k: usize, ) -> Result<Vec<CodeSearchResult>, CodememError>

Semantic code search: embed query, vector search, filter to sym:/chunk: IDs, enrich with graph node data.

pub fn summary_tree( &self, start_id: &str, max_depth: usize, include_chunks: bool, ) -> Result<SummaryTreeNode, CodememError>

Build a hierarchical summary tree starting from a given node. Traverses Contains edges: packages -> files -> symbols.

pub fn get_symbol( &self, qualified_name: &str, ) -> Result<Option<Symbol>, CodememError>

Look up a single symbol by qualified name with cache-through to the DB.

1. Check the in-memory IndexCache.
2. On miss, query storage for sym:{qualified_name} graph node.
3. Reconstruct a Symbol, insert into cache, return.

pub fn search_symbols( &self, query: &str, limit: usize, kind_filter: Option<&str>, ) -> Result<Vec<SymbolSearchResult>, CodememError>

Search symbols by name, with optional kind filter. Falls through to the DB when the in-memory cache is empty or has no matches.

impl CodememEngine

pub fn new( storage: Box<dyn StorageBackend>, vector: HnswIndex, graph: GraphEngine, embeddings: Option<Box<dyn EmbeddingProvider>>, ) -> Self

Create an engine with storage, vector, graph, and optional embeddings backends.

pub fn new_with_config( storage: Box<dyn StorageBackend>, vector: HnswIndex, graph: GraphEngine, embeddings: Option<Box<dyn EmbeddingProvider>>, config: CodememConfig, ) -> Self

Create an engine with an explicit config (avoids double-loading from disk).

pub fn from_db_path(db_path: &Path) -> Result<Self, CodememError>

Create an engine from a database path, loading all backends.

pub fn for_testing() -> Self

Create a minimal engine for testing.

pub fn lock_vector(&self) -> Result<MutexGuard<'_, HnswIndex>, CodememError>

pub fn lock_graph(&self) -> Result<MutexGuard<'_, GraphEngine>, CodememError>

pub fn lock_bm25(&self) -> Result<MutexGuard<'_, Bm25Index>, CodememError>

pub fn lock_embeddings( &self, ) -> Result<Option<MutexGuard<'_, Box<dyn EmbeddingProvider>>>, CodememError>

pub fn lock_index_cache( &self, ) -> Result<MutexGuard<'_, Option<IndexCache>>, CodememError>

pub fn scoring_weights( &self, ) -> Result<RwLockReadGuard<'_, ScoringWeights>, CodememError>

pub fn scoring_weights_mut( &self, ) -> Result<RwLockWriteGuard<'_, ScoringWeights>, CodememError>

pub fn storage(&self) -> &dyn StorageBackend

Access the storage backend.

pub fn has_embeddings(&self) -> bool

Whether an embedding provider is configured.

pub fn db_path(&self) -> Option<&Path>

Access the database path (if backed by a file).

pub fn config(&self) -> &CodememConfig

Access the loaded configuration.

pub fn metrics(&self) -> &Arc<InMemoryMetrics>

Access the metrics collector.

pub fn graph_mutex(&self) -> &Mutex<GraphEngine>

Access the raw graph Mutex (for callers that need &Mutex<GraphEngine>).

pub fn vector_mutex(&self) -> &Mutex<HnswIndex>

Access the raw vector Mutex (for callers that need &Mutex<HnswIndex>).

pub fn bm25_mutex(&self) -> &Mutex<Bm25Index>

Access the raw BM25 Mutex (for callers that need &Mutex<Bm25Index>).

pub fn embeddings_mutex(&self) -> Option<&Mutex<Box<dyn EmbeddingProvider>>>

Access the raw embeddings Mutex (for callers that need the Option<&Mutex<...>>).