zeph-memory

Semantic memory with SQLite and Qdrant for Zeph agent.

Overview

Provides durable conversation storage via SQLite and semantic retrieval through Qdrant vector search (or embedded SQLite vector backend). The SemanticMemory orchestrator combines both backends, enabling the agent to recall relevant context from past conversations using embedding similarity.

Recall quality is enhanced by MMR (Maximal Marginal Relevance) re-ranking for result diversity, temporal decay scoring for recency bias, and write-time importance scoring for content-aware ranking. All are configurable via SemanticConfig.

Query-aware memory routing (MemoryRouter trait, HeuristicRouter default) classifies each query as Keyword (SQLite FTS5), Semantic (Qdrant), or Hybrid and dispatches accordingly. Configure via [memory.routing].

Includes a document ingestion subsystem for loading, chunking, and storing user documents (text, Markdown, PDF) into Qdrant for RAG workflows.

SYNAPSE spreading activation enables multi-hop graph retrieval: seed entities are activated, then energy propagates through the entity graph with hop-by-hop decay (configurable lambda), lateral inhibition, and edge-type filtering. Configure via [memory.graph.spreading_activation].

MAGMA multi-graph memory provides typed edges (EdgeType enum: uses, related_to, part_of, depends_on, created_by, authored, manages, contains) for fine-grained relationship tracking and edge-type-aware traversal.

Structured anchored summarization preserves factual anchors (entities, relationships, key decisions) during compaction, producing summaries that maintain cross-session recall fidelity.

Compaction probe validation verifies compaction quality by generating probe questions from pre-compaction content and scoring the post-compaction text against them, detecting information loss before it becomes permanent.

Key modules

Re-exports: MemoryError, QdrantOps, ConversationId, MessageId, Document, DocumentLoader, TextLoader, TextSplitter, IngestionPipeline, Chunk, SplitterConfig, DocumentError, DocumentMetadata, PdfLoader (behind pdf feature), Embeddable, EmbeddingRegistry, ResponseCache, MemorySnapshot, TokenCounter, UserCorrection, FeedbackDetector, AnchoredSummary, CompactionProbeConfig, validate_compaction

Document RAG

IngestionPipeline loads, chunks, embeds, and stores documents into the zeph_documents Qdrant collection. When memory.documents.rag_enabled = true, the agent automatically queries this collection on every turn and prepends the top-K most relevant chunks to the context window.

zeph ingest ./docs/           # ingest all .txt, .md, .pdf files recursively
zeph ingest README.md --chunk-size 256 --collection my_docs

Configure via [memory.documents] in config.toml:

[!NOTE] RAG injection is a no-op when the zeph_documents collection is empty. Documents must be ingested with zeph ingest before retrieval has any effect.

Snapshot export/import

Memory snapshots allow exporting all conversations and messages to a portable JSON file and importing them back into another instance.

zeph memory export backup.json
zeph memory import backup.json

Response cache

ResponseCache deduplicates LLM calls by caching responses in SQLite. Cache keys are computed via blake3 hashing of the prompt content. Entries expire after a configurable TTL (default: 1 hour). A background task periodically removes expired entries; the interval is controlled by response_cache_cleanup_interval_secs.

Ranking options

User corrections and cross-session personalization

FeedbackDetector analyzes each user message for implicit correction signals ("actually", "that's wrong", "no, I meant") and extracts a UserCorrection when confidence meets correction_confidence_threshold. Corrections are stored in both the zeph_corrections SQLite table and the zeph_corrections Qdrant collection.

At context-build time, the top-K most similar corrections are retrieved by embedding and injected into the agent context, enabling cross-session personalization without explicit user re-stating preferences.

[!NOTE] Corrections are stored in the zeph_corrections Qdrant collection. If you use the sqlite vector backend, corrections are stored in the zeph_corrections SQLite virtual table instead.

ACP session storage

SqliteStore provides persistence for ACP session lifecycle, event replay, and per-session conversation isolation:

• create_acp_session_with_conversation(session_id, conversation_id) — creates a session record with an associated ConversationId foreign key (migration 026). Each ACP session maps to exactly one Zeph conversation.
• get_acp_session_conversation_id(session_id) — returns the ConversationId for a session, or None for legacy sessions created before migration 026.
• set_acp_session_conversation_id(session_id, conversation_id) — updates the conversation mapping for an existing session. Used to backfill legacy sessions on first resume.
• copy_conversation(source, target) — copies all messages and summaries from one conversation to another within a single transaction, preserving insertion order. Used by fork_session to clone history into a new isolated conversation.
• list_acp_sessions() — returns all sessions ordered by created_at DESC as Vec<AcpSessionInfo> (id + created_at). Used by _session/list to merge persisted sessions with in-memory state.
• import_acp_events(session_id, &[(&str, &str)]) — bulk-inserts events inside a single SQLite transaction. All events are written atomically (commit or rollback). Used by _session/import for portable session transfer.

[!NOTE] Event cascade delete is handled at the SQL level: deleting a session via delete_acp_session removes all associated events.

Graph memory

The graph module provides SQLite-backed entity-relationship tracking:

• Entities — named nodes with 8 types (person, tool, concept, project, language, file, config, organization)
• Typed edges — 8 relationship types (uses, related_to, part_of, depends_on, created_by, authored, manages, contains) enabling edge-type-aware traversal and filtering
• Entity canonicalization — canonical_name + alias table prevents duplicates from name variations ("Rust", "rust-lang", "Rust language" resolve to one entity). Alias-first resolution with deterministic first-registered-wins semantics
• Edges — directed relationships with bi-temporal timestamps (valid_from/valid_to for fact validity, created_at/expired_at for ingestion); edges_at_timestamp() returns edges valid at a given point in time, edge_history() returns all versions of an edge ordered by valid_from DESC, migration 030 adds partial indexes for temporal range queries
• Communities — groups of related entities detected via label propagation (petgraph) with LLM-generated summaries
• Graph eviction — automatic cleanup of expired edges, orphan entities, and entity cap enforcement via expired_edge_retention_days and max_entities config
• BFS traversal — cycle-safe breadth-first search with configurable hop limit; bfs_at_timestamp() variant traverses only edges valid at a given point in time for historical graph queries
• GraphFact — retrieval-side type with composite scoring for context injection; includes valid_from field for recency-aware scoring when temporal_decay_rate > 0
• graph_recall — query-time retrieval: splits the query into words, matches seed entities via FTS5 full-text index with BM25 ranking (including aliases), runs BFS up to max_hops, builds GraphFact structs with hop-distance-weighted composite scores, deduplicates by canonical name, and returns the top-K facts for context injection
• Embedding-based entity resolution — when use_embedding_resolution = true, entities are deduplicated via cosine similarity in Qdrant with a two-threshold approach (auto-merge at >= 0.85, LLM disambiguation at >= 0.70, new entity below); integrated after alias and canonical-name lookup steps; falls back to create-new on failure

GraphStore provides CRUD methods over five SQLite tables (graph_entities, graph_entity_aliases, graph_edges, graph_communities, graph_metadata). Schema is created by migrations 021, 023, and 024.

SemanticMemory::spawn_graph_extraction() runs LLM-powered extraction as a fire-and-forget background task with configurable timeout. recall_graph() performs fuzzy entity matching plus BFS edge traversal, returning composite-scored GraphFact values for context injection.

The HeuristicRouter in zeph-memory includes a Graph route variant: relationship queries (e.g., "related to", "connection between", "opinion on") are automatically routed to graph_recall.

Configure via [memory.graph] in config.toml:

[memory.graph]
enabled = true
max_hops = 2
recall_limit = 10
extraction_timeout_secs = 15
use_embedding_resolution = true     # semantic entity dedup via Qdrant (default: false)
entity_similarity_threshold = 0.85  # auto-merge threshold
entity_ambiguous_threshold = 0.70   # LLM disambiguation threshold
expired_edge_retention_days = 90    # Days to retain superseded edges
max_entities = 0                    # Max entities cap (0 = unlimited)
temporal_decay_rate = 0.0           # Decay rate for scoring older facts (0.0 = disabled); validated: must be in [0.0, 10.0], not NaN or Inf
edge_history_limit = 100            # Max edge versions returned by edge_history()

[memory.graph.spreading_activation]
enabled = false                     # Enable SYNAPSE spreading activation retrieval
lambda = 0.85                       # Decay factor per hop (energy × lambda at each step)
max_hops = 3                        # Maximum traversal depth from seed entities
max_activated = 50                  # Maximum nodes activated before stopping
timeout_ms = 500                    # Activation timeout to prevent runaway traversal

Importance scoring

Messages are scored at write time via compute_importance(). The score is stored in the importance_score column (default 0.5 for legacy rows). When importance_enabled = true on SemanticMemory, recall results are blended with importance scores for content-aware ranking.

Features

Installation

cargo add zeph-memory

# With PDF document loading
cargo add zeph-memory --features pdf

# With experiment result persistence
cargo add zeph-memory --features experiments

Documentation

Full documentation: https://bug-ops.github.io/zeph/

License

MIT