Expand description
§semantic-memory
Local-first semantic memory backed by authoritative SQLite state and an optional recoverable HNSW sidecar.
The crate stores facts, chunked documents, conversation messages, and searchable episodes in
SQLite. Search combines BM25 (FTS5) and vector retrieval with Reciprocal Rank Fusion, and
search_explained() returns the exact scoring breakdown from the live pipeline.
Concurrency uses one writer connection plus a pool of WAL-enabled reader connections. Durable writes are committed to SQLite first; any required HNSW sidecar mutations are journaled in SQLite and replayed on open, flush, rebuild, or reconcile.
search() targets facts, document chunks, and episodes by default. Message retrieval is
available through search_conversations() or by opting into
SearchSourceType::Messages.
Integrity tooling is strict about malformed stored data: invalid roles, JSON, enums, embedding
blobs, quantized blobs, and sidecar drift are surfaced through verify_integrity() instead of
being silently converted into defaults. reconcile() can rebuild FTS or fully re-embed and
rebuild derived state from SQLite.
store.graph_view() exposes a deterministic graph traversal layer over namespaces, facts,
documents, chunks, sessions, messages, episodes, and semantic/temporal/causal links derived
from SQLite state.
§Quick Start
use semantic_memory::{MemoryConfig, MemoryStore};
let store = MemoryStore::open(MemoryConfig::default())?;
// Store a fact
store.add_fact("general", "Rust was first released in 2015", None, None).await?;
// Search
let results = store.search("when was Rust released", None, None, None).await?;§Operational Notes
- SQLite is authoritative for all durable records and embeddings.
- HNSW is an acceleration sidecar. Pending sidecar mutations are journaled in SQLite, so a sidecar failure does not imply the SQLite write rolled back.
- WAL mode plus pooled reader connections allows concurrent reads while writes serialize through the writer connection.
search_explained()reflects the exact ranking math used by the active search pipeline, including reranking from exact f32 cosine similarity when configured.
Re-exports§
pub use config::ChunkingConfig;pub use config::DerivedVectorBackendPolicy;pub use config::EmbeddingConfig;pub use config::MemoryConfig;pub use config::MemoryLimits;pub use config::PoolConfig;pub use config::SearchConfig;pub use embedder::Embedder;pub use embedder::MockEmbedder;pub use embedder::OllamaEmbedder;pub use error::MemoryError;pub use quantize::pack_quantized;pub use quantize::unpack_quantized;pub use quantize::QuantizedVector;pub use quantize::Quantizer;pub use storage::StoragePaths;pub use tokenizer::EstimateTokenCounter;pub use tokenizer::TokenCounter;pub use types::ChunkManifestChunkMapping;pub use types::ChunkManifestEntry;pub use types::ChunkManifestIngestOptions;pub use types::ChunkManifestIngestResult;pub use types::DerivedCandidateReceiptV1;pub use types::Document;pub use types::EmbeddingDisplacement;pub use types::EpisodeAsOfReceiptV1;pub use types::EpisodeMeta;pub use types::EpisodeOutcome;pub use types::ExactnessProfile;pub use types::ExplainedResult;pub use types::ExplainedResultAnswerV1;pub use types::ExplainedSearchResponse;pub use types::Fact;pub use types::GraphDirection;pub use types::GraphEdge;pub use types::GraphEdgeType;pub use types::GraphView;pub use types::MemoryStats;pub use types::Message;pub use types::NamespaceDeleteReport;pub use types::ProjectionClaimVersion;pub use types::ProjectionEntityAlias;pub use types::ProjectionEpisode;pub use types::ProjectionEvidenceRef;pub use types::ProjectionQuery;pub use types::ProjectionRelationVersion;pub use types::ProveKvPoolArtifactBuildReceiptV1;pub use types::ProveKvPoolArtifactStatusV1;pub use types::ProveKvPoolGenerationStatus;pub use types::ProveKvPoolGenerationV1;pub use types::ProveKvPoolItemMapEntryV1;pub use types::ReceiptMode;pub use types::Role;pub use types::ScoreBreakdown;pub use types::SearchContext;pub use types::SearchReceiptAnswersV1;pub use types::SearchReplayReportV1;pub use types::SearchResponse;pub use types::SearchResult;pub use types::SearchSource;pub use types::SearchSourceType;pub use types::Session;pub use types::TextChunk;pub use types::VectorArtifactBuildReceiptV1;pub use types::VectorSearchReceiptV1;pub use types::VerificationStatus;pub use vector_backend::VectorBackend;pub use vector_backend::VectorHit;pub use vector_backend::VectorIndex;pub use vector_backend::VectorIndexConfig;pub use vector_codec::RawF32Codec;pub use vector_codec::Sq8Codec;pub use vector_codec::VectorArtifactV1;pub use vector_codec::VectorCodec;pub use vector_codec::VectorCodecProfileV1;pub use vector_snapshot::build_embedding_snapshot;pub use vector_snapshot::EmbeddingSnapshotRow;pub use vector_snapshot::EmbeddingSnapshotV1;
Modules§
- chunker
- Text chunking via recursive splitting with overlap.
- config
- embedder
- Embedding trait and implementations.
- error
- quantize
- Scalar quantization (SQ8) for f32 → i8 vector compression.
- quantize_
governed - Governed compression pipeline integration for semantic-memory.
- search
- Hybrid search engine: BM25 + vector similarity + Reciprocal Rank Fusion.
- storage
- Storage path management for the memory directory convention.
- tokenizer
- Pluggable token counting for context budget management.
- types
- vector_
backend - Vector index backend trait.
- vector_
codec - Vector codec profile and artifact boundary.
- vector_
snapshot - Deterministic snapshots over authoritative SQLite f32 embeddings.
Structs§
- AddGraph
Edge Params - Parameters for creating a graph edge.
- Integrity
Report - Result of an integrity verification.
- Memory
Store - Thread-safe handle to the memory database.
- Projection
Import Failure Receipt Entry - Public view of a durable failed projection import receipt.
- Projection
Import LogEntry - Public view of a V11 projection import log entry.
- Projection
Import Result - Result of a projection batch import (V11+).
- Stored
Graph Edge - A stored graph edge row.
Enums§
- Reconcile
Action - Action to take when integrity issues are found.
- Verify
Mode - How thorough the integrity check should be.
Functions§
- bytes_
to_ embedding - Decode a SQLite embedding BLOB back to f32 values.
- decode_
f32_ le - Decode a stable little-endian f32 persisted representation.
- embedding_
to_ bytes - Encode an f32 slice as bytes for SQLite BLOB storage.