tokio-agent-memory: a vector store finds what is similar, never what caused what
Why this exists: Agent memory gets built as a vector database, so an agent can retrieve the passage most like the question and cannot answer "what happened after the deploy failed" or "which fact did this replace".
Why this instead of the usual approach: Vector stores do similarity and nothing else. This keeps episodic events with causal links, a semantic fact graph, priority-based working memory and decay as plain Rust data structures in your process, with search by meaning (local or API embeddings) on top and a broadcast bus so several agents share one memory.
Why adopt it today: cargo add tokio-agent-memory, record events as your agent acts, and ask for causes and recent context without standing up a database.
Built for: Rust agent builders whose agents run for hours and need cause-and-effect and forgetting, not only nearest-neighbour recall.
Tradeoffs we chose on purpose:
- In-process: memory size is bounded by RAM; persistence is file snapshots, not a server.
- Decay forgets on purpose. Pin what must never fade.
Memory for Tokio-based LLM agents: an episodic event log with cause-and-effect links, a semantic fact graph, a fixed-size working memory, confidence decay, retrieval (exact, fuzzy, time range, tags), JSON file persistence and a broadcast bus for sharing memories between agents.
An agent that runs for hours needs more than its context window. This crate gives you the usual memory tiers as plain in-process data structures you can wire into any agent loop, with no database or server required.
Features
| Module | What it does |
|---|---|
episodic |
EpisodicMemory: record Events, recall them by time range, follow caused_by links with causal_chain, forget by id |
semantic |
SemanticMemory: subject, relation, object facts (Concept, Relation), query, multi-hop related(concept, depth), retract |
working |
WorkingMemory: fixed number of slots with SlotPriority (Low, Normal, High, Critical); when full, the lowest-priority oldest item is evicted |
decay |
DecayScheduler with Exponential or Linear confidence decay and a forget threshold; items with importance above 0.8 are never forgotten |
retrieval |
retrieve_from_items with RetrievalMode::Exact (tag), Fuzzy (skim fuzzy matching), Temporal, TagIntersection, plus a confidence floor and limit |
consolidation |
Consolidator: moves working-memory items into semantic memory (importance at or above a threshold) or episodic memory (the rest) |
persistence |
MemoryStore trait with InMemoryStore and FileStore (a JSON file, written on flush) |
shared |
SharedMemoryBus: Tokio broadcast channel of MemoryItems plus time-limited write leases per key |
semantic_search |
SemanticIndex: find memories by meaning with embeddings, from a local model (fastembed feature) or any OpenAI-compatible API (openai feature) |
All types are Send + Sync (backed by DashMap and parking_lot), and the crate denies unwrap, expect and panic through Clippy lints.
Quick start
use Duration;
use ;
async
Search by meaning
Tag and fuzzy retrieval only find memories that share words with the question. SemanticIndex embeds each memory once and ranks memories by cosine similarity to the question, so "any dietary restrictions I should know about?" finds "User is vegetarian and allergic to peanuts".
= { = "0.2", = ["fastembed"] } # local model, no API key
# or features = ["openai"] for OpenAI, Ollama, vLLM, LM Studio, text-embeddings-inference
use ;
use ;
async
With the openai feature, use OpenAiEmbedder::new("text-embedding-3-small").with_api_key(key), or .with_base_url("http://localhost:11434/v1") for Ollama. Any other model works through the one-method Embedder trait. Each memory is embedded once (in a batch) and cached; only the current items you pass are searched, so forgotten memories do not come back. The fastembed feature needs Rust 1.88.
Small local models match paraphrases well (the scores above are from the crate's test against the real model); questions that need a reasoning step ("what can I cook tonight?" for a dietary memory) need a larger embedding model.
Layout
src/
types.rs MemoryId, AgentId, MemoryItem (JSON content, tags, confidence, importance)
episodic/ EpisodicMemory, Event, CausalChain
semantic/ SemanticMemory, Concept, Relation, Fact, ConceptGraph
working/ WorkingMemory, SlotPriority
decay/ DecayScheduler, DecayPolicy
retrieval/ retrieve_from_items, MemoryQuery, RetrievalMode
consolidation/ Consolidator, ConsolidationPolicy
persistence/ MemoryStore, InMemoryStore, FileStore
shared/ SharedMemoryBus, WriteLease
semantic_search.rs Embedder, SemanticIndex, FastEmbedder, OpenAiEmbedder
tests/memory_integration.rs cross-module scenarios
tests/semantic_search_model.rs meaning-based recall against the real model (opt-in)
Status and limitations
Version 0.2. Everything lives in process memory; FileStore is the only on-disk option (the semantic index is in memory too). Retrieval is lexical (tags, fuzzy matching) or by meaning (SemanticIndex). The MemoryStore trait is synchronous. For a similar crate with a trait-based episodic/semantic store design, Ebbinghaus decay and binary snapshots, see tokio-memory.
License
MIT, see LICENSE.
Part of a set of Rust crates for LLM agents, see rust-crates.