Skip to main content

Crate topodb

Crate topodb 

Source
Expand description

§topodb

The memory terrain for AI agents — embedded, temporal, graph-native.

TopoDB is an embedded, local-first memory engine for AI agents, written in pure Rust: a scoped temporal property graph on redb, with an op-log write path, disk-resident MVCC reads, and k-hop temporal traversal — running in-process, no server.

Status: early development (0.0.x) — the engine core and the recall layer are implemented: op log, single-applier concurrency, scoped temporal traversal, BM25 full-text search, graph-scoped vector search, access stats, change feed, and replay-determinism property tests. The API is not yet stable; pin exact versions.

use topodb::{
    Db, Direction, IndexSpec, NodeId, Op, PropIndex, PropValue, Scope, ScopeId,
    ScopeSet, TraversalQuery, VectorQuery,
};

fn main() -> Result<(), topodb::TopoError> {
    // Declare what gets indexed up front (Db::open defaults to NO indexes;
    // the CLI/MCP layers declare this same spec for you automatically).
    let spec = IndexSpec {
        equality: vec![PropIndex { label: "Entity".into(), prop: "name".into() }],
        text: vec![PropIndex { label: "Memory".into(), prop: "content".into() }],
    };
    let db = Db::open_with("memory.topodb", spec)?;
    let scope = ScopeId::new();
    let (a, b) = (NodeId::new(), NodeId::new());

    // Every mutation is a batch of ops, applied atomically.
    db.submit(vec![
        Op::CreateNode {
            id: a,
            scope: Scope::Id(scope),
            label: "Memory".into(),
            props: [(
                "content".to_string(),
                PropValue::Str("ada wrote the first program".into()),
            )]
            .into(),
        },
        Op::CreateNode {
            id: b,
            scope: Scope::Shared,
            label: "Entity".into(),
            props: Default::default(),
        },
        Op::CreateEdge {
            id: Default::default(),
            scope: Scope::Id(scope),
            ty: "ABOUT".into(),
            from: a,
            to: b,
            props: Default::default(),
            valid_from: None,
        },
        // Embeddings are host-computed and submitted as ops (engine, not policy).
        Op::SetEmbedding { id: a, model: "my-embedder".into(), vector: vec![0.1, 0.2, 0.3] },
    ])?;

    // Every read is scoped — there is no unscoped read path.
    let scopes = ScopeSet::of(&[scope]).with_shared();

    // BM25 full-text recall.
    let _hits = db.search_text(&scopes, "first program", 10)?;

    // Vector recall, scoped, under the same model namespace.
    let _near = db.search_vector(&VectorQuery {
        scopes: scopes.clone(),
        model: "my-embedder".into(),
        vector: vec![0.1, 0.2, 0.3],
        k: 10,
        candidates: None,
    })?;

    // Scoped k-hop temporal traversal.
    let _sub = db.traverse(&TraversalQuery {
        scopes,
        seeds: vec![a],
        max_hops: 2,
        edge_types: None,
        direction: Direction::Out,
        as_of: None,
    })?;
    Ok(())
}

§Design principles

  1. Narrow and deep — an agent-memory engine, not a general graph database
  2. Format stability is a feature — versioned on-disk format, migrations always
  3. Honest benchmarks from day one
  4. Engine, not policy — no LLM calls inside the database, ever
  5. Embedded-first — servers and sync are future layers, never prerequisites

§Core properties

  • Temporal edges — facts supersede, never overwrite; as_of reads see history
  • Structural scoping — every read takes a ScopeSet; cross-scope edges require a Shared endpoint
  • Deterministic replay — the op log stores fully-resolved ops; replaying it reproduces state exactly (property-tested)
  • Single-applier concurrency — writers from any thread serialize through one applier; reads run in redb MVCC read transactions, never block each other, and never block redb’s storage commits, though a long-running read can briefly delay the applier’s next batch while it holds the dicts/scope-registry read guards

License: MIT OR Apache-2.0.

Structs§

AccessStats
Per-node access statistics: how many times a node has been returned by a scoped read, and the wall-clock millisecond timestamp of the most recent such read. Default (both zero) means “exists but never counted”.
AppliedBatch
The committed result of a successful crate::Db::submit/ crate::Db::submit_at call: the inclusive [first_seq, last_seq] range the batch’s ops were assigned in the durable op log, and the batch’s ops in their fully-resolved form (timestamps filled in) as actually written.
ChangeEvent
A single committed op paired with its op-log sequence number.
Db
A handle to an open database. Cloning shares the same underlying storage and applier thread — Db is Send + Sync + Clone. All writes funnel through a single applier thread (via submit/submit_at), so batches serialize deterministically even under concurrent callers.
DbOptions
Tuning knobs for Db::open_with_options. Additive: every field defaults to None, under which redb’s own default is used, so a fresh DbOptions::default() behaves identically to Db::open/Db::open_with.
EdgeId
EdgeRecord
IndexSpec
Declares which (label, prop) pairs get indexed, and how.
LinkSuggestion
One suggested (but non-existent) edge endpoint, with evidence. Never a typed edge — creating one, and typing it, is host/agent policy.
NodeId
NodeRecord
PropIndex
A single declared (label, prop) pair to index.
PropRetain
Mechanism-only string-prop allowlist for search results: a candidate survives iff the value of prop — a Str; a missing or non-Str value reads as absent_as when that is set, otherwise never matches — equals one of any_of. Applied BEFORE top-k truncation (a filtered hit never consumes the result window) and never access-bumped on a dropped node. The engine names no prop and no vocabulary — the host owns both (e.g. a host-defined category prop, where the host passes the category’s default as absent_as so an unstamped node matches it).
RecallQuery
One production hybrid-recall request. The engine fuses; the HOST resolves — vector is a pre-computed query embedding, expansions are pre-resolved synonym terms. The engine neither knows nor cares where either came from (spec: engine mechanics, host policy).
ScopeId
ScopeSet
The mandatory scope filter for every read. There is no unscoped read.
SearchOptions
Tuning for Db::search_text_with. Default disables every option, so search_text_with(scopes, query, k, &SearchOptions::default()) behaves identically to Db::search_text.
Subgraph
Result of a traversal: every in-scope seed plus everything reached, deduped, with the full edge records (fetched from the EDGES table by slot) for every traversed edge.
SuggestLinksQuery
A suggest_links request. model names the vector namespace for the semantic leg; None (or no stored embedding under it) = structure-only. as_of pins the adjacency view, None = now (traverse’s convention).
TraversalQuery
A bounded, scoped, temporal breadth-first traversal request.
VectorQuery
A vector-search request: cosine-rank the embeddings under model within scopes against vector, returning the top k.

Enums§

Direction
Which adjacency to walk from each frontier node.
Op
A fully-resolved mutation. INVARIANT: ops are appended to the log only after the applier resolves every default (timestamps especially) — a stored op never contains “now”. Replay must be deterministic.
PropValue
Scope
TopoError

Functions§

analyze
The v1 analyzer as a standalone utility for HOSTS that must agree with the engine’s tokenization (e.g. storing synonym terms in stemmed form so query-time lookup can’t miss a morphological variant). Same pipeline search_text/indexing use.

Type Aliases§

Props