Crate fathomdb

Expand description

FathomDB — a local-first retrieval and graph-oriented data system for application and agent workloads.

This is the crate to depend on from Rust. It is a thin facade that re-exports the public surface of the fathomdb-engine runtime, so you get the supported API without depending on engine internals.

FathomDB embeds SQLite (FTS5 + sqlite-vec) in your process — there is no server and no sidecar. One Engine owns the writer thread, a reader pool, the projection scheduler and (optionally) an in-process embedder.

§Do you want this crate?

Yes, if you want hybrid retrieval — a vector branch and an FTS5 branch fused by Reciprocal Rank Fusion — over data you also want to address by a stable identity, traverse as a graph, and be able to delete on request.
No, if you want a client for a remote database, or an approximate nearest-neighbour index at very large scale: vector retrieval here is a full scan, so latency grows with corpus size.

Related crates: fathomdb-cli (the operator binary — doctor / recover), fathomdb-embedder-api (the semver-stable embedder trait, versioned independently), and fathomdb-engine (the runtime this crate re-exports; prefer this facade).

§Example

use fathomdb::{Engine, PreparedWrite, SourceId};

let opened = Engine::open("./example.fdb")?;
let engine = opened.engine;

engine.write(&[PreparedWrite::Node {
    kind: "note".into(),
    body: "the sky is blue".into(),
    // Provenance is MANDATORY — see below.
    source_id: SourceId::new("doc-42")?,
    logical_id: Some("note:sky".into()),
    state: Default::default(),
    reason: None,
    valid_from: None,
    valid_until: None,
}])?;

for hit in engine.search("sky")?.results {
    // `hit.id` is a typed id-space carrier, not a row number.
    println!("{:?} {} {}", hit.id.space, hit.id.value, hit.body);
}

engine.close()?;

§Provenance is mandatory

Every canonical node and edge carries a SourceId. This is a type rather than a validation check on purpose: Engine::erase_source addresses rows by their source_id, so a row written without one could never be erased on request. SourceId::new is the only public constructor and refuses an empty id and the engine’s reserved _-prefixed namespace, which makes an un-provenanced write a compile error rather than a runtime surprise.

Treat a source_id as a public identifier: it is echoed on every search hit and recorded in a retention-exempt erasure-audit row, so keep personal data out of it.

§Deletion on request

Three verbs, differing in what they address, all on the default surface:

Engine::transition — move a governed node between existence states (promote, soft-delete, undelete).
Engine::purge — irreversibly hard-erase one governed node, addressed by its logical_id. Deleted-first and idempotent. There is no restore.
Engine::erase_source — erase every row carrying one source_id, including anonymous rows that have no logical_id and that Engine::purge therefore cannot reach.

§Feature flags

default — the governed application surface (dev/interfaces/rust.md § Governed-surface contract): recovery-name-free and raw-SQL-free at the method level. No method named recover, restore, repair, fix or rebuild resolves.
operator — un-gates the operator/recovery seam (rebuild_*, excise_source, dump_*, trace_source_ref, truncate_wal, verify_embedder, check_integrity, safe_export, recompute_mean and their report types). fathomdb-cli enables it. Gating, not deletion: engine behaviour is identical with the feature on. See AC-074 (dev/acceptance.md) and dev/design/slice-27-fix1-operator-gate-design.md.

§Stability

Pre-1.0, so beta: the surface may change between micro releases. The governed surface is pinned by src/conformance/governed-surface-allowlist.json and any change to it is a reviewed delta, but that is a change-control promise, not a semver one. PreparedWrite and SearchFilter are #[non_exhaustive].

Structs§

BoundaryCrossing: 0.8.20 Slice 10b (R-20-NV) — one node that crossed a validity boundary inside the interrogated interval, as reported by Engine::crossed_boundary_since.
CorruptionDetail: Stable corruption-on-open detail carried by EngineOpenError::Corruption.
CounterSnapshot: Snapshot of engine-internal counters returned by Engine::counters.
Engine
ExciseReport: Phase 9 Pack B excise report (AC-028a/b/c). Counts are post-excise totals; projections_invalidated reports the shadow-row invalidation total (FTS5 + vec0 + projection terminal) for the excised source.
Explanation: 0.8.8 EXP-OBS (Slice 5) — the opt-in retrieval explanation payload returned behind search_explained (the explain=true surface). Built from the engine’s OWN fusion/rerank machinery (fuse_three_arms per-arm ranks, ce_rerank blend components) — no parallel machinery (R-OBS-3). Carries a query-level QueryTrace plus a per-hit breakdown parallel to (and in the same order as) SearchResult.results.
ExtractDocument: G11 (Slice 15) — a document sent to a BYO-LLM extraction harness via Engine::ingest_with_extractor.
IngestWithExtractorReceipt: G11 (Slice 15) — receipt returned by Engine::ingest_with_extractor.
NodeRecord: Slice 30 (G2) — an active canonical node row returned by read.get / read.get_many.
OpenReport
OpenedEngine
PerHitExplain: 0.8.8 EXP-OBS (Slice 5) — per-hit provenance + score breakdown. One entry per returned SearchHit, same order. *_rank is the 0-based rank the hit’s body held in that arm’s pre-fusion list (None = absent from that arm).
ProjectionDelta: 0.8.20 Slice 15d (R-20-PR) — the diff Engine::configure_projections applied. Idempotent re-registration yields unchanged == true with all vecs empty (the “re-registration is a no-op” acceptance signal). A destructive change without an explicit drop is an Err, not a delta.
ProjectionFts: 0.8.20 Slice 15d (R-20-PR) — the searchable→FTS sub-target selector.
ProjectionSpec: 0.8.20 Slice 15d (R-20-PR / C-1) — a single declarative projection declaration. HITL-ratified shape (api-surface.md:85-89): { name, roles: Set<ProjectionRole>, fts?, vector? }. roles carries SET semantics (dedup + membership; an attribute can be Filterable AND Searchable) — encoded here as a sorted, de-duplicated BTreeSet. Named roles, not kind (kind is the node/edge type discriminator).
ProjectionVector: 0.8.20 Slice 15d (R-20-PR) — the searchable→vector sub-target selector.
QueryTrace: 0.8.8 EXP-OBS (Slice 5) — query-level retrieval trace. Reuses the existing search_reranked knobs + the active embedder identity; timings are coarse per-stage wall-clock (monotonic) captured only on the explain path.
ReadView: 0.8.20 Slice 10b (R-20-RV / R-20-NV) — the read view: the single knob that decides which canonical_nodes rows a read verb may see.
RecoveryHint: Recovery dispatch surface attached to a corruption detail.
SearchExpandResult: Slice 20 (G6) — result of Engine::search_expand: initial search hits plus nodes reached by bounded BFS expansion that are not already in the search hit set.
SearchFilter: G10 — closed metadata filter for Engine::search_filtered (Slice 10).
SearchResult: Hybrid search result. results carries structured SearchHits in vector-first, dedup-on-body order. Derives Clone, Debug, PartialEq but not Eq — each hit carries a score: f64.
SoftFallback: Soft-fallback signal carried on hybrid search results.
SourceId: 0.8.20 Slice 5c (R-20-E3) — the provenance of a canonical row: which source document it is attributable to, and therefore what excise_source must erase when that source is withdrawn.
Subscription: Handle returned by Engine::subscribe.
WriteReceipt

Enums§

ComparisonOp: G4 (Slice 35) — comparison operator for Predicate::JsonPathCompare.
CorruptionKind: Open-path corruption category.
CorruptionLocator: Locator pointing at the corrupted region of the database file.
DenseReadiness: 0.8.20 Slice 20 (R-20-DR) — the ENGINE-SET readiness of the searchable→vector projection, per dev/design/record-lifecycle-protocol/projection-registry-and-async-embed.md §3.
EngineError
EngineOpenError
InitialState: OPP-12 Phase-1 (0.8.19 Slice 5) — the CREATE-TIME subset of LifecycleState.
LifecycleState: OPP-12 record-lifecycle Phase-1 (0.8.19 Slice 5) — the existence axis.
OpenStage: Engine.open stage at which corruption was detected.
Predicate: G4 (Slice 35) — closed typed predicate for Engine::read_list filter.
PreparedWrite: Batch input shape for Engine::write.
ProjectionRole: 0.8.20 Slice 15d (R-20-PR, C-1) — one member of a ProjectionSpec’s role set. Exactly three members (HITL-ratified S8, api-surface.md:87): searchable→FTS and searchable→vector are NOT roles — they are tier labels carried by the fts/vector sub-objects of the spec, so an attribute is Searchable once and the sub-objects select FTS-only / vector-only / both.
ScalarValue: G4 (Slice 35) — scalar value for Predicate comparisons.
SoftFallbackBranch: Which retrieval branch produced a hit (or could not contribute).
TraversalDirection: Slice 20 (G5) — direction of graph traversal for Engine::graph_neighbors / Engine::search_expand.

Crate fathomdb

Crate fathomdb Copy item path

§Do you want this crate?

§Example

§Provenance is mandatory

§Deletion on request

§Feature flags

§Stability

Structs§

Enums§

Crate fathomdb