Crate omgbase_store

Expand description

§omgbase-store

The omgbase store, Rust implementation: the embedded SQLite database that owns block identity, history and the current state of a repository of authored files. Files stay the source of truth for content; the store adds stable block ids, an append-only history of revisions and commits, the dispositions the matcher recorded, and the derived indexes the query surfaces read. The contract is spec/store/README.md in the omgbase repository — schema.sql (embedded verbatim as SCHEMA_SQL) is the DDL, spec/store/cases/*.json the executable fixtures — and this crate opens the same databases the TypeScript reference writes.

use omgbase_reconcile::Config;
use omgbase_store::{BatchItem, Store};

let mut store = Store::open_in_memory()?;
let repo = store.create_repo("notes")?;
let ts = "2026-09-26T10:00:00.000Z";
let items = [BatchItem::observed("a.md", "# Title\n\nFirst.\n")];
let outcomes = store.observe_batch(&repo, &items, ts, &Config::default())?;
let o = outcomes[0].as_observed().unwrap();
assert!(o.converged && !o.echo);
assert_eq!(o.dispositions["inserted"], 2);
assert_eq!(store.reconstruct(&o.doc_id)?.as_deref(), Some("# Title\n\nFirst.\n"));

// Re-observing what the store holds is an echo: no commit, no mint.
let again = store.observe_batch(&repo, &items, ts, &Config::default())?;
assert!(again[0].as_observed().unwrap().echo);

§Layering

schema (§1, §3: the opener and migrations), ids (§2.1–2.2), time (§2.4), tree (§4.1 encodings), order_key (§4.3), writers (blobs, tree nodes, commits, revisions), observe (§5), properties (the properties rows of spec/properties, written in §5.4), history (the changes_since feed of spec/sync §6), graph (the nodes, external_nodes, edges and doc_edges rows of spec/graph, written in §5.4), search (spec/search: text_search, the embedding drain over the embeddings/doc_embeddings caches, vector, hybrid and resolve), read (§5.2, §6), derived (§4.5 sections, FTS, §7 rebuild and GC), and the spec/mutate host side (store 13.4): mutate (loading the working tree, apply with the file-CAS + known-id api commit), doc_store (the write-target seam), macros, links (link-destination rewriting), docs_ops (create / move / delete / set_meta, yaml_emit) and plan (the whole-document update planner).

Minted ids are opaque; the store asks its IdMinter for each one. The default is the CSPRNG-backed RandomMinter; a fixture runner installs a SequentialMinter through Store::open_in_memory_with_minter. Every draw passes through the checking Mint (§2.1, 13.5): an id already issued in this process or naming a row of its prefix’s table is redrawn.

Re-exports§

pub use derived::GcResult;
pub use derived::LIVE_LEAF_SQL;
pub use derived::RebuildTarget;
pub use doc_store::DocStore;
pub use doc_store::FsDocStore;
pub use doc_store::MemDocStore;
pub use doc_store::NullDocStore;
pub use docs_ops::DocMoveResult;
pub use docs_ops::DocOpContext;
pub use docs_ops::DocOpResult;
pub use docs_ops::Retargeted;
pub use error::Error;
pub use error::Result;
pub use graph::ResolvedEdge;
pub use history::ChangesPage;
pub use history::CommitDigest;
pub use history::DigestRevision;
pub use ids::IdMinter;
pub use ids::RandomMinter;
pub use ids::RepeatingMinter;
pub use ids::SequentialMinter;
pub use ids::is_valid_id;
pub use ids::prefix_of;
pub use links::InboundLink;
pub use macros::LinkRepair;
pub use macros::LinkRepairCount;
pub use macros::LinkRepairPlan;
pub use macros::RetargetHit;
pub use mint::Deferred;
pub use mint::MINT_GIVE_UP_AFTER;
pub use mint::Mint;
pub use mint::id_in_use;
pub use mutate::ApplyOrigin;
pub use mutate::ApplyRequest;
pub use mutate::ApplyResult;
pub use mutate::Diff;
pub use mutate::DocInfo;
pub use mutate::Revision;
pub use mutate::SetFrontmatter;
pub use mutate::find_doc_by_ref;
pub use mutate::is_id_ref;
pub use mutate::load_mut_doc;
pub use observe::BatchItem;
pub use observe::BatchOutcome;
pub use observe::Committed;
pub use observe::DeleteOutcome;
pub use observe::ObserveOutcome;
pub use observe::has_conflict_markers;
pub use read::RevisionRead;
pub use schema::SCHEMA_SQL;
pub use schema::SCHEMA_VERSION;
pub use search::BlockContexts;
pub use search::ContextScope;
pub use search::DocEmbedStats;
pub use search::DocVectorHit;
pub use search::DocVectorRow;
pub use search::DrainStats;
pub use search::EmbedStats;
pub use search::ForeignVectors;
pub use search::HybridHit;
pub use search::HybridQuery;
pub use search::QueryVector;
pub use search::ResolveHit;
pub use search::TextHit;
pub use search::TextSearchResult;
pub use search::VectorHit;
pub use search::block_vector;
pub use tree::TreeEntry;
pub use tree::canonical_attrs;
pub use tree::canonical_json;
pub use tree::serialize_tree_entries;
pub use tree::tree_hash;
pub use writers::NewCommit;
pub use writers::NewRevision;
pub use writers::Origin;
pub use writers::TreeInputBlock;
pub use self as mutate_kernel;

Modules§

derived: Derived tables (spec/store/README.md §3.2, §4.5, §7): sections, the FTS5 index over block text, rebuild and garbage collection, and the pool sweep.
doc_store: The doc store seam (spec/mutate/README.md §4 step 5, ADR-014): the write target of a mutation, keyed by repo-relative path. A filesystem store writes atomically (temp file + rename); an in-memory store backs the fixture runner (file-CAS and the written bytes stay checkable); a headless store does nothing.
docs_ops: Document operations (spec/mutate/README.md §6): create, move, delete, set-meta — api commits with generated reasons, each following the file-first protocol against a DocStore. Each has a dry run (spec/surface §4, 1.3) behind Store::dry_run: every check the real operation makes runs, nothing is written or committed, and the result carries the per-file Diffs the operation would produce.
error: The crate’s error type.
graph: The graph tables (spec/graph; spec/store §5.4 after the blocks, sections and properties): nodes + nodes_fts (§2), external_nodes and the edge resolution (§3.2), the edges intervals (§3.3), the doc_edges rollup (§3.4) and phantom adoption (§3.5). Extraction itself is omgbase-graph, pure; everything here reads or writes the database inside the commit transaction. Deletion (§3.6) touches none of it.
history: The change feed (spec/sync/README.md §6, the reference’s graph/history.ts changesSince): the repo’s commits after a cursor as digests, each with the revisions it wrote. seq is a dense per-repo total order, so a cursor is only meaningful against the repo it came from; head (the repo’s current max seq) tells “no new changes” from “cursor beyond this repo’s feed”.
ids: Identifiers (spec/store/README.md §2.1–§2.2): <prefix>_<7 chars> of lowercase Crockford base32 from a CSPRNG, and the minter seam that lets a fixture runner replace the CSPRNG with per-prefix counters. A minter only draws candidates; the store checks each against what is in use (crate::mint, §2.1) before handing it out.
links: Link-destination rewriting (spec/mutate/README.md §5 links_repair, §6 docs_move retargeting): the scanners the repair macro uses over a block’s raw (graph/link-destinations.ts), the destination-aware retarget of the inbound links to a moved document (graph/inbound-links.ts), and the inbound-link scan over the open edge index. Kept in the store for now — it reads blocks/edges — though the pure rewriting could move to omgbase-graph.
macros: Macros (spec/mutate/README.md §5): each expands deterministically to kernel ops the caller then applies (and sees). The expansions read the store — live hashes for the CAS tokens, the section runs, the nodes rows for node_set, the candidate blocks for links_repair.
mint: Uniqueness at mint (spec/store/README.md §2.1, store 13.5): every id the store hands out goes through Mint, which draws candidates from the inner IdMinter and redraws while a candidate is in use — already issued by this store in this process, or naming a row of the prefix’s table(s). 32⁷ candidates make a random collision certain by a few hundred thousand blocks (§10: a 3,000-document corpus failed at 360,338), so the check is what makes the production minter safe; the fixture minters never collide on a fresh database and pass through unchanged.
mutate: Changesets over the store (spec/mutate/README.md §1, §4): loading a document into the working tree, applying the ops in order across the documents they touch, and the commit protocol — file-CAS against the doc store, atomic write, and the api ingest with the ops’ known ids.
observe: Observation (spec/store/README.md §5): bytes at a path become a commit. Two passes — echo gate + reconcile without writes, the cross-document phase of spec/reconcile §7, then one transaction per member — plus the observed-deletion tombstone (§5.6).
order_key: Fractional order keys (spec/store/README.md §4.3): base-62 strings that sort bytewise among siblings; key_between(a, b) is strictly between its bounds.
plan: The whole-document update planner (spec/mutate/README.md §7): reconcile a proposed complete document against the stored tree (no pool), lower to kernel ops, verify by a dry-run apply, fall back to a full replace; and apply_opset with its preconditions.
properties: The properties table (spec/properties §6; spec/store §5.4 after the blocks refresh): the document’s rows are computed by omgbase-properties from the assigned body tree and the frontmatter block, deleted, then written with INSERT OR REPLACE on prop_id. Current-state only: rebuilt by re-ingest, never by rebuild_index.
read: Reads this spec pins (spec/store/README.md §5.2, §6): the old tree for the matcher, the pool, and byte reconstruction from the live rows or a revision’s Merkle root.
schema: The schema (spec/store/README.md §3): the embedded schema.sql, the opener and the migrations of §3.4.
search: Search over the store (spec/search, store 13.3): text_search (§1.3), the embedding tasks and the drain over the embeddings/doc_embeddings caches (§2), vector_search/doc_vector_search (§3), hybrid_search and resolve (§4). The pure pieces — sanitizer, inputs, pooling, cosine, fusion — are omgbase-search; this module runs the SQL around them.
time: Timestamps (spec/store/README.md §2.4): RFC 3339 UTC with exactly three fractional digits and a Z — JavaScript’s Date.toISOString(). Parsing accepts the format the store stores (an optional fraction of any length, truncated to milliseconds) so expires_ts = ts + 30 days can be computed without a calendar dependency.
tree: Canonical encodings (spec/store/README.md §4.1): tree-node entries, canonical attrs JSON and the Merkle node hash.
writers: Immutable writers (spec/store/README.md §4.1, §5.4 steps 4–6): blobs and tree nodes are content-addressed (INSERT OR IGNORE gives structural sharing); commits and revisions take the next per-repo / per-document sequence number.
yaml_emit: A YAML emitter matching the yaml npm package’s stringify defaults for the frontmatter docs_create/docs_set_meta compose (spec/mutate §6, §10 “the YAML emitter”): two-space indent, block sequences (indented under their key), plain scalars where legal — quoted when the text would resolve as another type, is empty, starts with an indicator, or contains : , #, a trailing space/colon — null as null. Fixtures pin flat mappings of strings, numbers, booleans and string lists; nested mappings and multi-line strings follow the same rules on a best-effort basis (long-line folding is not implemented).

Structs§

Boosts: The multiplicative boosts of a hit, each present only when it applies.
Config: The thresholds and weights (§6), named as the fixtures spell them. The reference’s ReconcileConfig uses the camelCase spellings.
DocEmbedBlockRef: A block’s contribution to a pooled document vector (§2.5): where its cached vector is keyed and its weight.
DocEmbedTask: One document to embed (§2.4).
EdgeDescriptor: An extracted edge before resolution (§3.1).
EmbedTask: One block to embed (§2.2–§2.3).
Evidence: The evidence a hybrid hit carries (§4 step 5).
Expect: §1.2 expectations.
MatchBlock: A flattened block (§1.2). Old blocks carry their id; new blocks do not until assigned. The positional key is stable within one tree and lets the phases compare parents and sibling order across the two trees — it is not an identity.
MutBlock: A block of the working tree (§1 MutBlock).
MutDoc: A document in the working tree (§1 MutDoc).
MutationError: A refused mutation: the reference’s MutationError (code, message, data).
OpResult: What an op returns (§2): the ids, plus removed (remove) or merged_into (merge).
Opset: §7 Opset.
PlanOp: One kernel op plus the identity consequence the planner attributes to it.
PoolEntry: A resurrection-pool entry (§5 phase 6b): a block deleted in an earlier checkpoint, matched by hash only.
ProjectedNode: One projected node before it has an id (§2.1, §2.2).
PropertyRow: One property row (§1).
Store: An open store: one connection, one id minter (behind the in-use check of mint). All writes go through it.
To: §1.1 placement.

Enums§

At: §1.1 To.at.
DocEmbedMethod: How a document vector was (or would be) computed (§2.4).
Op: One kernel op as a changeset carries it (§4 Op). String fields may be placeholders until resolve_op runs.
Parent: §1.1 To.parent.

Constants§

FORMAT_MARKDOWN: The one format this crate ingests (docs.format).
SPEC_VERSION: The spec/store/VERSION this crate implements (major.minor); the major is SCHEMA_VERSION.
VERSION: This crate’s own version (Cargo.toml), for the surface’s version tool.

Traits§

EmbeddingProvider: An embedding model: an external process or endpoint (see crate::external) or, in a runner, the FixtureEmbedder.

Functions§

cosine_bytes: Cosine similarity of two little-endian float32 vector blobs (spec/search §3, the reference’s cosineFloat32): products and sums in f64 over the shorter length (each blob floored to a multiple of 4 bytes); 0.0 when either norm is zero.