Crate datumesh_store 

A store for Datumesh: function descriptors, receipts, blobs, versions of cells over transaction time, lineage, namespaces and function names, on PostgreSQL or (since 0.4.0) in one SQLite file. PgStore and SqliteStore implement Store, and through it datumesh_core’s Resolver, so datumesh_core::verify can recompute receipts of stored cells from either; the two behave the same (one conformance suite runs against both).

The API is synchronous (the postgres driver and a small connection pool; rusqlite with SQLite compiled in, feature sqlite, on by default). Async callers run it on a blocking thread, e.g. tokio::task::spawn_blocking.

SqliteStore::open takes a file path (SqliteStore::connect a sqlite: URL); the file is in WAL mode, every write is one BEGIN IMMEDIATE transaction, and the append-only rules are triggers in the file, so they hold for writers in several threads and processes on one machine. Its schema comes from migrate::sqlite::apply.

PgStore::connect / PgStore::connect_with take a URL and PoolOptions; TLS (rustls) is used as the URL’s sslmode says and always verifies the server’s certificate and host name, against the Mozilla roots plus those added with PoolOptions::tls_root_certificates_pem. No type of the driver or of a TLS crate is part of the API, except in pg_unstable (any connection function, the driver itself), which is outside semver until 1.0.

What is stored, and how:

• Descriptors are checked by the core (FunctionDescriptor::from_json: since 0.3.1, spec-0.4.0’s, with signature.order and argument extent) and stored in canonical form (JCS) under their hash. Receipts are verified (datumesh_core::verify, SPEC 10.1 steps 1-6) against the key document stored for their signer host, so nothing unverified enters the store, and are stored canonical under their hash. Storing the same thing twice stores it once. A second, different valid signature over the same payload (possible only with a randomised signer: the payload names the key, so a rotated key gives a new hash) is refused (Error::Conflict) and the stored receipt kept.
• Blobs are stored in the database, under the SHA-256 of their content.
• Versions: from transaction time tx on, cell cell_id has the value of a receipt. Append-only and in tx order per cell, enforced by the database (triggers) as well as by Store::put_version, which also checks that the receipt is stored and is of that cell at that tx.
• Computed versions: from transaction time tx on, cell cell_id has the value a receipt computed, as of the receipt’s asof. Derived rather than facts, so they are added in any order, keyed on (cell, tx, asof), insert-only (Store::put_computed, Store::computed_asof).
• Answering as of A: a receipt answers as of A only when its tx and its asof are both at or before A (Store::stored_asof, and the Resolver’s evaluate of a stored function; Store::version_asof looks at tx only).
• Lineage: put_receipt records an edge to each of the receipt’s inputs, held or not: receipt hashes (Store::inputs, Store::dependents) and, since 0.3.1, slice entries (Store::slice_entries, Store::slice_dependents); Store::cell_dependents lifts both to cells.
• Slices (0.3.1, SPEC 15.13): every stored receipt’s cell is indexed by its typed arguments, so that Store::slice_members finds the members of a slice as of A with a range scan, in key order; Store::versions_between gives a version slice. Store::slice_values gives the members’ values without receipts (0.3.2) and Store::slice_changed_after whether they may answer differently later. Manifests (Store::put_manifest, served by Resolver::manifest) are kept by entry and found from each member (Store::member_manifests). A receipt citing a slice can be stored before its manifest, and a manifest before its members.
• Namespaces: a host’s key document (checked as SPEC 9.2 requires), with when it was validated. Store it before the host’s receipts. Replacing it later (a key revoked or dropped) does not touch receipts already stored: they verified when they were stored, and putting one of them again is still idempotent.
• Names: which function hash a fully qualified function name meant from a transaction time on. Append-only, like versions. Store::resolve_binding gives the binding answering as of A with when it began; Store::resolve_name its hash.

Instants are compared as text (fixed width, COLLATE "C"; SQLite’s default collation is the same byte order), which is exact; on PostgreSQL each is also stored as timestamptz, which round-trips every instant of the specification exactly. Numbers in the typed-argument index compare as exact decimals (PostgreSQL numeric; on SQLite a text encoding that sorts as the decimal does).

Every Store method returns a Result: a malformed argument is Error::Invalid with the core’s code, a database failure Error::Database. The Resolver methods keep the core’s Option signatures; there a failure is logged with tracing (target datumesh_store) and answered None, which verify reports as unavailable.

Store is sealed: only this crate implements it. The store is the trust boundary: everything in it was checked on the way in (every receipt verified against its host’s key document), and its callers rely on that rather than checking again. A dyn Store is still a trait object as before.

The schema comes from migrate::apply (PostgreSQL) or migrate::sqlite::apply (SQLite) (the datumesh-store migrate command, for either), through a store on the (possibly empty) database.

Modules

migrate

Schema migrations: plain SQL files (migrations/*.sql, embedded), applied in order with a tracking table, and a schema check that compares the live database’s catalog with the catalog the migrations produce in a scratch database.

pg_unstable

Outside semver until 1.0: this module exposes the postgres driver’s types, and any release may change it, including a patch release.

Structs

Binding

A name binding: from transaction time tx on, a fully qualified function name means the function fn_hash (Store::bind_name). Since 0.3.0.

DatabaseError

A database failure, Error::Database: what the database said (PostgreSQL: the server’s message and SQLSTATE; SQLite: its message and result code), or the driver’s description of a failure that is not the database’s.

Namespace

A namespace’s host and its key document.

PgStore

The store on PostgreSQL, with a connection pool. Send + Sync: share one per process (behind an Arc); each call checks a connection out for its duration.

PoolOptions

How the pool behaves, and the roots TLS connections are verified against.

SliceMember

A member of a slice (Store::slice_members, Store::versions_between): a stored version and the bytes of its receipt (as receipt returns them). Since 0.3.1.

SliceValue

A member of a slice with its value and without its receipt’s bytes (Store::slice_values). Since 0.3.2.

SqliteOptions

How a SqliteStore opens and uses its connections.

SqliteStore

The store in one SQLite file (bundled SQLite, no server), with the same behaviour as PgStore: every method, error and rule of Store and Resolver. Send + Sync: share one per process (behind an Arc); each call takes a connection for its duration. Since 0.4.0.

Version

A version of a cell: from transaction time tx on, the cell’s value is that of the receipt receipt_hash, which states it as of asof.

Enums

Error

What a Store method or a migration can fail with.

Traits

Resolver

What step 7 of SPEC 10.1 obtains: the descriptor, the input receipts (slice members included), the manifests of slice entries, and the evaluation. This crate evaluates no functions and fetches nothing; the caller supplies a resolver, and verify asks it only for what step 7 reaches for the receipt at hand:

Store

A Datumesh store: a Resolver that can also be written to.

Type Aliases

Result

Result with the store’s Error.