Struct Database 

pub struct Database { /* private fields */ }

Primary database handle for Macrame bitemporal ledger.

Implementations

impl Database

pub async fn open(path: impl AsRef<Path>) -> Result<Self>

Open a database file at path, configuring pragmas, running migrations, and spawning the Write Actor.

The snapshot cadence runs with SnapshotCadence::default. Use Database::open_with_cadence to tune or disable it.

pub async fn open_with_cadence( path: impl AsRef<Path>, cadence: Option<SnapshotCadence>, ) -> Result<Self>

Open with an explicit snapshot cadence, or None to run without one (§5.5, D-053).

None restores the pre-0.5.5 behaviour, where close() is the only thing that ever writes an anchor. That is the right setting for a short-lived process that will not accumulate a delta worth bounding, and for tests that assert on the contents of the snapshot directory.

pub async fn open_with_clock( path: impl AsRef<Path>, cadence: Option<SnapshotCadence>, clock: Arc<dyn Clock>, ) -> Result<Self>

Open with an injected clock (§5.1.2, defect K, D-062).

The reason this exists is testing: recorded_at is the transaction-time axis, and until now every test that wanted to assert on one had to either avoid it or drive a raw connection, because open() hardcoded SystemClock. FakeClock has been public and constructed in the test harness since 0.5.2 with nothing to inject it into — the compiler warned about the dead field on every build for three releases.

The clock is floored against the database before the actor starts. Clock::raise_floor is called with the newest recorded_at in the ledger, so an injected clock cannot issue a stamp below what is already stored — which would abort the next concept write on trg_concepts_monotonic_ra rather than merely being odd. This is the step whose absence kept the defect open: the obvious implementation (take an Arc<dyn Clock>, use it) produces a Database that fails on its first write against any non-empty file.

On a fresh database there is no floor, so an injected FakeClock issues exactly the stamps it was given.

pub fn read_conn(&self) -> &Connection

Read connection handle for queries, traversals, and folds.

pub fn path(&self) -> &Path

The file this handle opened.

pub async fn diagnostic_conn(&self) -> Result<Connection>

A new, independently owned, OS-level read-only connection to this database, for diagnostics (§4.7, T5.1, D-091).

Why this exists when read_conn() already does

Two different things, and the difference is the point:

• read_conn() returns a shared &Connection carrying PRAGMA query_only = ON. That pragma is per-connection and reversible by its holder in one statement, so it is a guardrail against accident, not a capability boundary. And because the reference is shared, a caller who runs a long reporting query on it is competing with every traversal and fold in the process.
• This returns a connection opened with SQLITE_OPEN_READ_ONLY, which is enforced by the engine below the pragma layer, and it is the caller’s own.

Measured on libSQL 0.9.30 rather than assumed (examples/readonly_open_probe.rs), against a live WAL database with the write actor running:

	read_conn()	diagnostic_conn()
SELECT, EXPLAIN QUERY PLAN	allowed	allowed
INSERT	refused	refused
PRAGMA query_only = OFF	allowed	allowed
INSERT after that	allowed	refused

The third and fourth rows are the whole difference: turning the pragma off restores writes on read_conn() and does not here. That is what “boundary rather than guardrail” means, and it is now a number rather than a claim.

One way this is more permissive, which is worth knowing

CREATE TEMP TABLE succeeds here and is refused by read_conn(). Temp tables live in a separate temporary database that is writable regardless of how the main one was opened, whereas query_only refuses them outright — which is the mechanism [D-050] measured when it removed TwoPhaseTempTable for returning SQLITE_READONLY (8) on the read connection. So the stronger boundary is not uniformly stronger, and a strategy that needs a temp table has a connection it could run on. That is recorded, not acted on: D-050 removed the strategy for two reasons and this addresses one of them.

Errors

The file must already exist. SQLITE_OPEN_READ_ONLY drops SQLITE_OPEN_CREATE with it, so a missing file is SQLITE_CANTOPEN rather than a fresh empty database — which is the right failure, and is surfaced as a typed error rather than as libSQL’s error 14.

pub async fn verify_snapshot_chain(&self, ts: &str) -> Result<ChainCheck>

Cross-check the snapshot chain against a fold from genesis (§5.5, T5.3, D-092).

write_final composes onto the previous snapshot, so snapshot n is derived from snapshot n−1 and nothing in the chain ever folds the whole log. An error at any link propagates forward forever and every read agrees with it, because every read descends from it. This is the check that would notice.

When to run it

Not on a schedule this crate chooses. A genesis fold is precisely the cost snapshots exist to avoid, so running it periodically by default would give every application the bill snapshots were bought to remove — on a database whose log is large enough for snapshots to matter, which is the only kind where this is worth doing. The plan calls it a scheduling problem and it is the caller’s schedule: an idle period, a nightly job, or once per N anchors, chosen against a log size this crate cannot see.

The cadence is deliberately left alone for the same reason — it runs on a connection shared with nothing and a fold there would compete with interactive reads at a moment nobody chose.

It reports; it does not repair

A divergence means the snapshots are a wrong cache, not that the ledger is corrupt: Doctrine VI makes them disposable, so deleting Self::snapshots_dir restores correctness and costs only speed. Rewriting the file here would destroy the evidence that composition has a defect, which is the only thing this can tell you that you did not already know.

Pair it with the actor counters ([Self::metrics], D-079) so a divergence found by a scheduled run is visible beside the write latency of the period that produced it.

pub fn clock(&self) -> &Arc<dyn Clock>

The clock every write is stamped with (§5.1.1).

pub fn schema_version(&self) -> u32

Schema version this handle opened against.

pub fn archive_path(&self) -> &Path

Cold database path, derived by convention from the main file.

pub fn snapshots_dir(&self) -> &Path

Snapshot directory, derived by convention from the main file.

pub async fn assert_edge(&self, edge: EdgeAssertion) -> Result<()>

Assert an edge (Doctrine III: a new row, never an update).

pub async fn retire_edge( &self, source: impl Into<String>, target: impl Into<String>, edge_type: impl Into<String>, valid_from: &str, valid_to: &str, ) -> Result<()>

Close an open interval by asserting its replacement (Doctrine III).

pub async fn upsert_concept(&self, concept: ConceptUpsert) -> Result<()>

Insert or update a concept.

pub async fn write_bulk_atomic( &self, edges: Vec<EdgeAssertion>, ) -> Result<usize>

Assert many edges in one transaction under one stamp (D-014).

This is the one write with no latency bound, and here is what it costs

The batch is one act under one recorded_at, so it cannot be chunked — splitting it is the thing this method exists not to do. That makes the actor’s hold a function of edges.len(), and until now the only statement of that anywhere was the prose “uncapped” in CHUNK_BUDGET’s table. A caller who stalls every other writer for eight seconds should have been able to predict it from the signature.

Measured on libSQL 0.9.30 (T1.3, D-081), holding the actor for:

rows	hold
500	~34 ms
2,000	~155 ms
10,000	~1.0 s
20,000	~2.6 s

estimated_bulk_hold is that curve as a function, and this method emits a tracing::warn! when it predicts more than BULK_ATOMIC_WARN_HOLD. The estimate is a shape, not a promise — see estimated_bulk_hold for what it is calibrated against and where it will be wrong.

A caller who needs the latency bound and not the atomicity wants Self::bulk_import, which is the same write chunked and explicitly not atomic overall (D-011).

pub async fn rebuild_current(&self) -> Result<RebuildReport>

Rebuild links_current from links and verify zero drift (§5.8).

pub async fn rebuild_current_chunked(&self) -> Result<RebuildReport>

Rebuild links_current beside itself, in chunks (§5.8, T1.2, D-082).

Same result as Self::rebuild_current, different latency profile. rebuild_current is one transaction holding the write lock for its whole duration, because D-023 will not let the DELETE and the INSERT be split: a reader landing between them sees a graph with no edges and no error. This builds the replacement in a shadow table instead — the live table stays live and trigger-maintained throughout — and swaps it in at the end.

Each step is its own actor turn, so an interactive assertion can jump the queue between chunks. That is the whole of the improvement, and it is why the loop is here rather than inside the actor’s arm (the same reasoning as Self::archive_windowed and Self::bulk_import).

What the swap still costs

Not microseconds. Index names are global and SQLite has no ALTER INDEX … RENAME, so the shadow cannot be built carrying links_current’s index names while links_current still holds them — and building it under other names would leave the table permanently indexed under names absent from CREATE_INDICES, so the next migration would create a second copy of each. DROP TABLE frees the names, so the swap transaction is where the three indexes get built. What the chunking moves off the lock is the projection — the window function over all of links — which is the O(E log E) term.

When this returns an error rather than a repair

DbError::RebuildInterrupted means an archive committed while the shadow was being built. Its deletions are invisible to a catch-up pass keyed on recorded_at — a deleted row has no recorded_at left to find it by — so the work is discarded rather than swapped in. links_current is untouched and the call can simply be retried.

Use Self::rebuild_current when the repair must be one atomic act, or when nothing else is contending for the actor and the extra turns are pure overhead.

pub async fn shadow_step(&self, step: ShadowStep) -> Result<ShadowOutcome>

Run one step of a chunked rebuild, for a caller doing its own scheduling.

Self::rebuild_current_chunked is this in a loop and is what almost everyone wants. This exists because that loop offers no seam: it drives Begin, then Fill to exhaustion, then Swap, and a caller who needs to do something between steps — pace them against a frame budget, abandon a rebuild that has run long enough, or provoke the archive interlock in a test — cannot get in.

The obligation that comes with it: epoch from ShadowOutcome::Started must be handed back to ShadowStep::Swap, or the archive interlock is defeated and a stale projection can be swapped in. The looping version cannot get that wrong; this one can.

pub async fn bulk_import(&self, edges: Vec<EdgeAssertion>) -> Result<usize>

Import edges on the background channel, chunked (D-011).

Atomic per chunk, not overall: a failure partway leaves earlier chunks committed. That is the tradeoff chunk_rows documents — use Database::write_bulk_atomic when the batch must be all-or-nothing.

Chunked at chunk_rows::EDGES, which is also faster in total than the larger chunks this used through 0.5.5 (D-058).

pub async fn write_concepts( &self, concepts: Vec<ConceptUpsert>, ) -> Result<usize>

Upsert many concepts on the background channel, chunked (D-011).

This is the bulk concept path, and every row it writes is a ledger write: it versions the concept and lands in transaction_log. Derived analytics output does not belong here — see Database::write_analytics_annotations and D-041.

Called write_annotations through 0.5.6, from when the two writes were one call. D-041 split them and the name stayed on the wrong one for three releases, so the crate had a write_annotations that wrote concepts sitting beside a write_analytics_annotations that wrote annotations (D-075).

pub async fn reconstruct(&self, ts: &str) -> Result<MaterializedState>

State as believed at ts (§5.5, D-026, D-049).

A read: it runs on read_conn and never touches the Write Actor, so a reconstruction and a full-speed write-back do not slow each other.

Prefer this to calling crate::temporal::reconstruct directly. The free function takes the archive path and the snapshot directory as arguments, and a caller who passes None for the second gets a correct answer that folds the whole log every time — the composition is opt-in at that layer and easy to leave off by accident. Here both come from the handle, so the fast path is the default one.

pub async fn register_model(&self, model: &ModelName, dim: usize) -> Result<()>

Create a model’s embedding table and DiskANN index (§5.9, D-048).

Idempotent: registering a model that already exists at the same dimension succeeds, and at a different dimension fails with DbError::DimMismatch naming both, rather than no-opping through IF NOT EXISTS and leaving the caller believing the dimension they asked for is the one in force.

This issues DDL, which everywhere else in the crate is the migration runner’s exclusive business (D-032). The exception is bounded and deliberate: a model’s table is created once, by an explicit call, and the alternative — a caller-supplied write connection — is the very thing the Write Actor exists to make impossible.

Latency

One small transaction, but it queues like any other write: see §5.1.8.

pub async fn upsert_embeddings( &self, model: &ModelName, rows: Vec<(String, Vec<f32>)>, ) -> Result<usize>

Store or replace vectors for model, chunked (§5.9, D-011, D-048).

The write path for embeddings. Before 0.5.4 there was none: [crate::vector::upsert_embedding] takes a raw connection, read_conn is query_only, and the write connection lives inside the actor — so an application could search vectors it had no way to store.

Low priority and chunked at chunk_rows::EMBEDDINGS, because embedding is bulk derived work: a 50,000-vector backfill must yield to an interactive assertion at every chunk boundary. That constant is the smallest of the four by a wide margin — DiskANN index maintenance makes an embedding the most expensive row in the system (D-058). Atomic per chunk, not overall, which is the same trade Database::bulk_import makes and is safer here than there — an embedding is derived (Doctrine VII), so a partially written batch is recoverable by re-embedding.

Fails with DbError::ModelNotRegistered if model has no table, and DbError::DimMismatch if a vector’s length is not the declared dimension. The dimension is read from the schema once per chunk (D-037): the crate keeps no registry of its own to fall out of date.

pub async fn rebuild_fts(&self) -> Result<()>

Reconstruct the concept-text search index from the ledger (§5.9, D-036).

The FTS index is derivative: D-036 promises every derivative table can be rebuilt from the ledger tables, and this is that promise made callable for concepts_fts. Needed after a restore that skipped the shadow tables, or if the index is ever suspected of drifting from the text — and, as a matter of policy, cheaper to run than to reason about.

The work is INSERT INTO concepts_fts(concepts_fts) VALUES('rebuild'), which is FTS5’s own operation over the content table, so this is not a second implementation of the sync triggers that could disagree with them.

pub async fn write_analytics_annotations( &self, annotations: Vec<Annotation>, ) -> Result<usize>

Write derived analytics results on the background channel, chunked (§5.4, D-041).

Rows go to analytics_annotations, which has no log trigger, so nothing written here reaches transaction_log and nothing here versions a concept. Rerunning an algorithm replaces the previous pass rather than recording that the world changed.

Low priority and chunked at chunk_rows::ANNOTATIONS — the largest of the four, because this is the only bulk table carrying no triggers at all and its rows are correspondingly cheap (D-058) — so a 50,000-label Louvain save yields to interactive writes at every chunk boundary and carries the per-chunk fidelity boundary of §5.1.6 — a partially written pass is recoverable by rerunning, which is the property that makes derived state safe to write this way and assertions not.

pub async fn archive(&self, cutoff: &str) -> Result<ArchiveReport>

Move closed intervals and superseded log rows older than cutoff to the cold database (§5.7, D-012).

pub async fn archive_windowed( &self, cutoff: &str, window: Duration, ) -> Result<Vec<ArchiveReport>>

Archive up to cutoff as a sequence of sessions, each covering at most window of transaction time (T1.1, D-080).

archive(cutoff) is one transaction whose size is set by how long it has been since the last one, which makes it the least bounded of the three operations exempt from CHUNK_BUDGET — its hold is a function of operational history rather than of anything a caller chose. This runs the same work as N complete sessions, each with its own marker, horizon row and rebuild, and returns one ArchiveReport per session in order.

D-012 is satisfied per session, and that is what it requires

The atomicity D-012 demands is that copy-then-delete never be split — a crash between the phases duplicates or loses rows. N small sessions satisfy that exactly as one large one does. The obligation windowing adds is that a partial run leave a coherent intermediate state, which it does: each session commits a valid horizon, so a failure at window k leaves a database archived up to boundary k−1 and nothing in between. The sequence is not atomic and does not claim to be — on error, the reports for the sessions that did commit are lost with it, but their effect is not, and re-running with the same cutoff completes the job.

Each session is its own actor turn, and that is the entire point

This loop lives here, on the handle, rather than inside the actor’s Archive arm. Putting it there would have produced N small transactions inside one hold, which shrinks the transaction and changes the latency not at all: the actor is single-threaded, so nothing else writes until its turn returns regardless of how many COMMITs the turn contains. Sending N commands returns the actor to its select! between sessions, which is where an interactive assertion gets to jump the queue — and it is high-priority, so it does.

The same reasoning is why Self::bulk_import chunks here and not there, and it is the trap T1.2 names for CREATE TABLE … AS SELECT.

Choosing a window

The bound is on transaction time, so the session count is set by how far back the hot file goes, not by how much it holds. A window is rejected rather than clamped if it would need more than MAX_ARCHIVE_SESSIONS sessions — see DbError::ArchiveWindow.

Windows containing nothing archivable are cheap but not free: each still opens a transaction and writes a horizon row. What they no longer do is re-project links_current, which archive_session now skips when its DELETE removed no rows — without that, windowing costs more in total than not windowing, because the repair term scales with the surviving table and not with the batch (D-077).

pub async fn close(self) -> Result<()>

Clean shutdown: stop the Write Actor, then write the final snapshot (§5.1.7).

Order matters. The snapshot is taken after the actor has stopped and been joined, so no write can land between the fold and the file — the anchor it records is the last thing that happened, not the last thing that happened to be visible.

A failed snapshot is reported rather than swallowed. It is not a durability loss — the ledger is in the WAL and the log replays without it — but it means the next open starts from an older anchor, and a caller that never hears about it cannot know why startup got slower.

The cadence stops first (§5.5, D-053). Both it and write_final end by running retention over the snapshot directory, and retention deletes files. Letting them overlap would mean one pass enumerating the directory while the other removes from it — not a correctness problem for the ledger, which is why the ordering is stated rather than locked, but a source of spurious warnings and of a final anchor that could be deleted by a cleanup that started before it existed. Stopping the cadence, then the actor, then taking the snapshot leaves exactly one writer at each step.

impl Database

pub async fn load_subgraph( &self, start_node: &str, max_hops: u32, now_ts: &str, byte_budget: usize, ) -> Result<Subgraph>

Load the topology reachable from start_node within max_hops (§5.4).

Runs on the read connection, so it cannot contend with the write actor. byte_budget bounds the result: a hub node in a dense graph can reach most of the database in three hops, and the budget is what turns that into DbError::SubgraphTooLarge rather than into an allocation failure.

Unfiltered: every edge type, every weight. See Self::load_subgraph_with for the filtered form, which this delegates to.

min_weight is NEG_INFINITY rather than [TraversalBuilder]’s default of 0.0, and the difference is load-bearing. A floor of 0.0 silently drops negative-weight edges — which is precisely the input DbError::NegativeEdgeWeight exists to report, since Dijkstra and A* are unsound over them and D-039 chose to refuse at the boundary rather than return a shortest path that is merely a path. Delegating with the builder default turned that typed refusal into a graph quietly missing edges; a_negative_edge_weight_is_refused_at_load caught it.

So the two mechanisms are made to agree instead of overlapping: an edge a caller has not filtered out reaches the weight guard, and an edge they have is theirs to exclude. See Self::load_subgraph_with for what that means when a caller passes a default builder.

pub async fn load_subgraph_with( &self, traversal: &TraversalBuilder, now_ts: &str, byte_budget: usize, ) -> Result<Subgraph>

Load the topology a [TraversalBuilder] describes, as a Subgraph (§5.4, D-073).

load_subgraph took neither edge_types nor min_weight while TraversalBuilder took both — the same walk over the same table with two fewer knobs. That was a reachability limit rather than a convenience one: the byte budget bounds the unfiltered neighbourhood, so a caller wanting one edge type out of a hub got DbError::SubgraphTooLarge for a graph whose filtered form would have fitted easily, and filtering the returned Subgraph afterwards cannot help because the refusal happens during the walk.

The filters apply to the walk and to the returned edges

This is the decision the change turned on, and the two are separable. TraversalBuilder applies its filters to the recursive step — which edges are followed — while this loader’s final projection returns every edge of every node it reached. Wiring the two together naively gives a caller who asked for CITES a graph reached via CITES and populated with KNOWS edges as well, which is surprising enough to be read as a bug.

So both halves filter. If a caller names edge types or a minimum weight, they are asking for a subgraph of those edges: the walk uses them to bound which nodes are reached, and the projection uses them to decide which adjacency lands in the result. load_subgraph passes a default builder — no types, weight ≥ 0 — so its behaviour is unchanged.

min_weight and the negative-weight guard

[TraversalBuilder] defaults min_weight to 0.0, so a default builder passed here filters negative-weight edges out rather than letting them reach DbError::NegativeEdgeWeight. That is a real difference from Self::load_subgraph, which passes NEG_INFINITY.

It is deliberate and it is the coherent reading: a caller who states a weight floor has asked to exclude what falls below it, and excluding it is not an error. A caller who states none should be told, because Dijkstra and A* are unsound over negative weights. Pass .min_weight(f64::NEG_INFINITY) to get the guard with a filtered builder.

attribute_mode is ignored: hydration here is always the live concept row, which is what a Subgraph has always carried.

Trait Implementations

impl Drop for Database

Notes a missed close() at warn!, and deliberately does not assert.

§7.3 offered option B — document close() as mandatory and debug_assert in Drop — and Wave 4.2 implemented it, measured the consequence, and reduced it to a warning. The assert fired on roughly thirty tests on its first run. That is the signal it was built to produce, and the right reading of it was not “thirty tests are wrong”.

What dropping actually costs is one final snapshot. Nothing else: every public write method awaits its responder, so by the time a caller can drop the handle, every write it issued has already committed; and the cadence stops on its own, because cadence_stop is a watch::Sender whose drop signals the task. A snapshot is derivative state under Doctrine VI — disposable, reconstructible, and never the only copy of anything. Losing one makes the next reconstruct fold from an older anchor, which is slower, not wrong.

A debug_assert aborts a test run. Spending that on a performance loss, in a project whose own notes say a suite that fails for reasons unrelated to the code under test trains people to ignore red, is the wrong trade — and paying it in thirty places would have made close() look mandatory by ceremony rather than by consequence. close() remains the right thing to call, and the two reasons to call it are now stated where they can be acted on: the snapshot, and the writer’s Result, which only close() can return.

Option A (“abort the actor and log”) stays rejected, for the reason it was rejected twice before: Drop cannot await, so it cannot drain, and cleanup that cannot clean up is worse than none — it looks like cleanup.

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.