Struct SyncCatalog 

pub struct SyncCatalog { /* private fields */ }

Methods from Deref<Target = Catalog>

pub fn apply_wal_records(&mut self, records: &[WalRecord]) -> Result<(), Error>

Apply an LSN-preserving WAL record stream without appending it to the local WAL. Sync callers must validate lineage and contiguity before calling this method.

Replication boundary: this is a storage adapter for powdb-sync, not a general mutation API. Callers must reject unsupported record classes, hold their own replica progress state, and pass only contiguous, transaction-complete ranges or chunks.

pub fn ensure_no_pending_wal_records(&self) -> Result<(), Error>

Sync callers use this before deciding an apply is a no-op. A replica with local WAL history is divergent until a higher layer explicitly repairs it.

pub fn checkpoint(&mut self) -> Result<(), Error>

Flush every dirty heap page and truncate the WAL. This is the “clean shutdown” point — after this returns, the on-disk heap files are fully consistent and the WAL is empty, so the next open will skip replay entirely.

Safe to call multiple times. Safe to call on a catalog that has performed zero mutations since the last checkpoint (in which case the flushes are no-ops and the truncate is a bounded syscall).

pub fn checkpoint_with_wal_archive<F>( &mut self, archive: F, ) -> Result<(), Error>

where F: FnMut(&Path, &[WalRecord]) -> Result<(), Error>,

Flush every dirty heap page, archive retained WAL records, then truncate the WAL. Sync-aware callers use this to make archive-before- truncate explicit without making storage depend on the sync crate.

Replication boundary: this hook is for retained-history publication. It should stay behind sync-aware lifecycle helpers rather than becoming an ordinary checkpoint surface for application code.

pub fn begin_transaction(&mut self) -> Result<(), Error>

Begin a connection/session-scoped explicit transaction.

pub fn commit_transaction(&mut self) -> Result<(), Error>

Commit the active explicit transaction by appending a durable boundary marker after its row records.

pub fn commit_autocommit(&mut self) -> Result<(), Error>

Commit any autocommit row mutations accumulated by the current statement. Pure reads/DDL have no pending tx ids and fall through to a cheap WAL flush/no-op.

pub fn sync_wal(&mut self) -> Result<(), Error>

Flush any buffered WAL records to disk. Called by the executor at the end of every mutating statement so the group-commit window is exactly one statement.

See Self::wal_log for the durability contract.

pub fn set_wal_sync_mode(&mut self, mode: WalSyncMode)

Set the WAL sync mode. Production code should leave this at the default (WalSyncMode::Full). Benchmarks set it to WalSyncMode::Off to compare apples-to-apples against :memory: SQLite (which has zero fsync cost).

Never call this with Off in production — a machine crash can lose any record written since the last sync_wal returned.

pub fn set_wal_sync_deferred(&mut self, defer: bool)

Defer Full-mode commit fsyncs (WAL group commit). While enabled, the commit paths register the WAL generation they need durable instead of fsyncing inline; the pending claim is retrieved with Self::take_wal_durability_ticket and the caller must wait on it before acknowledging the statement. This lets the fsync leave the engine’s exclusive-lock hold so overlapping committers can share one fsync. Normal/Off modes are unaffected.

pub fn take_wal_durability_ticket(&mut self) -> Option<WalDurabilityTicket>

Take the durability claim registered by deferred commit flushes since the last take, if any. See Self::set_wal_sync_deferred.

pub fn wal_fsync_count(&self) -> u64

Number of fsyncs issued against the WAL (test/metrics hook).

pub fn rollback_to_last_sync(&mut self) -> Result<(), Error>

Discard in-memory mutations made since the last sync_wal() and restore the catalog to its on-disk state. Used by ROLLBACK to undo an in-progress transaction’s changes.

This re-opens the catalog from the checkpoint file and replays only the durable (already flushed) WAL records. Any WAL records that were appended but not yet flushed are lost.

Critical: before replacing *self we must discard every dirty in-memory page across all heaps. Otherwise the old Catalog’s Drop impl calls checkpoint() which flushes those dirty pages to disk — and the freshly-opened replacement catalog would then read the flushed (uncommitted) rows back, defeating the entire rollback.

pub fn rollback_to_last_sync_with_wal_archive<F>( &mut self, archive: F, ) -> Result<(), Error>

where F: FnMut(&Path, &[WalRecord]) -> Result<(), Error>,

Roll back the active transaction, then reopen/replay any remaining WAL through an archive hook before recovery truncates it. Sync-aware callers use this when committed pre-transaction records must remain available to replicas after rollback.

pub fn data_dir(&self) -> &Path

Returns a reference to the data directory.

pub fn max_lsn(&self) -> u64

Highest page LSN across all tables (0 if nothing has been written). This is the durability high-water mark — the LSN a backup taken now corresponds to, and the value Catalog::open uses to restore next_lsn after a reopen/restore.

pub fn create_table(&mut self, schema: Schema) -> Result<(), Error>

pub fn create_table_with_defaults( &mut self, schema: Schema, defaults: Vec<Option<Value>>, ) -> Result<(), Error>

Create a table whose columns carry literal defaults. defaults is aligned to schema.columns by position (and may be shorter / empty for columns without a default).

pub fn create_table_full( &mut self, schema: Schema, defaults: Vec<Option<Value>>, auto_cols: Vec<bool>, ) -> Result<(), Error>

Create a table with per-column literal defaults and auto-increment flags. Both vecs are aligned to schema.columns by position (and may be empty). Defaults and auto flags are WAL-logged and persisted in the catalog so they survive a restart.

pub fn column_defaults(&self, table: &str) -> Option<&[Option<Value>]>

Per-column literal defaults for a table, aligned to its columns by position. None when the table is unknown; an empty slice when no column has a default.

pub fn auto_columns(&self, table: &str) -> Option<&[bool]>

Which columns of a table are auto, aligned to its columns by position. None when the table is unknown; an empty slice when none are auto.

pub fn assign_auto_columns(&mut self, table: &str, values: &mut [Value])

Fill any omitted (Empty) auto column in values from the table’s sequence and advance it. No-op when the table is unknown or has no auto columns.

pub fn table_slot(&self, name: &str) -> Option<usize>

Resolve a table name to its current slot index. DROP TABLE uses swap-remove, so prepared-query fast paths pair this value with Self::structure_generation before every slot-indexed access.

pub fn structure_generation(&self) -> u64

O(1) prepared-metadata validity token. It is process-local by design: prepared queries do not cross process boundaries, and a reopened or rollback-replaced Catalog must invalidate every cached slot/offset.

pub fn table_by_slot(&self, slot: usize) -> &Table

O(1) slot-indexed table access. Panics on an out-of-range slot — callers must have obtained the slot via table_slot().

pub fn table_by_slot_mut(&mut self, slot: usize) -> &mut Table

Mutable counterpart to Self::table_by_slot.

pub fn get_table(&self, name: &str) -> Option<&Table>

pub fn get_table_mut(&mut self, name: &str) -> Option<&mut Table>

pub fn table_has_overflow(&self, table: &str) -> bool

Whether table may hold v2 (spilled) rows (see Table::has_overflow_rows). Unknown table ⇒ false. The executor gates its v1-only raw-byte fast paths on this.

pub fn sweep(&mut self, table: &str) -> Result<usize, Error>

Mark-and-sweep one table’s overflow pages, returning the number of pages reclaimed (design 3.6, door D12). Reclaimed pages are logged as a single OverflowFree record so the reclamation is crash-safe, then returned to the free list for reuse. Intended to run under the table write lock.

pub fn sweep_all(&mut self) -> Result<usize, Error>

Sweep overflow pages across every table. Returns the total reclaimed.

pub fn insert( &mut self, table: &str, values: &Vec<Value>, ) -> Result<RowId, Error>

pub fn insert_by_slot( &mut self, slot: usize, values: &Vec<Value>, ) -> Result<RowId, Error>

WAL-logged insert addressed by table slot index instead of name. Backs the executor’s prepared-insert fast path, which resolves the slot at prepare time to skip the name→slot hash probe. Behaves exactly like Self::insert (logs the record with the real RowId, stamps the landing page’s LSN) — the prepared path previously called the raw Table::insert and bypassed the WAL entirely, silently losing every prepared insert on a crash.

pub fn get(&self, table: &str, rid: RowId) -> Option<Vec<Value>>

pub fn get_projected( &self, table: &str, rid: RowId, column_indices: &[usize], ) -> Result<Option<Vec<Value>>, Error>

pub fn delete(&mut self, table: &str, rid: RowId) -> Result<(), Error>

pub fn delete_many(&mut self, table: &str, rids: &[RowId]) -> Result<u64, Error>

Mission C Phase 12: bulk delete a list of rids, batching btree maintenance. See Table::delete_many for the full explanation and fall-through rules. Returns the number of rows removed.

pub fn scan_delete_matching<P>( &mut self, table: &str, pred: P, ) -> Result<u64, Error>

where P: FnMut(&[u8]) -> bool,

Single-pass scan-and-delete driven by a raw-bytes predicate. See Table::scan_delete_matching and HeapFile::scan_delete_matching for the fusion rationale.

Prefer Self::scan_delete_matching_logged from any caller that needs crash durability. This variant writes no WAL records, so a crash between the scan and the next checkpoint would lose the deletes. Kept here for internal paths (e.g. drop_table) where the whole heap is about to be removed anyway.

pub fn scan_delete_matching_logged<P>( &mut self, table: &str, pred: P, ) -> Result<u64, Error>

where P: FnMut(&[u8]) -> bool,

WAL-logged variant of Self::scan_delete_matching. Every matched row emits one WalRecordType::Delete record in the same single-pass scan (via the table’s _with_hook variant), so crash recovery sees every deletion. Used by the executor’s Delete(Filter(SeqScan)) and bare Delete(SeqScan) fast paths.

Performance cost vs the non-logged primitive is one per-row WAL append into the in-memory buffer plus one fsync at the end — the heap scan itself still runs as a single pass with one ensure_hot per page.

pub fn scan_patch_matching_logged<P, M>( &mut self, table: &str, pred: P, try_mutate: M, ) -> Result<(u64, Vec<RowId>), Error>

where P: FnMut(&[u8]) -> bool, M: FnMut(&mut [u8]) -> Option<u16>,

Single-pass fused scan + in-place patch with WAL logging. Evaluates pred on raw row bytes and applies try_mutate to each match on the same hot page — no second pass. Returns (patched_count, fallback_rids).

Perf sprint: update analogue of scan_delete_matching_logged. Eliminates the two-pass collect-then-patch pattern.

pub fn update( &mut self, table: &str, rid: RowId, values: &Vec<Value>, ) -> Result<RowId, Error>

pub fn update_hinted( &mut self, table: &str, rid: RowId, values: &Vec<Value>, changed_col_indices: Option<&[usize]>, ) -> Result<RowId, Error>

Mission C Phase 2: update with a hint about which columns actually changed. Lets Table::update_hinted skip the old-row read when the hint shows no indexed column is in the changed set.

pub fn with_row_bytes_mut<F>( &mut self, table: &str, rid: RowId, f: F, ) -> Result<bool, Error>

where F: FnOnce(&mut [u8]),

Mission C Phase 4: fast-path update that patches a row’s raw bytes in place, skipping decode/encode. Caller guarantees the mutation preserves the row length and touches no indexed column. Returns Ok(true) if the patch landed, Ok(false) if the row is gone.

This primitive does NOT log to the WAL. Executor callers must route through Self::update_row_bytes_logged (or Self::update_row_bytes_logged_by_slot) so crash recovery sees the patched bytes. This raw form is retained for replay itself and any future callers that can tolerate the non-durable contract.

pub fn update_row_bytes_logged<F>( &mut self, table: &str, rid: RowId, f: F, ) -> Result<bool, Error>

where F: FnOnce(&mut [u8]),

WAL-logged variant of Self::with_row_bytes_mut. Applies f to the live row bytes on the hot page, then reads the mutated bytes back and emits a WalRecordType::Update record so replay will re-apply the same patch after a crash.

Ordering: the hot-page mutation happens first (in-memory only, no disk I/O), then the WAL record is appended and flushed. A crash after the mutation but before the WAL flush loses the update, but the caller never saw success in that case, so the contract holds: any Ok(true) return is durable.

No hot-page eviction can happen between steps because this method holds the catalog’s &mut self exclusively.

pub fn update_row_bytes_logged_by_slot<F>( &mut self, slot: usize, rid: RowId, f: F, ) -> Result<bool, Error>

where F: FnOnce(&mut [u8]),

Slot-indexed counterpart to Self::update_row_bytes_logged. Used by prepared-query fast paths that already cached the table slot at prepare time and want to skip the name->slot probe on every execution.

pub fn patch_var_col_in_place( &mut self, table: &str, rid: RowId, col_idx: usize, new_value: Option<&[u8]>, ) -> Result<bool, Error>

Mission C Phase 10: var-column in-place update fast path. Patches a single variable-length column’s bytes directly into the row’s slot, shrinking the row if the new value is smaller. Returns Ok(false) if the new value would grow the row (caller must fall back to the full encode path) or the row is gone.

Caller guarantees no indexed column is touched — indexes are NOT maintained by this primitive.

Not WAL-logged. Executor callers should use Self::patch_var_col_logged instead.

pub fn patch_var_col_logged( &mut self, table: &str, rid: RowId, col_idx: usize, new_value: Option<&[u8]>, ) -> Result<bool, Error>

WAL-logged variant of Self::patch_var_col_in_place. Runs the in-place shrink on the hot page, then reads the mutated row bytes back and logs a WalRecordType::Update record. On a false return (grow-case bail) nothing is logged — the caller’s fall-through to update_hinted handles the WAL itself.

pub fn scan( &self, table: &str, ) -> Result<impl Iterator<Item = (RowId, Vec<Value>)>, Error>

pub fn for_each_row_raw<F>(&self, table: &str, f: F) -> Result<(), Error>

where F: FnMut(RowId, &[u8]),

Zero-copy scan: passes raw row bytes to the callback without any per-row allocation. Used by the executor’s fast paths.

pub fn try_for_each_row_raw<F>(&self, table: &str, f: F) -> Result<(), Error>

where F: FnMut(RowId, &[u8]) -> ControlFlow<()>,

Zero-copy scan with early termination. The callback returns ControlFlow::Break(()) to stop. Used by Limit fast paths so a limit 100 query doesn’t pay decode/predicate cost for every row in the table after the limit is reached.

pub fn create_index(&mut self, table: &str, column: &str) -> Result<(), Error>

pub fn create_index_unique( &mut self, table: &str, column: &str, unique: bool, ) -> Result<(), Error>

Create an index with an explicit uniqueness flag. unique = true for primary-key-like columns where duplicate values should overwrite. unique = false for secondary indexes that allow duplicate column values (the default via create_index).

pub fn active_catalog_version(&self) -> u16

pub fn next_index_id(&self) -> u64

pub fn index_metadata(&self, table: &str) -> Option<Vec<IndexMetadata>>

Return both legacy column-index and v6 expression-index identities.

pub fn expression_index_metadata( &self, table: &str, ) -> Option<Vec<ExpressionIndexMeta>>

pub fn expression_index_btree( &self, table: &str, index_id: u64, ) -> Option<&BTree>

pub fn expression_index_btree_mut( &mut self, table: &str, index_id: u64, ) -> Option<&mut BTree>

pub fn expression_index_lookup_all( &self, table: &str, index_id: u64, key: &Value, ) -> Result<Vec<RowId>, Error>

pub fn expression_index_range_rids( &self, table: &str, index_id: u64, start: Option<&Value>, end: Option<&Value>, ) -> Result<Vec<RowId>, Error>

pub fn expression_index_ordered_rids( &self, table: &str, index_id: u64, ) -> Result<Vec<RowId>, Error>

pub fn expression_index_ordered_rids_bounded( &self, table: &str, index_id: u64, direction: IndexOrderDirection, offset: usize, limit: usize, ) -> Result<Vec<RowId>, Error>

pub fn drop_expression_index( &mut self, table: &str, index_id: u64, ) -> Result<(), Error>

pub fn create_expression_index_metadata( &mut self, table: &str, canonical_version: u16, canonical_text: impl Into<String>, json_path: StoredJsonPathV1, unique: bool, ) -> Result<u64, Error>

Persist expression-index identity and create its backup-compatible .eidx file. The catalog stays at v5 until every validation and file creation step succeeds; the v6 catalog rename is the activation point.

pub fn is_index_unique(&self, table: &str, column: &str) -> Option<bool>

Whether table.column has a UNIQUE index. Returns Some(true) for a unique index, Some(false) for a non-unique index, and None when the column is not indexed or the table is unknown.

pub fn has_index(&self, table: &str, column: &str) -> bool

Whether table.column has any index (unique or non-unique).

pub fn index_lookup( &self, table: &str, column: &str, key: &Value, ) -> Result<Option<Vec<Value>>, Error>

pub fn list_tables(&self) -> Vec<&str>

pub fn schema(&self, table: &str) -> Option<&Schema>

pub fn drop_table(&mut self, name: &str) -> Result<(), Error>

Drop a table: remove from the catalog and delete its data files. Returns Err if the table doesn’t exist.

pub fn alter_table_add_column( &mut self, table: &str, col: ColumnDef, ) -> Result<(), Error>

Add a column to an existing table’s schema and backfill all existing rows to match the new shape.

Older versions of this method only mutated the in-memory schema and relied on a (false) claim that “the heap format already handles short rows gracefully”. It doesn’t: decode_row reads exactly n_var + 1 variable-column offsets from the row bytes using the CURRENT schema. Any row encoded with the old schema’s (smaller) offset table would walk off the end of its buffer and panic with “range end index X out of range for slice of length Y” — which is exactly what a bare Type scan triggered right after an ALTER ADD COLUMN.

The fix: rewrite every existing row through Table::rewrite_rows_for_schema_change so the on-disk encoding matches the new schema layout. Existing rows get Value::Empty for the new column.

If the new column is required we refuse to add it to a non-empty table — there is no default value to backfill with, and silently storing Empty in a required slot would just shift the invariant violation to the next query.

pub fn alter_table_drop_column( &mut self, table: &str, col_name: &str, ) -> Result<(), Error>

Remove a column from an existing table’s schema and rewrite every live row to match the new shape.

Older versions of this method only mutated the in-memory schema and claimed that “reads simply won’t decode the dropped column”. That was wrong in several ways:

1. The null bitmap is indexed by column position. Dropping a column shifts every later column’s bit left, but old rows still have bits in the original positions — so is_null checks silently lie for every column after the dropped one.
2. The bitmap’s byte width (ceil(n_cols/8)) can shrink when n_cols crosses an 8-boundary, shifting every subsequent byte of the row against the decoder’s cursor.
3. Fixed-region size and the variable-offset-table width both depend on the column set, so dropping any fixed or variable column slides every following byte.

The fix mirrors alter_table_add_column: snapshot the old schema, mutate to the new schema, then rewrite every row through Table::rewrite_rows_for_schema_change. Dropping a column from an empty table skips the rewrite.

Trait Implementations

impl Deref for SyncCatalog

type Target = Catalog

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl DerefMut for SyncCatalog

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

impl Drop for SyncCatalog

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.