Struct Pager 

pub struct Pager<F: FileBackend = FileHandle> { /* private fields */ }

The pager.

Owns the storage backend (file or in-memory), a FileHeader snapshot of page 0, a Cache of main-file pages, and (for file-backed databases) a Wal sidecar plus two in-memory overlays: a pending-transaction buffer and a committed-but-not- checkpointed view. All public methods take &mut self; concurrent access is the WAL’s problem to grow into in M6.

Generic over F: FileBackend (Rule 9: hot-path dispatch is static monomorphisation, never dyn). The default is the production FileHandle; the fault-injection harness substitutes Pager<FaultyFileHandle> to drive recovery against torn writes, dropped fsyncs, and bit flips.

Implementations

impl Pager<FileHandle>

pub fn open<P: AsRef<Path>>(path: P, config: Config) -> Result<Self>

Open or create a database file at path. A new file is initialised with a default FileHeader and no allocated pages beyond page 0.

Cache capacity is taken from config. The cache is allocated before any read or write; subsequent operations never call the global allocator on the cache hot path (Rule 3).

At M3, opening a file-backed database also opens (or creates) the WAL sidecar at <path>-wal and replays any committed-but- not-checkpointed frames before any read can succeed. If no WAL exists, or the existing WAL belongs to a previous generation (salt mismatch), the database opens as if the WAL were empty.

Errors

• Error::InvalidArgument if config.cache_frames == 0.
• Error::Io if the file cannot be opened or initialised.
• Error::InvalidFormat if an existing main file does not look like an obj database, or if an existing WAL has a header that disagrees with the main file’s format.

pub fn memory(config: Config) -> Result<Self>

Construct a fresh in-memory pager. Cache capacity is taken from config; the backing store starts at one page (the header). The in-memory pager has no WAL — all writes go straight to the in-memory buffer.

Errors

Returns Error::InvalidArgument if config.cache_frames is zero.

impl<F: FileBackend> Pager<F>

pub fn open_with_backends( main: F, wal: F, wal_path: PathBuf, config: Config, ) -> Result<Self>

Open a file-backed pager on top of caller-supplied backends.

main is the database file; wal is the WAL sidecar at wal_path. Both must already be open and writable. The WAL is walked for recovery (#15 / #21) before any user-visible read can succeed.

Production callers SHOULD use Pager::open; the fault-injection harness uses this entry point to drop a FaultyFileHandle into the pager and a separate one into the WAL.

Errors

• Error::InvalidArgument if config.cache_frames == 0.
• Error::Io on any syscall failure.
• Error::InvalidFormat if the existing main file does not look like an obj database, or if the WAL header disagrees with the main file’s format.
• Error::WalCorruption if the WAL contains a CRC-invalid frame before its last commit marker (#21).

pub fn page_count(&self) -> u64

Total number of pages in the database, including page 0.

pub fn page_size(&self) -> u16

On-disk page size in bytes (4096 at format major 0). Surfaced for the M12 obj stat CLI surface; callers who want the compile-time constant should reach for crate::pager::page::PAGE_SIZE directly.

pub fn format_version(&self) -> (u16, u16)

(format_major, format_minor) from the on-disk header. Surfaced for the M12 obj stat CLI surface so a forensic tool can confirm the file’s format vintage without re-reading page 0.

pub fn freelist_head(&self) -> u64

The current freelist head (0 = empty). Useful for tests.

pub fn root_catalog(&self) -> u64

The catalog B-tree root page-id, or 0 if no catalog has yet been installed. The catalog (M5) uses this field to bootstrap on first open; older format_minor = 0 databases (M2..M4) always carry zero here.

pub fn set_root_catalog(&mut self, root: u64) -> Result<()>

Update the catalog B-tree root page-id and persist the change in the file header.

The catalog (M5 issue #38) calls this exactly once per open_or_init when it allocates a fresh empty catalog root, and on every catalog mutation that produces a new root via the B+tree’s copy-on-write contract.

As of M6 issue #51 the header update is WAL-staged on file-backed pagers: the call records the new value in the in-memory header (so subsequent reads from this writer see it) AND stages the encoded page-0 into the current WAL transaction. The on-disk header at offset 0 is NOT touched until checkpoint; reader snapshots therefore see the pre-commit value of root_catalog (whichever value the committed WAL view held at snapshot time). For in-memory pagers the call still writes the header into the in-memory backend buffer immediately (no WAL exists).

Errors

Returns Error::Io on syscall failure writing the header (in-memory backend only — the file-backed path no longer performs an immediate write).

pub fn alloc_page(&mut self) -> Result<PageId>

Allocate a new page. If the freelist is non-empty, recycles its head; otherwise appends a brand-new page to the file.

As of issue #22, alloc_page is transactional: the freelist-page mutation it performs is staged in the current WAL transaction (the freelist link page is written through the WAL just like a regular user page write). As of issue #64, the file-header update (freelist_head / page_count) also rides the WAL (via the same private stage_or_write_header pathway that M6.5 #51 installed for Pager::set_root_catalog) — a crash between the WAL frame durability and the header write can no longer leave the on-disk header pointing at a not-yet-durable freelist link page. Callers SHOULD call Pager::commit before relying on the allocation being durable; pending allocations are lost on Pager::open after a crash, exactly like uncommitted user writes.

Errors

• Error::Io on syscall failure when extending the file.
• Error::Corruption if the freelist head fails to decode (indicates a previously-written freelist page has been damaged).
• Error::InvalidArgument in the unrealistic case that page_count would overflow u64 or the resulting file size would overflow.

pub fn read_page(&mut self, id: PageId) -> Result<PageRef<'_>>

Read page id. Returns a borrow-shaped PageRef that references bytes resident in one of (a) the in-flight transaction buffer, (b) the committed-but-not-checkpointed WAL view, (c) the LRU cache, or (d) — on a cache miss — a freshly-inserted cache frame populated by a single pread.

Read priority: in-flight transaction buffer → committed (WAL) view → cache → main file. The first three are in-memory hash-map / cache lookups; the last is a pread.

Allocation contract (Rule 3)

read_page performs no heap allocation on cache hits or WAL-overlay hits. On a cache miss, a single pread is issued and the page is inserted into the cache; the returned PageRef then borrows that cache frame. The previous M2 signature returned Result<Page> and cost one Page clone per call regardless of hit/miss; the borrow API removes that per-call clone.

Lifetime contract

The returned PageRef<'_> borrows self. The borrow checker forbids any mutating call on the same pager (write_page, commit, checkpoint, alloc_page, free_page, close, flush) while a PageRef is alive. Callers that need an owned page across mutating calls can use PageRef::to_owned_page.

Errors

• Error::InvalidArgument if id is out of range.
• Error::Io if a cache-miss read from disk fails.
• Error::Corruption if the page trailer fails to verify on a cache-miss path.

pub fn write_page(&mut self, id: PageId, page: &Page) -> Result<()>

Write page back to id. For file-backed databases, the write is staged in the WAL transaction buffer; for in-memory databases, the write goes straight to the cache as in M2.

To make the write durable, call Pager::commit.

Errors

• Error::InvalidArgument if id is out of range.
• Error::Io if a dirty eviction triggered by this insert fails to write its predecessor to disk (memory pager only).

pub fn free_page(&mut self, id: PageId) -> Result<()>

Free a previously-allocated page, returning it to the freelist. id must refer to a currently-allocated page; freeing the same page twice is a caller bug.

As of issue #22, the freelist link page is staged in the current WAL transaction (file-backed pagers) rather than written directly to the main file. As of issue #64, the header freelist_head update also rides the WAL (via the same stage_or_write_header pathway that M6.5 #51 installed for Pager::set_root_catalog); a crash mid-txn no longer leaves the on-disk header pointing at a freelist link that is only durable in the WAL view. Call Pager::commit before relying on the free being durable.

Errors

• Error::InvalidArgument if id is out of range.
• Error::Io if the freelist record or header write fails.

pub fn commit(&mut self) -> Result<Lsn>

Commit the in-flight transaction. Writes every staged frame to the WAL with a single sync_data at the end (group commit). Returns the LSN of the last committed frame, or 0 if the transaction was empty.

If the WAL’s committed-frame count exceeds Config::checkpoint_threshold after the commit, the pager inlines a Pager::checkpoint call. Auto-checkpoint amortises recovery time across writers without surfacing as a separate API call to the caller.

For in-memory pagers (no WAL) this is a no-op returning 0.

Errors

Returns Error::Io on syscall failure.

pub fn close(self) -> Result<()>

Perform a final checkpoint and remove the WAL sidecar.

close does NOT auto-commit a pending transaction — write_page calls without a matching commit() are dropped silently, matching the “uncommitted writes are not durable” half of the design.md ACID contract. If you want the pending txn to land on disk, call commit() before close().

After close() returns, a fresh Pager::open on the same path observes a database with no WAL — the design.md “no sidecar files left behind after a clean shutdown” invariant.

For in-memory pagers close is a no-op (no WAL to remove).

Errors

Returns Error::Io on syscall failure.

pub fn flush(&mut self) -> Result<()>

Backward-compatible flush. At M3 this is commit() + checkpoint() + fsync(main) for file-backed databases. For the in-memory backend it preserves the M2 “drain dirty cache + fsync” semantics. Kept as a stable alias so M2 tests continue to work — new code SHOULD call Pager::commit + Pager::checkpoint / Pager::close directly.

Errors

Returns Error::Io on syscall failure.

pub fn checkpoint(&mut self) -> Result<()>

Roll every committed WAL frame forward into the main file.

Protocol (see docs/format.md § Salt rotation):

1. For every page-id in the WAL view, write the page (with its CRC32C trailer) into the main file.
2. sync_data(SyncMode::Full) on the main file. Only after this returns Ok are the main-file writes durable.
3. Rotate the WAL salt via Wal::reset_after_checkpoint and truncate the WAL to header-only with the new salt.
4. Stamp the new salt into the main file’s wal_salt header field and sync_data the main file again.

Idempotent: a second invocation on an empty view is a no-op.

Crash-recovery model: a crash before step 3 leaves the old WAL with the old salt; the next Pager::open recovers it (idempotent — re-applying writes the same bytes step 1 already wrote). A crash after step 3 but before step 4 leaves the WAL with the new salt and the main file with the old; the next open reads the OLD salt from the main header, fails to match, treats the WAL as empty, and proceeds — recovery loses no data because step 2 made the main file authoritative before the salt rotated.

In-memory pagers have no WAL; checkpoint is a no-op for them.

Errors

Returns Error::Io on syscall failure.

pub fn reader_snapshot(&mut self) -> Result<ReaderSnapshot<F>>

Open a new MVCC reader snapshot at the current WAL end-LSN.

The snapshot captures (1) the LSN of the most-recent committed frame in the WAL at the moment of the call, and (2) a clone of the pager’s in-memory committed view (WalState.view). Reads through the snapshot use the cloned view + the main file; pending writes from a concurrent WriteTxn and frames committed AFTER the snapshot was taken are invisible.

On in-memory pagers (no WAL) the snapshot captures pinned_lsn = 0 and an empty frozen view — every read falls through to the main backend.

The returned ReaderSnapshot registers a pin in the pager’s live-snapshots map and removes the pin on drop. Checkpoint consults snapshots.values().min() when deciding whether it is safe to reclaim WAL frames.

Power-of-ten Rule 9: no dyn. The snapshot is generic over F: FileBackend.

Errors

Returns Error::Io only via underlying syscalls; the in- memory portion of this call cannot fail.

pub fn header_snapshot(&self) -> HeaderSnapshot

Snapshot the pager’s in-memory header AND WAL committed view for txn-rollback purposes. Returned to the caller and passed back into Self::restore_header_snapshot on rollback.

The view is captured because Self::free_page removes per-page entries from the WAL view immediately (the page’s committed content becomes stale once the id is back on the freelist). Without snapshotting the view, a rolled-back txn that freed a page would leave readers no way to find the page’s committed content — it sits below state.view (now missing the entry) and below the on-disk main file (never checkpointed). Snapshot/restore closes that gap.

Header fields (root_catalog, freelist_head, page_count) are written direct to disk (not through the WAL) so a pure pending-buffer discard leaves the header inconsistent with the rolled-back page bodies. The snapshot/restore pair closes that gap for the M6 Db::transaction rollback path.

pub fn restore_header_snapshot(&mut self, snap: HeaderSnapshot) -> Result<()>

Restore the in-memory header AND WAL view from a previously-captured snapshot, then write the restored header to disk. Used by crate::txn::WriteTxn::rollback to undo direct header writes + view mutations that happened during the rolled-back txn.

Errors

Returns Error::Io on syscall failure when writing the restored header to disk.

pub fn rollback_pending_writes(&mut self)

Discard every page in the in-flight transaction buffer. Used by crate::txn::WriteTxn::rollback. Idempotent — calling on an in-memory pager or on a file pager with an empty pending buffer is a no-op.

M6 #51: also clears the header-dirty flag so a rolled-back set_root_catalog does not emit a stray page-0 frame on the next commit. The in-memory self.header.root_catalog restoration is the caller’s job — WriteTxn uses Self::header_snapshot + Self::restore_header_snapshot.

pub fn live_snapshot_count(&self) -> usize

Number of live reader snapshots. For diagnostics and tests.

pub fn min_pinned_lsn(&self) -> Option<Lsn>

Lowest LSN any live reader has pinned, or None if no snapshots are live.

pub fn is_memory_backed(&self) -> bool

true iff this pager has no WAL — i.e. it was constructed via Pager::memory. In-memory pagers have no MVCC surface; a ReaderSnapshot against one reads the live cache rather than the (absent) WAL frozen view. Public so callers in peer crates (e.g. M11 Db::backup_to) can dispatch on the in-memory case without reaching across the privacy boundary.

pub fn read_main_file_page_zero(&self, buf: &mut [u8; 4096]) -> Result<()>

Read the first PAGE_SIZE bytes from the main backend into buf, bypassing the cache and the WAL overlay. Used by crate::backup to capture page 0 (the file header) for inclusion in a backup. Errors with Error::BackupNotSupportedForMemoryPager on an in-memory pager (which has no on-disk file to read from).

Errors

Returns Error::Io on syscall failure or Error::BackupNotSupportedForMemoryPager when the pager has no file backend.

pub fn read_main_file_page(&self, id: PageId) -> Result<Page>

Read page id consulting ONLY the main backend (no WAL overlay, no cache). Used by ReaderSnapshot::read_page when the frozen view does not contain the page.

Internal to the snapshot path; the on-disk page’s CRC32C trailer is verified before the page is returned. Read page id consulting ONLY the main backend (no WAL overlay, no cache). Verifies the on-disk page trailer before returning the bytes.

Used by ReaderSnapshot::read_page when the frozen view does not contain the page, and by the crate::backup module to materialise the source’s main-file pages into a destination backup file.

Errors

• Error::InvalidArgument if id is out of range.
• Error::Io on syscall failure.
• Error::Corruption if the on-disk trailer fails to verify.

pub fn begin_txn(&mut self)

M6 #51: mark the start of a WAL transaction. Called by crate::txn::WriteTxn::begin. The pager tracks a depth counter so a future nested-txn API (M8+) can bump/decrement without breaking the Catalog’s debug-assert. For the in- memory pager this is a no-op (no WAL transactional surface).

pub fn end_txn(&mut self)

M6 #51: mark the end of a WAL transaction. Symmetric with Self::begin_txn; called by crate::txn::WriteTxn::commit and crate::txn::WriteTxn::rollback (and the implicit Drop rollback). Saturating-decrement so a stray end without a matching begin does not underflow.

pub fn in_txn(&self) -> bool

M6 #51: true if the pager is currently inside a WAL transaction (file-backed) or is an in-memory pager (no WAL transactional surface — every mutation is immediately visible). Catalog mutations debug-assert this at their entry points so the M5 direct-write bug class cannot regress.

Trait Implementations

impl<F: Debug + FileBackend> Debug for Pager<F>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.