Struct Database 

pub struct Database { /* private fields */ }

A read-only view over the raw bytes of a SQLite database file.

Holds the whole file in memory — adequate for the spike and for browser evidence DBs (tens of MB). A Read + Seek / mmap backend is a later refinement and does not change the parsing logic proven here.

Implementations

impl Database

pub fn open(bytes: Vec<u8>) -> Result<Self, Error>

Parse the file header and validate magic + page size. No WAL overlay.

pub fn open_path<P: AsRef<Path>>(path: P) -> Result<Self, Error>

Open a database from a filesystem path with a bounded-memory paged read (roadmap §3.1): pages are streamed on demand through a small LRU cache instead of loading the whole file into a Vec<u8>, so a multi-GB database opens without proportional RAM. Main file only — for the WAL-applied view use Database::open_with_wal (WAL sidecars are small and stay in memory).

Read-only and panic-free: an unreadable file or a malformed header is a typed Error (Error::Io carries the std::io::ErrorKind); nothing is written back.

pub fn open_with_wal(bytes: Vec<u8>, wal: &[u8]) -> Result<Self, Error>

Parse the main database plus a -wal sidecar, overlaying the newest committed page versions from the WAL on top of the main file.

This is the forensic-safe alternative to libsqlite checkpointing: neither file is mutated. The resulting Database answers read_table with the WAL-applied view (use Database::open for the main-only view). Frames past the last commit frame, or whose salt does not match the WAL header, are ignored — they are uncommitted / superseded and not part of the consistent snapshot.

pub fn rollback_prior(&self, journal: &[u8]) -> Result<PriorSnapshot, Error>

Materialize the single pre-transaction state from a rollback -journal, binding it to THIS database (design §5). The journal’s page images (the bytes BEFORE the last transaction) are overlaid on the live pages, yielding a PriorSnapshot — a DISTINCT read-only view, never a Database, so a prior/deleted row can never be read as “live” (secure-by-design).

The main db’s page size is authoritative (a PERSIST journal has a zeroed header). Errors with Error::JournalModeConflict when self was opened WAL-applied (Database::open_with_wal): WAL and rollback-journal modes are mutually exclusive timelines and must not be overlaid.

Robust and panic-free: a malformed/truncated journal yields a prior snapshot with fewer overlaid pages (degrading toward the live image), never a panic; a non-power-of-two page size is a typed Error::BadJournalPageSize.

pub fn wal_applied(&self) -> bool

Whether a non-empty WAL overlay is in effect (at least one committed frame was applied on top of the main file).

pub fn wal_frame_pages(&self) -> &[WalFramePage]

Every committed -wal frame’s page image, in file order, with provenance.

Empty when the database was opened without a WAL (or the WAL held no committed frames). The carver scans these page images for deleted-cell residue that lives ONLY in the uncheckpointed WAL — the genuinely-different records the on-disk pages do not hold — tagging each with the (salt1, salt2, frame_index) log-sequence identity.

pub fn wal_timeline(&self) -> Option<WalTimeline>

Build the bespoke, format-exact WalTimeline for this database’s -wal sidecar, if one was supplied to Database::open_with_wal.

Returns None when the database was opened without a WAL, or the WAL held no committed frame (no materializable state). The timeline enumerates the segment’s CommitSnapshots — the only materializable database states — each addressable by CommitId; see WalTimeline.

This consults the original -wal bytes retained at open time, re-parsing them into the richer temporal model (the on-open WalOverlay keeps only the consistent-view pages; the timeline keeps every segment, snapshot, and residue tail). A page-size mismatch or malformed header surfaces as None here — use Database::wal_timeline_from when you need the typed WalValidationError.

pub fn wal_timeline_from( bytes: &[u8], wal: &[u8], ) -> Result<WalTimeline, WalValidationError>

Parse a main database + -wal sidecar directly into a WalTimeline, surfacing the typed WalValidationError when the WAL is malformed.

This is the validation-tier entry point: a page-size mismatch between the DB header and the WAL header is a HARD STOP (WalValidationError::PageSizeMismatch), not a silently mis-sliced overlay; a bad magic / unparsable header is WalValidationError::BadMagic. Both are caught at the physical-validation tier before any replay.

pub fn header(&self) -> Header

pub fn page_count(&self) -> u32

Number of pages in the database file.

Prefers the in-header DB size (offset 28) when it is a valid, non-zero value that is consistent with the file length; otherwise falls back to file_len / page_size. A mismatch between the two is itself a forensic signal (see Database::header_page_count / Database::file_page_count).

pub fn header_page_count(&self) -> u32

The page count recorded in the file header (offset 28). May be 0 (legacy “size not valid” sentinel) or disagree with the file length after an out-of-band truncation/extension.

pub fn file_page_count(&self) -> u32

The page count implied by the raw file length (file_len / page_size).

pub fn freelist_count(&self) -> u32

The freelist page count recorded in the file header (offset 36).

pub fn freelist_pages(&self) -> Result<Vec<u32>, Error>

Walk the freelist trunk/leaf chain and return every free (unallocated) page number, in trunk order. Free pages retain the bytes of whatever they last held — on a secure_delete=OFF database that includes deleted records, which the analyzer can carve.

Bounded against crafted cyclic trunk chains: a page already visited, an out-of-range page, or a leaf-pointer count larger than a trunk page can hold aborts with Error::MalformedFreelist rather than looping.

pub fn freelist_pages_split( &self, ) -> Result<(BTreeSet<u32>, BTreeSet<u32>), Error>

Walk the freelist and return its leaf and trunk page numbers separately (task #73). The distinction is load-bearing for chain-aware overflow recovery: a freed page that became a freelist leaf keeps its former content byte-for-byte, while a trunk page has its head (next-trunk pointer + leaf count + leaf-number array) written over the former content (file-format §“The Freelist”). Only leaves are content-preserving, so Database::read_freed_overflow_chain accepts a chain page only when it is a leaf.

Bounded identically to Database::freelist_pages: a cyclic trunk chain, an out-of-range page, or an over-large leaf count aborts with Error::MalformedFreelist rather than looping.

pub fn read_freed_overflow_chain( &self, first: u32, remaining: usize, usable: usize, freed_leaves: &BTreeSet<u32>, ) -> Result<(Vec<u8>, Vec<u32>), ChainBreak>

Follow a freed overflow-page chain starting at first, reading raw main-file pages only (carving wants on-disk residue, not the WAL view), and assemble up to remaining content bytes (task #73). The carve-side dual of Database::read_overflow_chain, with one extra discipline that makes it the 0-FP-relevant guard: every chain page must be a freelist leaf (freed_leaves). A page that is not a leaf is live, a trunk, or unreachable — following its pointer would risk reading reused or clobbered content, so it is a ChainBreak (Codex ruling #2: the leaf requirement, not the UTF-8 gate, is what rejects a destroyed chain).

Returns the assembled content and the ordered list of chain pages on success. Robustness (Paranoid Gatekeeper, design §4.2): the anti-bomb cap rejects upfront any remaining above what the freelist leaves can deliver ((usable - 4) × freed_leaves.len()), so an attacker-declared huge payload dies before any allocation; cycles are caught by a visited set; a premature next == 0 with bytes still wanted, an out-of-range page, or page 0 mid-chain all break. Never panics — every read is bounds-checked.

pub fn raw_page(&self, page: u32) -> Option<PageBytes<'_>>

Raw bytes of the 1-based page from the main file only, ignoring any WAL overlay. Carving wants the on-disk page (where deleted residue lives), not the WAL-applied view. Returns None for page 0 or out-of-range pages.

pub fn carve_cells( &self, page_bytes: &[u8], column_count: usize, ) -> Vec<CarvedCell>

Scan a slice of page bytes for record-shaped table-leaf cells of exactly column_count columns, recovering each as a CarvedCell.

This is the carving primitive the forensic analyzer drives over free / unallocated regions: at every byte offset it speculatively parses a payload_len varint, a rowid varint, and a record header, accepting the candidate only when the serial-type count matches column_count, the declared lengths stay within the slice, and every value decodes. Strict validation keeps the false-positive rate low; confidence reflects how strongly the bytes are record-shaped. Bounded: each offset does O(record) work and the scan is linear in the slice length.

pub fn carve_cells_inferred(&self, page_bytes: &[u8]) -> Vec<CarvedCell>

Carve record-shaped cells from a page slice inferring each record’s column count from its own serial-type array, instead of requiring a fixed count. This is what makes dropped-table / schema-gone recovery possible: the page’s table was DROPped, so sqlite_master no longer records a column count, but each record still self-describes its columns.

Inferring the count removes one validity check, so the remaining self-consistency checks are kept strict to hold the false-positive rate down: header_len + body_len == payload_len, every serial type legal, rowid > 0, the payload fully in-bounds, and at least MIN_INFERRED_COLUMNS columns. Records carved this way are graded a notch lower in confidence than fixed-count carving.

pub fn carve_leaf_cells(&self, page_bytes: &[u8]) -> Vec<CarvedCell>

Decode every cell present in a table-leaf page image (type 0x0D) by walking its cell-pointer array, inferring each record’s column count from its own serial-type array. Unlike Database::carve_free_regions (which scans only free space and excludes live cells), this returns the cells the page itself records as allocated.

This is the primitive WAL-frame recovery needs: a -wal frame is a full page snapshot at one point in time, so a cell that is allocated in an EARLIER frame’s image but absent from the final WAL-applied view is a row that was deleted later and survives ONLY in that superseded frame. The caller filters the returned cells against the final live view to isolate exactly those genuinely-deleted rows (so a still-live row is never re-surfaced — the filter is the caller’s responsibility, mirroring the freeblock-reconstruction discipline).

Bounded and panic-free: a malformed cell pointer or record simply yields fewer cells. Non-leaf pages yield nothing.

pub fn carve_free_regions( &self, page_bytes: &[u8], column_count_hint: usize, ) -> Vec<CarvedCell>

Carve deleted records from the free (unallocated) regions of an allocated table-leaf page (type 0x0D), never re-surfacing a live cell.

On an allocated leaf, deleted-cell residue survives in two places: the unallocated gap between the cell-pointer array and the cell-content area, and the slack between/after live cells (a former freeblock whose chain pointer may already be gone). This method computes the exact byte ranges occupied by live cells and carves only the complement — so a live (allocated) cell can never be returned as a deleted record. That is the 0-false-positive guarantee, enforced structurally rather than by a filter.

page_bytes is one whole page. column_count_hint, when non-zero, is the table’s known column count (matched exactly); pass 0 to infer the count per record (for a page whose schema is gone). Non-leaf pages yield nothing.

pub fn carve_overflow_records( &self, page_bytes: &[u8], ) -> Vec<(CarvedCell, Vec<u32>)>

Recover spilled deleted records on a table-leaf page whose payload continued onto a freed overflow-page chain (task #73). Scans the page’s free regions (the complement of the live cells — same discipline as Database::carve_free_regions, so a live cell is never re-surfaced) for a SpilledCell, then resolves each chain through freelist leaf pages only and assembles the full payload.

A resolved record is returned only when ALL hold (design §5):

1. the chain is intact through freelist leaves (Codex ruling #2: the leaf requirement is the load-bearing 0-FP guard — a trunk/live/off-freelist chain page is rejected);
2. the assembled bytes total exactly the declared P and decode cleanly;
3. strict UTF-8 on chain-resident TEXT — an EXTRA reject signal, not a correctness proof (Codex ruling #2: a clobbered chain can still be valid UTF-8, so this cannot prove integrity; it only catches the cases where the lossy decoder would otherwise mask an overwrite as U+FFFD).

Each returned tuple is (cell, chain) where chain is the ordered list of overflow pages the bytes came from (for provenance). Confidence is graded BELOW the in-page full-row tier (Codex ruling #1: overflow Tier-1 is a graded recovery, NOT part of the structural 0-FP guarantee — a freelist leaf can be stale, holding unrelated bytes that happen to decode). Bounded and panic-free; a malformed page or chain simply yields fewer records.

pub fn carve_overflow_template_records( &self, page_bytes: &[u8], ) -> Vec<(CarvedCell, Vec<u32>)>

Reconstruct freeblock-clobbered spilled cells (task #73, design §2.2 / Codex ruling #5). When a freed cell whose payload spilled is also freeblock-clobbered, its declared P is destroyed but re-derivable from the surviving structure: P = header_len + Σ serial_body_len over the full (template + surviving) serial array. When that P exceeds usable - 35 the record is spilled by construction, so we read the 4-byte first-overflow pointer that follows the local payload and resolve the chain through freelist leaves, exactly as the intact-prefix path does — but with rowid = 0 (the prefix’s rowid varint was clobbered, never invented).

UNPROVEN-BY-CORPUS (Codex ruling #5): no real Nemetz 0E cell is both freeblock-clobbered and spilled — every measured spilled cell kept an intact prefix in the unallocated gap. This path is therefore validated against a synthetic fixture only; it is the general solution the no-special-case rule requires (it applies the same spill formula to the clobbered class), but its real-data behavior is not yet observed.

Returns (cell, chain) per fully-resolved record. Bounded and panic-free.

pub fn carve_overflow_fragments(&self, page_bytes: &[u8]) -> Vec<CellFragment>

Tier-2 salvage for spilled cells whose overflow chain is broken (task #73, Codex ruling #4): when Database::carve_overflow_records rejects a recognized spilled cell because its chain failed (a trunk-clobbered or reused chain page), the cell’s intact LOCAL prefix still holds the columns whose bodies fit entirely on the leaf page. Those are salvaged as a CellFragment — the same Tier-2 surface freeblock reconstruction uses.

Only columns whose body lies wholly within the local payload are kept; the chain-resident columns are lost (untrusted by definition — the chain that would supply them is the thing that failed). A fragment is emitted only when the salvaged prefix carries ≥ 1 distinctive cell (TEXT ≥ 4 bytes of valid UTF-8, or REAL — the §3.1 gate), so a lone integer prefix never anchors one. Bounded and panic-free.

pub fn reconstruct_freeblock_records( &self, page_bytes: &[u8], ) -> Vec<CarvedCell>

Reconstruct deleted records from the freeblock chain of an allocated table-leaf page (type 0x0d) — the records a forward parse cannot recover because their first four bytes were destroyed by freeblock conversion.

When SQLite frees an in-page cell it converts it into a freeblock (file-format §1.6): the cell’s first two bytes become the next-freeblock offset and the next two the freeblock size, overwriting the cell’s payload-length + rowid varints, the record header_len varint, and the leading serial type(s). The record’s surviving serial-type tail and its whole value body remain intact after those four bytes.

This method rebuilds each freed cell from that surviving tail plus a schema template derived from a LIVE cell on the same page (the table’s column count, header length, and the serial types of the leading columns that fall inside the clobbered prefix). The destroyed rowid is surfaced as unknown (0) — never invented — and the record is graded LOW.

Precision discipline (task #56): a candidate is emitted only when its body decodes cleanly with every serial type legal AND the record fits within the freeblock’s [offset, offset + size) bounds. Implausible or out-of-bounds candidates are rejected, so reconstruction does not manufacture phantom rows. (The forensic layer additionally drops any reconstruction whose values match a live row, so a live row is never re-surfaced.)

Bounded and panic-free: every freeblock pointer, size, and serial length is range-checked against the page before use, and the chain walk is capped at MAX_FREEBLOCKS_PER_PAGE to defeat a crafted cyclic next chain. Non-leaf pages, pages with no freeblock chain, and pages with no usable schema template yield an empty result.

pub fn reconstruct_freeblock_fragments( &self, page_bytes: &[u8], ) -> Vec<CellFragment>

Tier-2 partial salvage: the CellFragments abandoned by Database::reconstruct_freeblock_records on this page.

At every anchor where full reconstruction failed — an illegal serial in the surviving tail, a tail that overruns the span, or a body that does not fit — the columns that DID decode cleanly before the failure are salvaged as the maximal decodable prefix. A fragment is emitted only when that prefix contains at least one distinctive cell (TEXT ≥ 4 bytes of valid UTF-8, or REAL): a lone surviving integer pattern is coincidence-prone and never anchors a fragment.

Mutually exclusive with the full reconstructions of Database::reconstruct_freeblock_records by construction: an anchor yields a cell or a fragment, never both. Inherits the same anchor discipline — no sliding scan, no strings-style hunt — so Tier-2 carries Tier-1’s precision architecture. Bounded and panic-free identically.

pub fn index_leaf_cells(&self, page_bytes: &[u8]) -> Vec<Vec<Value>>

Parse the LIVE cells of an index-b-tree leaf page (type 0x0a) into their decoded key records (roadmap §1.4 foundation). A regular index on a rowid table stores each entry as (indexed columns…, rowid); a WITHOUT ROWID table stores its whole row here (the row IS the key). This is the structural read every later index-carve / WITHOUT ROWID recovery builds on — the second substrate for a table’s data, where key columns survive even when the table-leaf residue is gone.

Reads live cells only (via the cell-pointer array); returns empty for any non-index-leaf page, so a table page is never mis-read. Bounded and panic-free — every read is bounds-checked; a cell whose payload does not decode is skipped rather than panicking.

SCOPE (foundation): decodes the LOCAL payload only. An index key large enough to spill onto an overflow-page chain is decoded up to its on-page bytes (the leading key columns still resolve); full overflow following, and carving DELETED index entries from index-page freeblocks, are follow-ups.

pub fn without_rowid_table_rows(&self) -> Vec<WithoutRowidTable>

The live rows of every WITHOUT ROWID user table (roadmap §1.4).

A WITHOUT ROWID table stores its whole row in an index b-tree — there is no separate table b-tree and no rowid — so the ordinary read_table reader (which walks table pages 0x0d/0x05) is blind to it. This resolves each such table from sqlite_master, walks its index b-tree (interior 0x02 → leaf 0x0a), and returns its live rows, keyed by table name. Ordinary rowid tables are not returned.

Bounded and panic-free: a malformed/cyclic b-tree stops the walk (visited set + page cap) rather than looping; an unreadable schema yields an empty result. Rows are the decoded index records, in the table’s column order.

pub fn has_user_table(&self) -> bool

Whether sqlite_master (the schema table rooted at page 1) lists at least one user table — i.e. a type='table' row whose name is not an internal sqlite_* table. A database where every table was DROPped (or that never had one) returns false; the forensic carver uses this to label freed content as dropped-table residue. Errors (unreadable schema) are treated as “no user table” so the carver degrades safely.

pub fn live_rowids(&self) -> BTreeSet<i64>

Collect the rowids of every currently-live row across all user table b-trees (the roots listed in sqlite_master). The forensic carver uses this to drop any carved “deleted” record whose rowid is in fact still live — a stale copy of a live row can linger in free space after a b-tree rebalance moved the row to another page, and reporting it as deleted would be a false positive. Rowid collection ignores the column count (the rowid is in the cell prefix), so it works even when a schema row is malformed.

Bounded and panic-free: unreadable schema or a malformed b-tree yields a partial (possibly empty) set rather than an error.

pub fn live_rows(&self) -> BTreeMap<i64, Vec<Value>>

Collect every currently-live row’s decoded column values, keyed by rowid, across all user table b-trees. This is the value-aware companion to Database::live_rowids: the forensic carver uses it to tell a stale rebalance copy (same rowid AND same values → drop) from a deleted prior version (same rowid but DIFFERENT values → recover, e.g. an edited message or a changed amount).

Column values are decoded by inferring the column count from each live cell’s own serial-type array (the same self-describing record format the carver uses), so no schema column count is required. Best-effort, bounded, and panic-free: a malformed b-tree yields a partial map.

pub fn live_schema_rows(&self) -> Vec<Vec<Value>>

Decode every currently-live sqlite_master row (the schema table rooted at page 1) into its column values: (type, name, tbl_name, rootpage, sql). This is the schema-table companion to Database::live_rows, which collects only USER-table b-trees and so never sees the schema rows themselves.

The forensic carver folds these into the same value-based live set it uses to drop stale copies of live user rows: a record carved from a materialized page 1 whose values equal a CURRENT schema row is the live schema entry re-surfaced (drop it), whereas a genuinely-deleted PRIOR schema version has different values (e.g. an old CREATE TABLE) and is still recovered.

Best-effort, bounded, and panic-free: an unreadable schema yields an empty vector rather than an error.

pub fn live_tables(&self) -> Vec<LiveTable>

Every live (schema-present) user table, as attribution::LiveTable: name, rootpage, parsed column names (or None when low-confidence), and declared column affinities. Internal sqlite_* tables are excluded.

The forensic attribution step uses this to know each table’s real column names (Tier-1) and its shape signature (Tier-2). Best-effort, bounded, panic-free: an unreadable schema yields an empty vector.

pub fn schema_sql(&self) -> BTreeMap<String, String>

The live sqlite_master as a name -> CREATE SQL map for every user table (internal sqlite_* tables excluded) — the CURRENT-schema half of the Detector-B sidecar schema-change comparison (docs/design/drop-recreate-attribution.md).

Reads the same page-1 schema b-tree as Self::live_tables but keeps the raw CREATE SQL text (not just parsed columns), so a caller can compare the verbatim schema against a sidecar’s prior sqlite_master. Best-effort, bounded, panic-free: an unreadable schema yields an empty map.

pub fn row_histories(&self) -> Vec<TableHistory>

Per-table, per-rowid VERSION HISTORY reconstructed from this database’s WAL temporal model (or just the live view when no -wal is present).

See row_history for the full model. Walks each salt epoch’s commit snapshots in commit order, then the final live view, and emits — per rowid — the sequence of distinct record values it held (insert / update / delete / reinsert), with evidence-based row_history::ViewState and NO timestamps. Degrades cleanly to live-only history when Database::wal_timeline is None. WITHOUT ROWID tables are recorded with without_rowid = true and no versions (they have no rowid to key a history on).

pub fn sqlite_sequence(&self) -> BTreeMap<String, i64>

The sqlite_sequence table SQLite maintains for AUTOINCREMENT tables, as name → seq — seq being the highest rowid ever assigned to that table (its monotonic INSERT high-water mark).

sqlite_sequence exists only once at least one AUTOINCREMENT table has been created; a database with none returns an empty map (never a fabricated seq = 0), so a caller can distinguish “no high-water mark” from “high-water mark of 0”. Best-effort, bounded, panic-free: an unreadable sqlite_sequence b-tree, or a malformed row, is omitted rather than erroring. Note sqlite_sequence is a mutable user table — seq tracks the INSERT high-water mark, not live rowid assignment — so this is a forensic HINT input, not proof of any row’s provenance.

pub fn live_table_rows(&self) -> Vec<LiveTableDump>

Dump every live user table for export: name, header columns, and all live rows in rowid order. The base layer the combined live + recovered workbook is built over.

For each Database::live_tables entry, the b-tree is read via Database::read_table (so rows arrive in ascending-rowid b-tree order). The header is the table’s real column names when the schema parse was confident, otherwise generic c0..c{N-1} sized to the widest row — a header is always present and never a fabricated name. Best-effort and panic-free: a table whose b-tree is unreadable contributes an empty row set rather than erroring.

pub fn page_to_table_map(&self) -> BTreeMap<u32, String>

A map from each allocated page that belongs to a live table’s b-tree to that table’s name. Built by walking every live table’s b-tree page set from its rootpage (interior + leaf pages). A page carved as Tier-1 in-page residue resolves to its owning table through this map.

Best-effort and bounded, mirroring live_rowids’s b-tree walk: a malformed b-tree contributes fewer entries rather than erroring.

pub fn read_table( &self, root_page: u32, column_count: usize, ) -> Result<Vec<Row>, Error>

Walk a single table b-tree rooted at root_page (1-based) and collect every leaf row as typed values. column_count is the table’s declared column count, used to apply the INTEGER PRIMARY KEY rowid-alias rule.

Shares ONE b-tree/overflow walk with the snapshot-scoped read (CommitSnapshot::read_table) via an internal page-source abstraction, so the live and historical paths can never diverge.