Crate sqlite_forensic

Expand description

sqlite-forensic — Tier-2 anomaly auditor over sqlite_core.

WS-C skeleton. The reader (sqlite-core) answers “what does this database header show?”; this crate grades the forensically-notable observations into severity-ranked forensicnomicon::report::Findings, so a SQLite evidence database’s anomalies aggregate uniformly with the partition / container / filesystem layers.

Each anomaly is an observation (“consistent with …”); the examiner draws the conclusions.

§Capabilities

carve_deleted_records — recover deleted rows from free (unallocated) pages, the headline capability rusqlite structurally cannot provide. Each recovered row is confidence-graded, flagged allocated: false, and carries page/offset/rowid provenance.
audit grades header reserved-space, a non-empty freelist (prior deletions), an active WAL overlay (uncheckpointed state), and a header/file page-count mismatch into severity-ranked forensicnomicon::report::Findings.
audit_journal grades the rollback--journal design-§6 observations (hot journal, recoverable PERSIST pre-images, checksum mismatch, journaled schema page, duplicate page record, db-size delta) — additive to audit.

Deferred: a full anomaly suite (overflow-chain integrity, schema-format / text-encoding checks) and a fuzz harness.

Modules§

blob: Recovered-BLOB enrichment (roadmap §4.5): make carved binary payloads addressable in a case by a magic-based media type and a SHA-256 content hash.
carve: Whole-database SQLite carver satisfying the fleet forensic_carve::Carver contract (ADR 0001 §4).
case_uco: CASE/UCO JSON-LD export of recovered BLOBs (roadmap §4.5).
interpret: The blob-interpreter seam (roadmap §4.5 follow-up): a dependency-inversion contract so consumers can decode opaque BLOB values in schema context — WebKit Local Storage UTF-16, or the general blob-decoder crate (plist / gzip / JSON / …) — WITHOUT this reader library taking on any decode dependency or MSRV cost.

Structs§

Anomaly: A SQLite forensic anomaly: an observation graded by severity, with a stable code and note derived from its AnomalyKind so they cannot drift.
CarveTiers: The two strictly-separated recovery tiers returned by carve_with_fragments.
CarvedFragment: A Tier-2 partial recovery: a freed cell whose full row could not be reconstructed but at least one distinctive cell (TEXT ≥ 4 bytes of valid UTF-8, or REAL) survived at a structural anchor.
CarvedRecord: A deleted record recovered from unallocated space — the headline capability rusqlite cannot provide. Carries the decoded row plus provenance so the examiner can weigh it as a “consistent with a deleted row” observation.
JournalCounts: Structured counts of what the rollback-journal recovery found — the NIST SFT-03 “number of deleted/modified records” report (design §5/§6.11).
JournalRecovery: The result of recovering deletions + modifications from a rollback journal (design §5). Deleted rows are full-fidelity prior allocated rows (NOT free-space carves); modified rows carry the prior and current values.
OverflowProvenance: Provenance for a record reassembled across an overflow-page chain (task #73): the pages whose bytes were concatenated to recover the row. An examiner citing the row as evidence can name exactly where its bytes came from. Present only on chain-reassembled rows; None for every contiguous recovery.
PriorRow: A row recovered from the prior (pre-transaction) snapshot of a rollback journal (design §5). In the prior state it was a LIVE allocated row that the last transaction DELETED — so it is recovered at full fidelity and is NOT a free-space carve (allocated was true in the prior state). Carries full journal provenance in RollbackJournalSource.
PriorVersionRecord: A row present in BOTH the prior snapshot and the live db with DIFFERENT values — the last transaction modified it (design §4). Carries the PRIOR (pre-edit) values and the CURRENT values. replaced_rowid is set because a delete + insert reusing the same rowid is indistinguishable from an in-place UPDATE here, so identity may differ — never asserted as “UPDATE”.
RecoveredSchema: A schema-table (sqlite_master) row recovered from free space — a dropped or replaced table/index/view/trigger whose definition no longer appears in the live schema. Recovering it tells the examiner an object existed and its structure (the CREATE statement), which in turn explains residual data on the freelist. A finding, not a certainty: it is “consistent with a dropped <name>” — the examiner draws the conclusion.
RollbackJournalSource: Provenance for a record recovered from a rollback journal (design §5): where in the journal its page image lived and how trustworthy the parse was. Copy (all fields are scalar) so it threads through the existing attribution → output pipeline exactly like WalProvenance; the journal’s path is supplied once at the API boundary, not duplicated per row.
WalProvenance: Provenance for a record carved from a -wal frame: the (salt1, salt2, frame_index) log-sequence identity of the frame it came from (the LSN task #55 will formalize).

Enums§

AnomalyKind: The classified SQLite forensic anomalies this auditor can grade.
Attribution: How confidently a carved deleted row has been reattached to a live table.
JournalHeaderState: Whether a rollback-journal header was parsed intact (Tier A) or reconstructed from a zeroed (PERSIST post-commit) header (Tier B). A reconstructed header could not verify page checksums (the nonce was zeroed), so its recoveries rest on cross-record structural consistency — a graded distinction the examiner weighs.
RecoverySource: Which class of free space a deleted record was carved from. Records the recovery provenance so the examiner can weigh reliability by class.
TableInstanceRisk: A non-overclaiming diagnostic HINT about a recovered record’s relationship to the instance of the table it was attributed to — ORTHOGONAL to Attribution (it never changes the tier, the routing, or the table claim).

Functions§

attribute_record: Attribute one carved record to a tier given the live tables and the page→table map. Pure (no DB access) so it is directly unit-testable.
attribute_records: Attribute every carved record, reading the live tables and page→table map from db once. The returned vector is parallel to records.
audit: Audit an opened Database for forensically-notable anomalies.
audit_carved_findings: Carve deleted records and convert each to a canonical Finding under source. The per-record confidence is threaded into the finding’s context so downstream consumers can filter low-confidence recoveries.
audit_findings: Audit an opened Database and convert each anomaly to the canonical Finding under the supplied Source, ready to merge into a Report.
audit_journal: Audit a rollback -journal (raw bytes) bound to db for the design-§6 observations, additive to audit (which covers main-db anomalies only).
audit_journal_findings: Audit a rollback -journal and convert each design-§6 observation to the canonical Finding under source, ready to merge into a Report. The additive counterpart to audit_findings for the journal sidecar.
carve_all_deleted_records: Recover deleted records across every free-space class — the full-coverage carver. Drives, in order:
carve_at_commit: Carve the deleted residue of one materialized commit snapshot of the -wal, the per-commit temporal building block of the N-snapshot carve.
carve_deleted_records: Recover deleted records by carving the database’s free (unallocated) pages.
carve_rollback_journal: Recover the last transaction’s deletions and modifications from a rollback -journal by diffing the pre-transaction snapshot against the live database (design §4) — the temporal inverse of WAL recovery.
carve_with_fragments: Two-tier deleted-record recovery: Tier-1 full rows plus Tier-2 partial fragments, in one pass.
matching_tables: The names of every live table whose shape a record’s values match, in the order the tables were listed. Zero matches → unattributable shape; one → a clean guess; more than one → ambiguous.
recover_dropped_schemas: Recover dropped or replaced schema definitions from free space: carved sqlite_master-shaped rows whose (name, sql) no longer matches a live schema entry. A DROP TABLE/DROP INDEX deletes the object’s sqlite_master row (its bytes survive in page-1 free space under secure_delete=OFF); this surfaces that residue as a structured RecoveredSchema rather than a raw 5-column carved record.
row_histories_with_residue: The core per-rowid VERSION HISTORY (Database::row_histories) augmented with free-space CARVED RESIDUE — the forensic-layer completion of Phase 1.
shape_matches: Whether a carved record’s values match a live table’s shape: equal column count AND every value storage-compatible with the corresponding column affinity (value_fits_affinity).
table_instance_risks: Per-record TableInstanceRisk, parallel to records and their attributions — the Detector-A diagnostic pass.
table_instance_risks_with_sidecar: Per-record TableInstanceRisk over BOTH Detector A (bare AUTOINCREMENT high-water) AND Detector B (sidecar schema-change), the sidecar-aware pass the CLI composes when a -wal/-journal is in play (docs/design/drop-recreate-attribution.md).
value_fits_affinity: Whether a carved value is storage-compatible with a column of the given declared affinity — the per-cell test the shape matcher applies.