Skip to main content

Module overflow

Module overflow 

Source
Expand description

Overflow storage for cells that don’t fit on one table-leaf page.

Two pieces live here:

  • OverflowRef — the on-page marker that replaces a full cell when the cell’s body is too large to keep inline. It carries the rowid (so the page’s slot directory stays rowid-ordered), the total size of the external body, and a pointer to the first overflow page of the chain that holds the body.
  • write_overflow_chain / read_overflow_chain — turn raw bytes into a chain of Overflow-typed pages and back. Each overflow page reuses the existing 7-byte page header (type tag + next-page + payload length) — we’re not adding a new page format.

Decision to inline the rowid and NOT inline any of the body. SQLite’s leaf-cell scheme keeps a prefix of the body inline before spilling, so small lookups by rowid don’t need a chain walk. We’d still have to chase the chain for most columns anyway, so for simplicity this implementation spills the entire body. A later optimization can split cells at a threshold and keep a prefix inline without changing the page layout.

Overflow threshold. Inserting a cell whose encoded length is more than roughly a quarter of the page payload area (≈ 1000 bytes) is a good candidate for overflow — on a ~4 KiB page you can still keep at least 3-4 cells per page. The exact threshold is the caller’s choice; this module just exposes OVERFLOW_THRESHOLD as a suggestion.

Structs§

OverflowRef
On-page marker that stands in for a cell whose body lives in an overflow chain. Rowid is inlined so the page’s binary search over slots still works without chasing the chain.

Enums§

PagedEntry
An on-page entry: either a full local cell, or a pointer to an overflow chain carrying the cell’s body.

Constants§

OVERFLOW_THRESHOLD
Inline cell-body size above which the caller should consider overflowing. Sized so at least 4 inline cells can coexist on a page alongside their slot directory.

Functions§

read_overflow_chain
Walks an overflow chain starting at first_page and concatenates its payload bytes. Reads exactly total_body_len bytes — a mismatch between what the chain carries and what the OverflowRef claims is a corruption error.
write_overflow_chain
Writes bytes into a chain of Overflow-typed pages starting at start_page, using consecutive page numbers. Returns the first page number after the chain (i.e., the next free page to hand out).