Module sqlite3_ondisk

Expand description

SQLite on-disk file format.

SQLite stores data in a single database file, which is divided into fixed-size pages:

+----------+----------+----------+-----------------------------+----------+
|          |          |          |                             |          |
|  Page 1  |  Page 2  |  Page 3  |           ...               |  Page N  |
|          |          |          |                             |          |
+----------+----------+----------+-----------------------------+----------+

The first page is special because it contains a 100 byte header at the beginning.

Each page consists of a page header and N cells, which contain the records.

+-----------------+----------------+---------------------+----------------+
|                 |                |                     |                |
|   Page header   |  Cell pointer  |     Unallocated     |  Cell content  |
| (8 or 12 bytes) |     array      |        space        |      area      |
|                 |                |                     |                |
+-----------------+----------------+---------------------+----------------+

The write-ahead log (WAL) is a separate file that contains the physical log of changes to a database file. The file starts with a WAL header and is followed by a sequence of WAL frames, which are database pages with additional metadata.

+-----------------+-----------------+-----------------+-----------------+
|                 |                 |                 |                 |
|    WAL header   |    WAL frame 1  |    WAL frame 2  |    WAL frame N  |
|                 |                 |                 |                 |
+-----------------+-----------------+-----------------+-----------------+

For more information, see the SQLite file format specification:

https://www.sqlite.org/fileformat.html

Structs§

BuildSharedWal: Stream through frames in chunks, building frame_cache incrementally Track last valid commit frame for consistency Non-blocking driver for WAL recovery on open.
CacheSize: Read/Write file format version.
DatabaseHeader: Database Header Format
IndexInteriorCell
IndexLeafCell
OverflowCell
PageInner
PageSize: Read/Write file format version.
RawVersion: Raw version byte for use in DatabaseHeader where Pod is required. Use Version::try_from(raw.0) to convert to the validated enum.
TableInteriorCell
TableLeafCell
TextEncoding: Text encoding.
WalFrameHeader: Immediately following the wal-header are zero or more frames. Each frame consists of a 24-byte frame-header followed by bytes of page data. The frame-header is six big-endian 32-bit unsigned integer values, as follows:
WalHeader: The Write-Ahead Log (WAL) header. The first 32 bytes of a WAL file comprise the WAL header. The WAL header is divided into the following fields stored in big-endian order.

Enums§

BTreeCell
PageType
Version: Read/Write file format version.

Constants§

CELL_PTR_SIZE_BYTES
FREELIST_LEAF_PTR_SIZE
FREELIST_TRUNK_HEADER_SIZE
FREELIST_TRUNK_OFFSET_FIRST_LEAF_PTR
FREELIST_TRUNK_OFFSET_LEAF_COUNT
FREELIST_TRUNK_OFFSET_NEXT_TRUNK_PTR
INTERIOR_PAGE_HEADER_SIZE_BYTES
LEAF_PAGE_HEADER_SIZE_BYTES
LEFT_CHILD_PTR_SIZE_BYTES
MINIMUM_CELL_SIZE: The minimum size of a cell in bytes.
WAL_FRAME_HEADER_SIZE
WAL_HEADER_SIZE
WAL_MAGIC_BE
WAL_MAGIC_LE

Functions§

begin_read_page: Send read request for DB page read to the IO if allow_empty_read is set, than empty read will be raise error for the page, but will not panic
begin_read_wal_frame
begin_read_wal_frame_raw
begin_sync
begin_write_btree_page
begin_write_wal_header
build_shared_wal: Blocking shim over BuildSharedWal. Retained for the unit test and any caller not yet lifted to drive the recovery state machine directly.
checksum_wal: The checksum is computed by interpreting the input as an even number of unsigned 32-bit integers: x(0) through x(N). The 32-bit integers are big-endian if the magic number in the first 4 bytes of the WAL header is 0x377f0683 and the integers are little-endian if the magic number is 0x377f0682. The checksum values are always stored in the frame header in a big-endian format regardless of which byte order is used to compute the checksum.
finish_read_page
parse_wal_frame_header
payload_overflows: Checks if payload will overflow a cell based on the maximum allowed size. It will return the min size that will be stored in that case, including overflow pointer see e.g. https://github.com/sqlite/sqlite/blob/9591d3fe93936533c8c3b0dc4d025ac999539e11/src/dbstat.c#L371
prepare_wal_frame
read_btree_cell: read_btree_cell contructs a BTreeCell which is basically a wrapper around pointer to the payload of a cell. buffer input “page” is static because we want the cell to point to the data in the page in case it has any payload.
read_integer
read_u32
read_value: Reads a value that might reference the buffer it is reading from. Be sure to store RefValue with the buffer always.
read_value_serial_type
read_varint: Reads varint integer from the buffer. This function is similar to sqlite3GetVarint32
read_varint_partial: Reads a varint from the buffer, returning None if more data is needed.
validate_serial_type
varint_len: Compute the length of a varint encoding for a given u64 value.
write_pages_vectored: Write a batch of pages to the database file.
write_varint
write_varint_to_vec

Type Aliases§

PageContent: Type alias for backward compatibility - PageContent is now PageInner