Module record 

The record, which is the unit of everything: a commit, a replay step, and a thing compaction either copies or drops.

06 section 2.1 for the layout, 06 section 3 for the ordering rule that makes the layout safe.

Two details that the table in the specification does not make obvious, and that everything downstream depends on.

len is exact and the stride is aligned. Records are eight byte aligned, but the alignment is padding between records rather than padding inside one. If len were rounded up, a value’s length would be len minus the header minus the key minus somewhere between zero and seven, and nobody could say which. So len is the exact byte count and the next record starts at RecordRef::stride bytes later. A reader that walks by len instead of by stride desynchronises on the first odd sized value, which is why there is a test for exactly that.

The trailer, and why it is not optional. 07 section 4 says a per record checksum lives in the record’s own trailer for records marked durable, and the field table in 06 does not list one. This module reconciles them: record_flags::CHECKSUMMED is the mark, and when it is set the last four bytes of the record are a CRC32C over everything before them, len included.

It started out as a real choice, on the reasoning that in none mode nobody reads the record back off a disk and four bytes on a sixty byte record is real money. That reasoning is wrong, and the way it is wrong is worth keeping written down, because it is the sort of thing that reads as reasonable right up until a fuzzer finds it.

The flag lives in the record. So a single bit flip in the flags byte turns the check off, and with it off there is nothing left to notice that the bit flipped. Worse than undetected: clearing the bit also moves the trailer boundary, so those four checksum bytes become four bytes of value, and a reader hands back a value four bytes longer than the one that was stored and is confident about it. A self describing checksum cannot describe its own absence.

So RecordRef::parse refuses a record with the bit clear rather than believing it, and RecordHeader::fill always sets it. The flag stays in the layout so a later version can define what an unchecksummed record means with a second bit that is itself covered by something.

Modules

record_flags

The flags byte at offset 5.

Structs

RecordHeader

A record about to be written.

RecordIter

Walks the records in a page payload.

RecordRef

A record read back out of a page.

Enums

RecordKind

What a record holds. 06 section 2.1.

Constants

HEADER_LEN

The header without a TTL: len, kind, flags, klen, prev.

HEADER_LEN_TTL

The header with a TTL, which adds eight bytes at offset 16.

MAX_KEY_LEN

The largest key, because klen is a u16.

TRAILER_LEN

The trailer, when record_flags::CHECKSUMMED is set.

Functions

header_len

The header length implied by flags.

seal_len

Stores len, which is what publishes a record.

total_len

The bytes needed for a record with this shape.

trailer_len

The trailer length implied by flags.