Module pool 

Blob wrappers for reading and writing data with integrity guarantees, plus a buffer pool that manages read caching over the data.

Page-oriented structure

Blob data is stored in pages having a logical page_size dictated by the managing buffer pool. A physical page consists of page_size bytes of data followed by a 12-byte CRC record containing:

| len1 (2 bytes) |  crc1 (4 bytes) | len2 (2 bytes) | crc2 (4 bytes) |

Two checksums are stored so that partial pages can be re-written without overwriting a valid checksum for its previously committed contents. A checksum over a page is computed over the first [0,len) bytes in the page, with all other bytes in the page ignored. This implementation always 0-pads the range [len, page_size). A checksum with length 0 is never considered valid. If both checksums are valid for the page, the one with the larger len is considered authoritative.

A full page is one whose crc stores a len equal to the logical page size. Otherwise the page is called partial. All pages in a blob are full except for the very last page, which can be full or partial. A partial page’s logical bytes are immutable on commit, and if it’s re-written, it’s only to add more bytes after the existing ones.

Structs

Append

A Blob wrapper that supports write-cached appending of data, with checksums for data integrity and buffer pool managed caching.

PoolRef

A reference to a page cache that can be shared across threads via cloning, along with the page size that will be used with it. Provides the API for interacting with the buffer pool in a thread-safe manner.

Replay

Replays logical data from a blob containing pages with interleaved CRCs.