Skip to main content

Module memory_map

Module memory_map 

Source
Expand description

Persisted memory id to page id map, replayed forward from the WAL on open. Persisted memory map: which page holds which memory.

Opening a database used to rebuild the memory id to page id map by loading every page, which on network storage costs minutes for large databases. The map is instead persisted at checkpoint time and replayed forward from the WAL on open, so a cold open reads only the pages the WAL touched.

§Anchoring

Each checkpoint appends a Checkpoint WAL entry whose payload is a random 16 byte token, then writes the snapshot stamped with the same token, then truncates the WAL to that entry. The snapshot is the map as of its checkpoint entry, so the map now is the snapshot plus every WAL entry after the entry carrying its token. A snapshot whose token is not in the WAL (the WAL was truncated by a checkpoint or recovery that did not write a snapshot, the snapshot write failed, or the file is from another database) can never be matched again, because tokens are never reused for a changed map. On any doubt (missing, corrupt, unmatched, inconsistent page count) the caller falls back to a full page scan.

The table is keyed by page, mirroring exactly what a full scan observes: a page holds a memory when its last write decodes as one, and a freed page holds none. Each entry also carries a stamp, a random value refreshed on every write applied to that page, which lets derived caches (mode probes) persist alongside and detect, without loading the page, that the memory changed since they were captured.

Structs§

MappedMemory
A memory the map knows about: its id, its page, and the page’s write stamp.

Constants§

MEMORY_MAP_FILE
File name of the persisted memory map inside the database directory.