Module page

Expand description

Pages and their three sharded free lists (mirrors upstream page.c data side). A page is a slice-span inside a segment holding blocks of ONE size.

The three lists (the mimalloc signature move):

free — the allocation fast path pops here; when it runs dry the slow path runs at a regular cadence (the heartbeat).
local_free — frees from the owning thread; swapped into free on collect. Separate so the fast list running dry MEANS a heartbeat is due.
xthread_free — frees from OTHER threads: an atomic word packing a block-list head with a 2-bit protocol flag (loom-modeled in tests/loom_xthread.rs, which is the specification): NORMAL remote pushes land here; DELAYED remotes nudge the OWNER’s delayed list instead (page invisible to scans: full queue / large span); FREEING transient guard while a remote dereferences the heap pointer — the abandoner spins this out before heap teardown; NEVER abandoned.

Owner-only fields (everything non-atomic) are mutated exclusively by the owning thread (Segment::thread_id gates entry in alloc::free).

Modules§

pflags: Page::flags bits. The FREE fast path must answer one question — “is this a plain binned page I can just push onto?” — and it used to answer it with three separate loads (has_aligned, bin == BIN_HUGE, in_full) plus the segment’s kind. Folding them into one byte turns that into a single load and a single test-against-zero (M9 brick #3).

Block: A free block: the first word of the block memory itself links the list.
DelayedList: A heap’s cross-thread delayed-free list. Lives inside the owner’s HeapBox; pages carry its address in xheap so remote threads can reach it without knowing the heap type. Plain Treiber push / owner swap-drain.
EmptyPage: Wrapper so the sentinel can be a static.
Page: Page metadata. Lives in the owning segment’s header slice; the payload (“page area”) is the corresponding slice span.

XFLAG_DELAYED: Remote frees push onto the owning heap’s delayed list.
XFLAG_FREEING: Transient: a remote holds the heap pointer; others spin, abandoner waits.
XFLAG_NEVER: Abandoned: no owning heap; remote frees use the page list (adopter drains).
XFLAG_NORMAL: Remote frees push onto the page’s own xthread list.
XMASK: Flag mask in the xthread_free word (blocks are ≥ 8-aligned).

block_next^⚠: Read a free-list link (decoding under secure).
block_set_next^⚠: Write a free-list link (encoding under secure).
debug_validate_page^⚠: debug_checks invariant guard (our dmi equivalent): a page slot handed to the hot paths must be a live SPAN START with self-consistent counters. Catching a violated invariant here turns “mystery access violation” into “this field was wrong, at this call site”.
empty_page_ptr: Pointer to the shared empty page, for Heap::direct slots with no page.
page_all_free^⚠: Whether every block of the page is free (as seen by the owner; remote frees count only after a collect).
page_collect^⚠: Collect: swap local_free and steal the xthread list (flag preserved) into free. Called on the slow path when free is dry — the heartbeat.
page_extend^⚠: Lazily extend the free list into never-used capacity (mi_page_extend_free).
page_pop^⚠: Pop a block off the fast list. Returns null when dry (→ generic path).
page_push_local^⚠: Push a block on the owner free list (mi_free local path).
page_set_flag^⚠: Owner/abandoner flag transition, spinning out any in-flight FREEING.
remote_free^⚠: Remote (non-owner) free — the loom-modeled protocol.