pub struct Heap { /* private fields */ }Expand description
One shard’s heap. Not Sync: exactly one thread owns it, which is
what removes the atomics from the fast path.
Implementations§
Source§impl Heap
impl Heap
Sourcepub fn flush_claims(&mut self)
pub fn flush_claims(&mut self)
Retire every class’s claim — the write-back before anything that reads span occupancy as truth (reclaim’s page sweep, drop).
Source§impl Heap
impl Heap
Sourcepub fn drain_foreign(&mut self)
pub fn drain_foreign(&mut self)
Move every slot other shards freed back onto its own span’s list.
Source§impl Heap
impl Heap
Sourcepub const fn new(id: usize) -> Self
pub const fn new(id: usize) -> Self
A heap owning nothing. id identifies the shard in stats and in
segment headers.
Sourcepub const fn with_class_cap(id: usize, class_cap: u32) -> Self
pub const fn with_class_cap(id: usize, class_cap: u32) -> Self
A heap with a tighter per-class ceiling than PER_CLASS_CAP.
The default is a runaway guard set beyond any real workload, which leaves the refusal path unreachable in a test. This makes it reachable without pretending the default is smaller than it is.
Sourcepub fn ensure_identity(&mut self)
pub fn ensure_identity(&mut self)
Adopt this heap’s address as its identity, once.
Segments record their owner so a free arriving on the wrong
thread can be routed home. The address of the heap itself is a
ready-made unique identifier — no counter, no registry, and it
cannot collide while the heap is alive. 0 means “not yet set”,
which is why Heap::new can stay const.
Sourcepub fn alloc(&mut self, size: usize, align: usize) -> Option<NonNull<u8>>
pub fn alloc(&mut self, size: usize, align: usize) -> Option<NonNull<u8>>
Allocate size bytes aligned to align, or None if the OS or
a class cap says no.
Alignment up to class::MAX_NATIVE_ALIGN is served by choosing
a suitable class. Stricter requests fall to the direct-mapping
path, which returns page-aligned memory; anything beyond a page
belongs to the GlobalAlloc shim’s over-aligned path.
Sourcepub unsafe fn try_resize_in_place(
&mut self,
ptr: NonNull<u8>,
old_size: usize,
new_size: usize,
align: usize,
) -> bool
pub unsafe fn try_resize_in_place( &mut self, ptr: NonNull<u8>, old_size: usize, new_size: usize, align: usize, ) -> bool
Grow or shrink in place when the block’s size class does not change, reporting whether it worked.
This is the capability a general-purpose allocator gets from its
chunk headers: glibc can often extend a block where it lies
instead of moving it. Without it, GlobalAlloc’s default
realloc allocates, copies and frees on every growth — and a
profile of pub/sub showed exactly that, with libc realloc
visible and cheap on the system side and nothing corresponding
on ours.
Refuses when the block belongs to another thread. Adjusting the owner’s counters from here is precisely what this design does not do, and the caller falls back to allocate-copy-free, which routes the release home correctly.
§Safety
ptr must be a live allocation from this allocator made with
old_size and align.
Sourcepub unsafe fn dealloc(&mut self, ptr: NonNull<u8>, size: usize, align: usize)
pub unsafe fn dealloc(&mut self, ptr: NonNull<u8>, size: usize, align: usize)
Return an allocation. size must be the one it was made with —
the sized-dealloc contract is what lets us store no headers.
§Safety
ptr must come from Self::alloc on this heap with this
size, and must not be used afterwards.
Source§impl Heap
impl Heap
Sourcepub fn reclaim(&mut self)
pub fn reclaim(&mut self)
Return free pages to the OS — whole spans where nothing is live, and individual pages inside spans that still are. The second half is v2 (RFC §5.1): M3 measured the whole-span rule returning 3 % because a span only empties when all its slots die together, while glibc works at page granularity. Now so do we.
Drains foreign frees first: slots parked by other shards pin their pages exactly as live slots do, so sweeping before draining under-returns for no reason.
The retained count is per sweep rather than cumulative. A running counter looked equivalent and was not: it only ever grew, so the second sweep found it already past the threshold and returned everything, which made the hysteresis vanish after one call.
Source§impl Heap
impl Heap
Sourcepub fn snapshot(&self) -> Stats
pub fn snapshot(&self) -> Stats
Where every mapped byte is (bench/V5-ACCOUNTING-CONTRACT.md §1).
Walks the segments rather than maintaining seven counters on the
hot path: only live and rounding depend on the requested size
and must be tracked as allocations happen. Stats are read on INFO,
not per operation.