Skip to main content

decommit

Function decommit 

Source
pub unsafe fn decommit(base: *mut u8, start: usize, end: usize)
Expand description

Decommit pages [base + start, base + end): hint the OS to return their physical backing while keeping the address-space reservation alive.

Programmatically check platform guarantees: use Reservation::decommit_reclaims_and_zeroes to query whether the current platform guarantees reclaim+zero-fill semantics. Returns true on Linux/Windows, false on Darwin/BSD where decommit is advisory-only.

Platform behavior:

  • On Linux and Windows this is guaranteed to return physical backing and zero-fill on next access (Linux MADV_DONTNEED, Windows MEM_DECOMMIT).
  • On the Darwin family (macOS/iOS/tvOS/watchOS) and the four BSDs (FreeBSD/DragonFly/NetBSD/OpenBSD), this is a best-effort hint with no zero-fill or reclaim guarantee — the physical pages may remain resident and old data may be observed after a decommit+recommit roundtrip. See Reservation::decommit_reclaims_and_zeroes.

start and end must be multiples of page_size() and within the span. A no-op if the range is empty AND page-aligned — and a VIOLATED range (start > end, or an endpoint not a multiple of page_size() — which includes an empty MISALIGNED range such as decommit(ptr, 1, 1)) is a silent no-op in a release build; see “Contract violations, by build profile” below for the debug-build tripwire and the fallible try_decommit form.

§Safety

  • base must be the as_ptr of a live reservation the caller owns.
  • end <= reservation.len() (the reservation’s usable span, in bytes) — this is a MANDATORY precondition of the pointer arithmetic the backends perform (base.add(start) in BOTH real backends’ decommit_pages_impl — Windows (src/os/windows.rs) before its VirtualFree(MEM_DECOMMIT) call, Unix (src/os/unix.rs) before its madvise call; the miri backend is a no-op that ignores base, and under aligned_vmem_mock no backend call happens at all, but the contract is stated platform-independently), not merely a functional/behavioral preference. Task #1235 correction: since task #1213/L2 (1522d25) this bullet enumerated the arithmetic as “base.add(start) / base.add(end)” — the second half never existed. No backend or FFI wrapper forms a pointer from end (both decommit_pages_impl bodies and the winapi_virtual_decommit / libc_madvise wrappers they call were read in full, task #1235): end’s only arithmetic role is the subtraction end - start — which cannot wrap on this function’s paths, since this function returns on start >= end before the backend is reached — whose result is handed to the OS as a byte LENGTH. With start <= end, this single bound is what keeps the one offset that IS computed, base.add(start), inside the allocation, and what keeps the OS call’s span [base+start, base+end) inside the reservation. This requirement is stated here explicitly (task #1213/L2) rather than left to the summary line above (“within the span”) — for an unsafe fn, a bounds requirement that determines whether pointer arithmetic is even defined belongs inside # Safety itself, restated in full, not referenced from an adjacent paragraph a caller auditing only this section could miss. Passing end > reservation.len() is undefined behavior, distinct from — and a strictly worse violation than — the page_size()-multiple contract below, which is merely a silent no-op on violation, never UB.
  • [base+start, base+end) must contain no data the caller still needs — its contents are discarded.

Contract violations, by build profile (task #1051): this entry point is intentionally infallible — the () return carries no write-permitting sentinel to misuse — so a violated range (start > end, or an endpoint not a multiple of page_size()) is a silent no-op in a RELEASE build: no OS call is made and nothing is recorded. In a DEBUG build the same violation trips the debug_assert! below before anything happens, so a consumer’s own test fails at the mistake instead of quietly decommitting nothing and leaving the memory resident; zero cost in release. try_decommit is the fallible form for callers who need the violation reported: it returns Err on every profile and never trips the tripwire.

A poisoned page-size query is a DIFFERENT case and never panics, on any profile (task #1145/#1139, sharpened task #1173/L1): if the one-time OS page-size query itself failed (see page_size()’s “If the one-time OS query fails”), this function fails closed silently — no debug_assert!, no tripwire — because the caller’s arguments are not at fault and the crate-wide poison contract promises an unconditional no-op here, matching decommit_lazy’s no-tripwire design and the README’s “never panics” list. This is distinct from the range-contract tripwire immediately above, which fires only in debug builds and only for a violated range under a HEALTHY page-size query.

Platform divergence, not just a data-loss concern: on Windows, MEM_DECOMMIT genuinely unmaps the pages, so a write to [base+start, base+end) before recommit is a hard STATUS_ACCESS_VIOLATION crash, not a soft re-fault. On Linux, MADV_DONTNEED keeps the mapping resident and transparently re-faults a fresh zero page on next write, so the same code that is safe on Linux can crash on Windows. This exact divergence already crashed an in-repo consumer that assumed the Linux semantics — see https://github.com/PHPCraftdream/sefer-alloc/blob/main/docs/CORRECTNESS_OPEN_ITEMS.md item 6 (filed 2026-07-30) for the incident record and status.

Huge-page granularity (task #843 V4/finding R4-4, corrected task #1140): on huge-page reservations (those returned by reserve_aligned_huge with Reservation::is_huge == true), on Windows, decommit does not work at all: VirtualFree with MEM_DECOMMIT unconditionally fails on large-page regions.

On Linux/Android, whether decommit works depends on the requested range and the running kernel, not on whether the mapping is huge — madvise(2) documents that MADV_DONTNEED gained HugeTLB support in Linux 5.18, with the same requirement it already has for ordinary mappings: [base+start, base+end) must be aligned to the mapping’s huge page size (2 MiB on this crate’s supported targets) at BOTH endpoints. This crate’s own Linux/Android huge-pages contract already requires reserve_aligned_huge’s size/align to be multiples of that same 2 MiB, so a huge-aligned [start, end) is not a hypothetical — decommitting an entire huge reservation, or any 2-MiB-granular sub-range of it, is exactly such a range. A page_size()-granular (e.g. 4 KiB) but NOT 2-MiB-granular offset still gets EINVAL from the kernel and does nothing — this free function issues the syscall regardless of eligibility (unlike Reservation::decommit, which can consult Reservation::is_huge and the requested range to skip the ineligible case before the syscall — see that method’s doc for the exact split), so an ineligible range here is a wasted syscall that the kernel itself turns into a no-op, not a Rust-level skip. On a pre-5.18 kernel, EVERY range is ineligible regardless of alignment (the capability did not exist yet), so decommit is unconditionally a no-op there, matching the prior (task #843) documented behavior exactly. Either way — ineligible range, or eligible range on a pre-5.18 kernel — the effect is indistinguishable from a silent no-op: the caller’s RSS does not decrease, and subsequent reads return the old (stale) data rather than zeroed pages.

Documented per the madvise(2) man page cited above, and — since task #1152 (F1) — empirically exercised by this crate’s own CI: the aligned-vmem-hugetlb-real job (.github/workflows/ci.yml) configures a real nr_hugepages pool and hard-asserts (via a dedicated path-activation oracle) that reserve_aligned_huge actually received a MAP_HUGETLB grant rather than silently falling back to ordinary pages. Under that real grant, the job runs huge_aligned_range_takes_the_real_backend_path_not_the_skip_path and huge_decommit_attempts_increments_on_huge_reservation (tests/decommit_capability.rs), which drive a huge-page-eligible [start, end) through Reservation::decommit’s eligible-huge branch — the same decommit_pages_impl/MADV_DONTNEED backend call this free function itself makes. What that job proves, stated precisely (task #1160/F1 correction of an earlier overclaim; strengthened tasks #1164 and #1174): the eligible-range case genuinely REACHES the real madvise(2)/MADV_DONTNEED backend call under a real MAP_HUGETLB grant, rather than silently taking the Rust-level skip path — AND, since task #1164’s ci_hugetlb_real_pool_kernel_actually_accepts_eligible_madvise (tests/decommit_capability.rs), the kernel’s own syscall-level response is also asserted: under bench-internals, libc_madvise (src/os/unix.rs) records whether the syscall returned 0 or -1, and that job hard-asserts it returned 0 for this eligible-range call — AND, since task #1174’s ci_hugetlb_real_pool_decommit_actually_zeroes_memory_on_reaccess (tests/decommit_capability.rs), the zero-fill half of the effect (as opposed to the return code) is no longer reasoned from the man page either: that test writes a non-zero byte pattern across the whole eligible range, calls Reservation::decommit, then reads every byte back and hard-asserts each one is zero — zero-fill-on-readback is proven for this eligible-range/post-5.18 case on a Linux runner (the code path is gated Linux and Android as a pair; the Android half is inherited from that shared cfg, not separately executed by any CI job). What still remains NOT proven, deliberately kept separate from that zero-fill result: that the kernel actually returned the physical backing to the OS/hugetlb pool — the job logs HugePages_Free around that test as an observation only, never a pass/fail gate, because it is a kernel-global counter shared with the job’s other huge-page reservations. On builds WITHOUT bench-internals, libc_madvise still discards the return value entirely (task #719) — the kernel-response proof above is scoped to the one CI job that enables the counters. It also does not call this free function’s own entry point directly (no test invokes decommit outside a Reservation method), so this function’s own unconditional-syscall behavior on an INELIGIBLE range (still a no-op by kernel contract, not by Rust-level skip) remains reasoned-from-spec, not independently exercised under a real pool.

Diagnostic visibility: under the bench-internals feature, the huge_decommit_attempts counter (not an intra-doc link: bench-internals is excluded from the published docs.rs feature set) is incremented each time Reservation::decommit/Reservation::try_decommit skip the backend call on a huge-page reservation — it is NOT incremented by calls through this free function (which has no is_huge() to consult and always issues the syscall) or by an eligible Linux/Android >= 5.18 huge-aligned call through the safe methods (those forward to the real backend instead of skipping). Use reserve_aligned instead of reserve_aligned_huge if you need decommit to work unconditionally, regardless of range shape or kernel version.

Darwin zero-fill gap (confirmed as a real, failing-test-level gap by this crate’s first real-macOS CI run, 2026-08-13 — the underlying hazard was already known repo-wide since Round 9, see https://github.com/PHPCraftdream/sefer-alloc/blob/main/docs/CORRECTNESS_OPEN_ITEMS.md item 48): MADV_DONTNEED on Darwin and the four BSDs (FreeBSD/DragonFly/ NetBSD/OpenBSD) is advisory-only for anonymous memory — unlike Linux, it does not reliably unmap the physical pages, so a decommit + recommit roundtrip on these OS families (macOS/iOS/tvOS/watchOS — all share XNU and the same MADV_DONTNEED semantics, not just macOS — plus the four BSDs which use identical MADV_DONTNEED semantics) can observe the OLD data still resident instead of a fresh zero page. This is the same “indistinguishable from a silent no-op” shape as the huge-page case above, but for ORDINARY (non-huge) reservations on Darwin and the BSDs specifically. See https://github.com/PHPCraftdream/sefer-alloc/blob/main/docs/CORRECTNESS_OPEN_ITEMS.md item 48 for the open item; no fix is implemented yet (the real fix needs re-mmap(MAP_FIXED) over the range, a larger change deserving its own review round). Note: this caveat applies only to the EAGER decommit path (which uses MADV_DONTNEED on all Unix); the lazy decommit_lazy path uses MADV_FREE-family advice on Darwin/BSDs and DOES free pages on those platforms.