1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
//! [`Segment`] — the **OS segment aperture**: SEGMENT-aligned raw memory spans
//! handed up to the safe Cartographer.
//!
//! This module is now a **thin wrapper** over the `aligned-vmem` crate
//! (`crates/vmem`), which carries all the platform-specific OS syscalls
//! (`mmap`/`munmap`/`madvise` on Unix, `VirtualAlloc`/`VirtualFree` on
//! Windows). The `unsafe` live in `aligned_vmem`; this file exposes a safe
//! interface that matches the original `os.rs` contract so the rest of
//! `alloc_core` is unchanged.
//!
//! ## What changed vs. the old os.rs
//!
//! | Item | Old | New |
//! |------|-----|-----|
//! | `SEGMENT`, `PAGE`, `segment_base_of`, `segment_base_of_ptr` | defined here | defined here (unchanged) |
//! | `Segment { base, len, reservation, reservation_len }` | full impl | thin wrapper over `aligned_vmem::Reservation` |
//! | `release_segment(reservation, reservation_len)` | direct OS call | `aligned_vmem::release(ptr, len, SEGMENT)` |
//! | `decommit_pages` / `recommit_pages` | direct OS calls | `aligned_vmem::decommit` / `aligned_vmem::recommit` |
//!
//! ## Miri aperture
//!
//! `aligned-vmem` already contains the miri fallback (`std::alloc` with the
//! requested alignment); no miri-specific code is needed here.
// The crate is `#![deny(unsafe_code)]` with `alloc-core` on; this is one of
// the documented `unsafe` seams. `allow` lifts the crate-level `deny` for
// this file only so we can call `aligned_vmem::release` (an `unsafe fn`).
use NonNull;
use aligned_vmem as vmem;
/// The segment size and alignment, in bytes. 4 MiB — mimalloc's default. Every
/// [`Segment`] handed up by this module is aligned to a multiple of this value,
/// so [`crate::alloc_core::segment_of`] can find an allocation's owning segment
/// header in O(1) by masking the low bits of its address.
///
/// This is exposed (read-only) as [`super::SegmentLayout::SEGMENT`].
pub const SEGMENT: usize = 1 << 22;
/// A page size used by the page-granularity `PageMap`. 4 KiB — the smallest
/// unit both `mmap` (unix) and `VirtualAlloc` (windows) will commit/decommit.
/// Kept independent of [`SEGMENT`] so the page tables stay small (1024 pages
/// per segment) while the alignment mask is the segment mask.
///
/// Re-exported from `aligned_vmem::PAGE` for a single source of truth.
pub const PAGE: usize = PAGE;
/// Convert an address to the SEGMENT-aligned base it falls within.
///
/// Pure safe arithmetic — this is part of the Cartographer and lives outside
/// the `unsafe` seam logically, but is so tightly coupled to [`SEGMENT`] that
/// it is defined here next to the constant. Re-exported via
/// [`super::SegmentLayout`].
pub const
/// Convert a raw pointer to the SEGMENT-aligned base pointer it falls within,
/// **preserving provenance**.
///
/// This is the strict-provenance–clean equivalent of the old
/// `segment_base_of(ptr as usize) as *mut u8` idiom: `ptr as usize` strips
/// provenance (exposed-address cast, forbidden under `-Zmiri-strict-provenance`),
/// while `ptr.map_addr(|a| a & !(SEGMENT - 1))` rounds the address down within
/// the same provenance domain — sound under both permissive and strict-provenance
/// models. The returned pointer carries the same provenance as `ptr` and points
/// to the SEGMENT-aligned base of the segment that contains `ptr`.
///
/// `ptr` MUST lie within a segment owned by this allocator (the Cartographer's
/// invariant); the result is the base of that segment.
pub
/// A owning handle to one SEGMENT-aligned span of raw memory.
///
/// `base` is non-null, aligned to `SEGMENT`, and valid for `len` bytes for the
/// lifetime of this `Segment`. Dropping the `Segment` returns the whole
/// underlying OS reservation to the OS exactly once. The span is **not**
/// initialised — callers must not read uninitialised bytes (matching the
/// `GlobalAlloc::alloc` contract).
///
/// Internally wraps [`aligned_vmem::Reservation`]; exposes the same API as the
/// original `Segment` type so all call sites are unchanged.
///
/// `Segment` is `Send` (but not `Sync`): the span is owned exclusively by the
/// sending thread. `&Segment` grants only read access to the metadata.
;
// `Send` is already implemented on `aligned_vmem::Reservation`; we simply
// forward it. `Segment` wraps one `Reservation`, which is `Send`, so `Segment`
// is `Send`.
// SAFETY: `Segment` owns its OS reservation exclusively. Same reasoning as
// for the old `Segment`: moving it to another thread transfers exclusive
// ownership of every byte with no aliasing on the origin thread.
unsafe
// NOTE: `Segment` is intentionally NOT `Sync` (same as before). Writes into
// the span happen via raw pointers in the `node` seam under the single-threaded
// Phase 8 invariant; no synchronisation for concurrent writes.
/// Release a whole OS reservation. Thin wrapper over
/// [`aligned_vmem::release`].
///
/// # Contract (caller's invariant — not enforced by the type system)
///
/// `reservation` must be a pointer previously returned by
/// [`Segment::reserve`] (specifically its [`Segment::reservation`]) and not
/// yet released. `reservation_len` must be the matching length. Must be called
/// exactly once per reservation.
pub
/// Decommit the payload pages of a segment: return their physical backing to
/// the OS while keeping the address-space reservation alive. Thin wrapper over
/// [`aligned_vmem::decommit`].
///
/// `base` is the SEGMENT-aligned base. We decommit `[base + start_offset,
/// base + end_offset)` — typically the payload region past the metadata. The
/// offsets MUST be page-aligned and within the segment.
// M6 infrastructure; exercised by the soak test.
pub
/// Recommit previously-decommitted pages within a segment. Thin wrapper over
/// [`aligned_vmem::recommit`].
// M6 infrastructure; future heap integration.
pub