size_classes/lib.rs
1//! `size-classes` — const-built size-class tables + a compile-time-derived
2//! O(1) size→class lookup + an alignment-divisibility classifier.
3//!
4//! Every slab / pool / arena allocator reinvents the same trio: a table of
5//! block sizes, an O(1) map from a requested byte size to the smallest class
6//! that fits it, and a classifier that also honours alignment via stride
7//! divisibility. This crate packages that trio as a `const`-evaluated,
8//! `no_std`, zero-dependency, `#![forbid(unsafe_code)]` unit — the table
9//! shape is a parameter, so a consumer can bake its own scheme and still
10//! get the derived lookup and the alignment-aware classifier for free.
11//!
12//! ## The three pieces
13//!
14//! - [`build_table`] — a `const fn` sorted-merge of a geometric progression
15//! (`geo_count` classes, each `round_up(ceil(prev * num / den), min_block)`)
16//! with a strictly increasing, `min_block`-multiple, `>= min_block` list of
17//! explicit `extras` (page-aligned classes, an exact size the geometric
18//! run skips, a feature-gated medium tier, …).
19//! - [`build_size2class`] — derives the O(1) `size→class` lookup from a table
20//! at compile time with the monotone-pointer technique
21//! (`O(buckets + classes)` const-eval) and a compile-time `u8` pin.
22//! - [`SizeClasses::class_for`] — an O(1) fast path for `align <= min_block`
23//! and a provably-equivalent *jump* slow path for larger alignments: round
24//! `block` up to the next multiple of `align` via a bitmask, re-seed through
25//! the lookup, and so skip whole runs of non-divisible classes instead of
26//! stepping by one. Without it, a request whose `align` exceeds what the
27//! caller's classifier happens to handle silently falls through to the
28//! caller's whole-segment path — a real bug class in hand-rolled allocators
29//! (`sefer-alloc`'s own motivating case, the allocator this crate was
30//! extracted from: `align >= 512`). The classifier picks an
31//! `align`-*divisible* stride; see [`SizeClasses::class_for`]'s
32//! `# Preconditions` for the separate base-address requirement this crate
33//! cannot check.
34//! [`SizeClasses::try_class_for`] is the checked twin -- validates `align`
35//! instead of assuming it. Use it unless `align` is already known-valid by
36//! construction (e.g. taken from a [`core::alloc::Layout`]).
37//!
38//! ## The `huge` threshold is a policy parameter
39//!
40//! [`SizeClasses::is_huge`] compares against a caller-supplied
41//! [`Params::huge_threshold`]. The crate has no notion of an OS segment size;
42//! the consumer picks the threshold that separates "large" from "huge" for its
43//! own segment policy.
44//!
45//! ## Deriving lengths
46//!
47//! [`SizeClasses`] is generic over both the table length `N` (`geo_count` +
48//! `extras.len()`) and the lookup length `L` (`max_class / min_block + 1`,
49//! via [`size2class_len`]). Both are pure functions of the [`Params`], but
50//! `L` needs the built table's LAST entry (`max_class`) — there is no
51//! shortcut around building `TABLE` once to read it; [`SizeClasses::build`]
52//! then builds the same table again internally, from the same [`Params`],
53//! so the two never drift apart:
54//!
55//! ```text
56//! const PARAMS: Params = Params::new(MIN_BLOCK, (5, 4), GEO_COUNT, EXTRAS, HUGE_THRESHOLD);
57//! const N: usize = GEO_COUNT + EXTRAS.len();
58//! const TABLE: [usize; N] = build_table::<N>(PARAMS);
59//! const L: usize = size2class_len(TABLE[N - 1], MIN_BLOCK);
60//! static SC: SizeClasses<N, L> = SizeClasses::build(PARAMS);
61//! ```
62//!
63//! (Runnable form with concrete values in `crates/size-classes/README.md`.)
64
65#![forbid(unsafe_code)]
66#![deny(missing_docs)]
67#![no_std]
68
69/// Parameters for a size-class scheme, consumed by [`build_table`],
70/// [`build_size2class`] and [`SizeClasses::build`].
71///
72/// All fields are plain data so the whole thing is usable in `const` context.
73///
74/// `#[non_exhaustive]`, so a future policy field is a semver-minor addition
75/// rather than a breaking one. Construct with [`Params::new`] — a `const fn`,
76/// since downstream `#[non_exhaustive]` rejects struct-literal construction
77/// (functional-record-update included), leaving `new` as the only
78/// construction path, and `const` context needs that path callable. The
79/// non-breaking half rests on the fields being `pub`, not on `new`'s
80/// parameter list: a future `pub` field would extend the struct but not
81/// `new`'s positional signature, so existing `Params::new(..)` call sites
82/// keep compiling and a consumer opts in with `let mut p = Params::new(..);
83/// p.new_field = value;` — post-construction assignment to a `pub` field,
84/// which works outside this crate and in `const` context too.
85#[derive(Debug, Clone, Copy)]
86#[non_exhaustive]
87pub struct Params<'a> {
88 /// The minimum block size and the fundamental small-class alignment. Must
89 /// be a power of two. Every generated class is a multiple of it -- see
90 /// [`SizeClasses::class_for`]'s `# Preconditions` for what that does and
91 /// does not guarantee about block addresses.
92 pub min_block: usize,
93 /// The geometric growth ratio as `(num, den)` — each class after the first
94 /// is `round_up(ceil(prev * num / den), min_block)`, with a minimum step
95 /// of `min_block` so two adjacent classes never collide. `(5, 4)` is the
96 /// classic mimalloc 1.25× small spacing.
97 pub growth: (usize, usize),
98 /// How many classes the geometric progression contributes (starting at
99 /// `min_block`).
100 pub geo_count: usize,
101 /// Explicit extra classes to merge into the geometric run — a **strictly
102 /// increasing** list, each entry a multiple of `min_block` and `>=
103 /// min_block` (the builder sorted-merges them). All three preconditions
104 /// are **machine-checked**: a non-`min_block`-multiple entry, an entry
105 /// below `min_block` (rejects the degenerate `0` "class"), or a
106 /// non-strictly-increasing entry panics identically in `const` evaluation
107 /// (compile error) and at runtime in [`build_table`]. It also checks
108 /// disjointness from the geometric run at its own chokepoint (the merged
109 /// table must itself be strictly increasing). [`build_size2class`] keeps
110 /// the same check as defense-in-depth for a hand-built table that
111 /// bypasses [`build_table`] entirely. Typical uses: page-aligned classes,
112 /// an exact size the geometric run skips, a feature-gated medium tier.
113 ///
114 /// Borrowed rather than owned because this is a `no_std`, zero-alloc
115 /// crate; in the usual `const PARAMS: Params = Params::new(.., EXTRAS,
116 /// ..)` form `'a` resolves to `'static`, but nothing requires that.
117 pub extras: &'a [usize],
118 /// The "huge" policy threshold: [`SizeClasses::is_huge`] reports `true` for
119 /// a size `>=` this. Pure bookkeeping for the crate — the consumer decides
120 /// what "huge" means for its own segment policy (guard pages, eager
121 /// decommit, …).
122 pub huge_threshold: usize,
123}
124
125impl<'a> Params<'a> {
126 /// Construct a [`Params`] from its component fields.
127 ///
128 /// `const fn`, so it works in `const PARAMS: Params = Params::new(..);`
129 /// — the construction path for a `#[non_exhaustive]` type.
130 #[must_use]
131 pub const fn new(
132 min_block: usize,
133 growth: (usize, usize),
134 geo_count: usize,
135 extras: &'a [usize],
136 huge_threshold: usize,
137 ) -> Self {
138 Self {
139 min_block,
140 growth,
141 geo_count,
142 extras,
143 huge_threshold,
144 }
145 }
146}
147
148/// The `size2class` array length for a scheme whose largest class is
149/// `max_class`: one `u8` per `min_block`-sized bucket from `0` up to and
150/// including `max_class`. A consumer uses this in a `const` expression to pin
151/// the `L` generic of [`SizeClasses`].
152///
153/// # Memory cost
154///
155/// `L` (`max_class / min_block + 1`) is the byte size of the `size2class`
156/// LUT `SizeClasses` embeds, and it is NOT something a consumer picks
157/// directly -- it falls out of `min_block`, `growth`, `geo_count`, and
158/// `extras` together. It scales with `max_class / min_block`, not with the
159/// number of classes `N`, so a scheme with FEWER classes can still produce a
160/// LARGER LUT than one with more: a realistic scheme (`min_block = 16`,
161/// `growth = (5, 4)`, `geo_count = 40`, nine extras up to 16 KiB; the crate
162/// itself has no defaults) with 49 classes and `max_class = 258752` gives
163/// `L = 16173`. `table` itself is only `N * size_of::<usize>()` = 392 bytes
164/// on a 64-bit target; the LUT dominates -- `table` + `size2class` together
165/// are ~16.18 KiB, and `size_of::<SizeClasses<49, 16173>>()` itself is
166/// ~16.20 KiB on a 64-bit target, the difference being the struct's two
167/// scalar fields plus alignment padding. But a smaller `min_block` can
168/// outweigh a smaller class count entirely: `min_block = 8` with just 24
169/// classes (`growth = (3, 2)`, no `extras`) reaches `max_class = 145648` and
170/// `L = 18207`, a LARGER object than the 49-class example above. Concretely,
171/// for that same 49-class example the sparsity this scaling implies is
172/// large: buckets `888..=16172` — 15285 of the 16173 total, 94.5% — all
173/// resolve to just the 14 largest classes (indices `35..=48`), because
174/// class sizes grow geometrically while the LUT's own resolution stays a
175/// flat `min_block`.
176///
177/// # Panics
178///
179/// Panics -- identically in `const` evaluation and at runtime, since this is
180/// a `pub const fn` callable either way -- if `min_block` is not a power of
181/// two, or if `max_class / min_block + 1` overflows `usize` (reachable only
182/// for `min_block == 1` and `max_class == usize::MAX`; for any `min_block >=
183/// 2` the quotient cannot reach `usize::MAX`).
184///
185/// The `+ 1` overflow check is explicit rather than relying on the profile's
186/// default: a release-profile `const` evaluation reached through a `const fn`
187/// call follows the crate's `overflow-checks` setting and can silently wrap
188/// to `0` otherwise (<https://github.com/rust-lang/rust/issues/74823>).
189#[must_use]
190pub const fn size2class_len(max_class: usize, min_block: usize) -> usize {
191 assert!(
192 min_block.is_power_of_two(),
193 "size2class_len: min_block must be a power of two"
194 );
195 match (max_class / min_block).checked_add(1) {
196 Some(len) => len,
197 None => panic!("size2class_len: max_class / min_block + 1 overflows usize"),
198 }
199}
200
201/// Build the size-class table at compile time: a geometric progression merged
202/// with `params.extras` in sorted order, returned as `[usize; N]` where `N`
203/// must equal `params.geo_count + params.extras.len()`.
204///
205/// Spacing: start at `min_block`, then each next class is
206/// `round_up(ceil(prev * num / den), min_block)`, with a minimum step of
207/// `min_block`. The `extras` are merged in sorted order (a plain sorted-merge —
208/// `const fn` cannot call `slice::sort`), keeping the combined table strictly
209/// increasing and every entry a multiple of `min_block`.
210///
211/// `growth = (num, den)` with `num <= den` (including `(0, den)`) is a
212/// deliberately valid scheme, not a contract violation: a ratio `<= 1` makes
213/// the geometric term always `<= prev`, so every class falls back to the
214/// `min_block`-step minimum, degrading the whole run to a flat `min_block`,
215/// `2 * min_block`, `3 * min_block`, … sequence.
216///
217/// # Panics
218///
219/// Panics -- identically in `const` evaluation and at runtime, since this is
220/// a `pub const fn` callable either way -- if any of:
221///
222/// - `N != geo_count + extras.len()`;
223/// - `min_block` is not a power of two;
224/// - `geo_count == 0`;
225/// - `params.growth.1` (the growth denominator) is `0`;
226/// - any `extras` entry is not a multiple of `min_block`;
227/// - any `extras` entry is less than `min_block` (the scheme's minimum
228/// block size);
229/// - `extras` is not strictly increasing;
230/// - the geometric progression's advance step overflows `usize` (see the
231/// worked example below);
232/// - the merged table (geometric run + `extras`) is not itself strictly
233/// increasing -- the per-entry `extras` checks above catch misshapen
234/// `extras`, but not an `extras` entry that DUPLICATES a value the
235/// geometric run also produces, which only the merged table reveals. An
236/// `extras` entry landing strictly BETWEEN two geometric values is fine,
237/// and is one of the main reasons `extras` exists.
238///
239/// The advance-step overflow is reachable not just with an extreme
240/// `min_block`/`growth` combination but with a large enough `geo_count`
241/// alone: with `min_block = 16`, `growth = (5, 4)` (this crate's own tests'
242/// example scheme; the crate itself has no defaults), `geo_count = 183`
243/// already overflows on a 64-bit `usize` (`84` on a 32-bit one -- the
244/// boundary scales with `usize::BITS`). At the top of that range (roughly
245/// the last half-dozen steps -- the intermediate `cur * num` product first
246/// exceeds `usize` only once `cur > usize::MAX / num`) is exactly the
247/// widened-arithmetic case: the next class fits even though the
248/// intermediate `cur * num` product does not fit `usize`.
249#[must_use]
250pub const fn build_table<const N: usize>(params: Params) -> [usize; N] {
251 let min_block = params.min_block;
252 assert!(
253 min_block.is_power_of_two(),
254 "min_block must be a power of two"
255 );
256 assert!(params.geo_count > 0, "geo_count must be > 0");
257 // Only the DENOMINATOR is rejected: `num == 0` degrades to a linear
258 // min_block-step table via the min-step fallback below (a valid scheme,
259 // see this function's rustdoc), but `den == 0` has no such fallback and
260 // would otherwise surface as a bare "attempt to divide by zero".
261 assert!(params.growth.1 > 0, "growth denominator must be > 0");
262 let geo_count = params.geo_count;
263 let extras = params.extras;
264 // `checked_add` for the diagnostic, not for soundness: a wrapped sum
265 // could pass this check, but the merge below still runs the true
266 // iteration count and would panic on `out[oi]` with a bare index error.
267 // This names the actual bad parameter instead.
268 let n_matches = match geo_count.checked_add(extras.len()) {
269 Some(sum) => sum == N,
270 None => false,
271 };
272 assert!(n_matches, "N must equal geo_count + extras.len()");
273
274 let mask = min_block - 1;
275
276 // `extras` preconditions (documented on `Params::extras`): each entry a
277 // multiple of `min_block` (so the fast path's stride-divisibility
278 // predicate, implicit for `align <= min_block`, stays valid -- whether
279 // the resulting ADDRESS is aligned is a separate, caller-owned
280 // precondition; see `SizeClasses::class_for`'s doc), and the
281 // list strictly increasing (so the sorted-merge below actually produces
282 // a sorted table instead of silently reordering). Checked here — not
283 // just at the merged-table monotonicity checks below (this function's
284 // own, or `build_size2class`'s downstream defense-in-depth one) —
285 // because a non-multiple-of-`min_block` extra can still land in
286 // strictly increasing position relative to the geometric run
287 // (misalignment alone does not always break monotonicity), so global
288 // monotonicity is not sufficient to catch it.
289 {
290 let mut i = 0;
291 while i < extras.len() {
292 assert!(
293 extras[i] & mask == 0,
294 "Params::extras: every entry must be a multiple of min_block"
295 );
296 // Catches `0`, which the multiple-of check above accepts (0 is a
297 // multiple of everything) but which would land in the table as a
298 // zero-sized class no `Layout` can resolve to. A caller wanting a
299 // smaller tier should lower `min_block`, not smuggle it in here.
300 assert!(
301 extras[i] >= min_block,
302 "Params::extras: every entry must be >= min_block (min_block is the \
303 scheme's minimum block size)"
304 );
305 if i > 0 {
306 assert!(
307 extras[i] > extras[i - 1],
308 "Params::extras: must be strictly increasing"
309 );
310 }
311 i += 1;
312 }
313 }
314
315 let (num, den) = params.growth;
316
317 let mut out = [0usize; N];
318
319 // Merge the geometric run (generated lazily) with `extras` (already sorted)
320 // into one strictly-increasing `out`. Both sources are non-decreasing, so a
321 // classic two-pointer merge settles it without an intermediate buffer.
322 let mut gi = 0; // geometric index
323 let mut ei = 0; // extras index
324 let mut oi = 0; // output index
325 let mut cur = min_block; // current geometric value (valid while gi < geo_count)
326 while gi < geo_count || ei < extras.len() {
327 let take_geo = if gi >= geo_count {
328 false
329 } else if ei >= extras.len() {
330 true
331 } else {
332 cur < extras[ei]
333 };
334 if take_geo {
335 out[oi] = cur;
336 gi += 1;
337 // Advance the geometric value for the next iteration:
338 // next = round_up(ceil(cur * num / den), min_block), min step min_block.
339 if gi < geo_count {
340 // Widened to u128 so only the ACTUAL next class has to fit
341 // `usize`: guarding `cur * num` instead would reject schemes
342 // whose product overflows but whose quotient does not (e.g.
343 // min_block = 2^62, growth = (3, 3) at cur = 2^63).
344 //
345 // Checked at all because `cur` accumulates geometrically and
346 // this is a library: an unchecked wrap would be masked by the
347 // min-step fallback below into a valid-looking but silently
348 // wrong table, in release builds and release-profile const
349 // evaluation alike. `checked_mul`/`checked_add` (rather than a
350 // hand-proved "cannot overflow" comment) so the bound is
351 // enforced, not just argued -- for every `usize <= 64` target
352 // this crate supports today. A hypothetical 128-bit `usize`
353 // would need a genuinely overflow-free multiply/divide here
354 // (`cur * num` can then overflow `u128` even when the true
355 // quotient still fits `usize`); `checked_mul` alone would
356 // reproduce the same class of bug one width higher.
357 let scaled = match (cur as u128).checked_mul(num as u128) {
358 Some(p) => p.div_ceil(den as u128),
359 None => panic!("geometric progression: cur * num overflows u128"),
360 };
361 // Round up to a multiple of min_block, still in u128.
362 let rounded = match scaled.checked_add(mask as u128) {
363 Some(v) => v & !(mask as u128),
364 None => panic!("geometric progression: scaled + mask overflows u128"),
365 };
366 assert!(
367 rounded <= usize::MAX as u128,
368 "geometric progression overflows usize -- reduce geo_count/growth"
369 );
370 let mut next = rounded as usize;
371 if next <= cur {
372 // Checked for the same reason as the widened math above,
373 // and reached far more often: under `growth.0 == 0` this
374 // fallback is the ONLY advance path, so every step goes
375 // through it. Unchecked, a `min_block` near 2^62 wraps to
376 // a duplicate zero-sized class -- not even monotone.
377 next = cur
378 .checked_add(min_block)
379 .expect("geometric progression overflows usize -- reduce geo_count/growth");
380 }
381 cur = next;
382 }
383 } else {
384 out[oi] = extras[ei];
385 ei += 1;
386 }
387 oi += 1;
388 }
389
390 // The merged table is where an extra/geometric DUPLICATE first becomes
391 // visible: the per-entry checks above compare each extra only against
392 // `min_block` and the other extras, never against the run it is about to
393 // merge with. `min_block = 16, extras = [16, 32]` passes every check
394 // above yet duplicates the run's first two classes. Checked here so a
395 // standalone `build_table` caller gets the guarantee its rustdoc
396 // promises, rather than discovering it in `build_size2class` downstream.
397 {
398 let mut i = 1;
399 while i < N {
400 assert!(
401 out[i] > out[i - 1],
402 "build_table: merged table must be strictly increasing -- an \
403 extras entry duplicates a value the geometric run also \
404 produces (each extras entry is only checked against \
405 min_block-alignment and the other extras entries before \
406 merging; an extra landing strictly BETWEEN two geometric \
407 values is fine)"
408 );
409 i += 1;
410 }
411 }
412
413 out
414}
415
416/// Build the O(1) `size→class` lookup **from a table** at compile time — so the
417/// lookup and the table cannot drift. The caller indexes it as
418/// `size2class[(size - 1) >> log2(min_block)]`, so bucket `k` covers every size
419/// in `(k * min_block, (k + 1) * min_block]`; `size2class[k]` is the smallest
420/// class whose `block_size >= (k + 1) * min_block` -- EXCEPT the top bucket
421/// `L - 1`, whose ideal `need` (`L * min_block` mathematically -- NOT
422/// guaranteed to fit `usize` even for a valid scheme, e.g. `min_block =
423/// 1 << 62, L = 4`; the builder computes it as `(k + 1).checked_mul(min_block)`
424/// and folds that same overflow into the clamp below, never evaluating the
425/// unrepresentable product) exceeds `table[N - 1]` (the
426/// largest class), so no such class exists; that bucket is clamped to
427/// `table[N - 1]` itself instead. For [`SizeClasses::class_for`] specifically
428/// this is harmless: it never queries bucket `L - 1` for any in-range `size`
429/// (its own early-rejection guard catches every size that would land there),
430/// so THERE the clamped entry is an unreachable sentinel, not
431/// an observable answer. A caller driving this array directly (bypassing
432/// `class_for`) can still observe it -- and for a hand-built `table` whose
433/// `small_max` is not a multiple of `min_block`, bucket `L - 1` need not even
434/// be a sentinel: it can be the correct, reachable answer for sizes in
435/// `((L - 1) * min_block, small_max]`.
436///
437/// `L` must equal [`size2class_len`]`(max_class, min_block)`, where `max_class`
438/// is `table[N - 1]`.
439///
440/// `table` need not come from [`build_table`] -- this function is a
441/// standalone building block, callable with any hand-built strictly
442/// increasing array. Note, though, that [`build_table`]'s own output always
443/// has every entry a multiple of `min_block`; a hand-built `table` that
444/// violates that (while still passing every check below) can produce an
445/// entry the documented bucket lookup never selects -- e.g. `min_block =
446/// 16`, `table = [16, 24, 32]`: bucket `(16, 32]` resolves straight to `32`,
447/// leaving `24` monotonicity-valid but permanently unreachable through the
448/// public lookup path. (There is no public constructor that feeds a
449/// hand-built `table` into [`SizeClasses::class_for`] -- [`SizeClasses::build`]
450/// always derives its table from [`build_table`] -- so this is a property of
451/// the derived LUT itself, not of `class_for`.)
452///
453/// # Panics
454///
455/// Panics -- identically in `const` evaluation and at runtime, since this is
456/// a `pub const fn` callable either way -- if the table is empty, if `L` is
457/// wrong (including if computing the expected `L` via
458/// [`size2class_len`]`(table[N - 1], min_block)` itself overflows `usize`),
459/// if `min_block` is not a power of two, if `table.len() > 256` (entries are
460/// `u8` CLASS INDICES, so the largest representable table has 256 classes,
461/// indices `0..=255`; a 257th class would silently truncate), or if `table`
462/// is not strictly increasing.
463#[must_use]
464pub const fn build_size2class<const N: usize, const L: usize>(
465 table: &[usize; N],
466 min_block: usize,
467) -> [u8; L] {
468 assert!(N > 0, "table must be non-empty");
469 assert!(
470 min_block.is_power_of_two(),
471 "min_block must be a power of two"
472 );
473 // Entries are `u8` class INDICES, so the bound is on the largest index
474 // (`N - 1`), not on the count: `N == 256` yields indices `0..=255`, all
475 // representable. `class_idx` never reaches `N` (the `need` clamp below
476 // guarantees `table[N - 1] >= need`, so the inner scan always breaks),
477 // so 256 classes cannot truncate; 257 would.
478 assert!(
479 N <= u8::MAX as usize + 1,
480 "size2class entries are u8 class indices; the class count must not exceed 256"
481 );
482 // Global monotonicity of `table` — the monotone-pointer algorithm below
483 // *depends* on it. `build_table` already rejects a `Params::extras`
484 // overlap with the geometric run at its own chokepoint, so a table
485 // reaching this check via `SizeClasses::build` is already known-good;
486 // this stays as defense-in-depth for a hand-built table that bypasses
487 // `build_table` entirely (e.g. a `const` array literal) — an overlap
488 // there collapses two table slots to equal values, which is a
489 // duplicate, not a strict increase, and would otherwise leave the
490 // colliding slot silently unreachable.
491 {
492 let mut i = 1;
493 while i < N {
494 assert!(
495 table[i] > table[i - 1],
496 "table must be strictly increasing (hand-built tables must \
497 satisfy this directly -- Params-driven tables already do, \
498 via build_table's own check)"
499 );
500 i += 1;
501 }
502 }
503 let small_max = table[N - 1];
504 // Reuse `size2class_len` rather than re-deriving its formula: a second
505 // copy is how one of the two silently drifts out of sync (and loses the
506 // overflow check).
507 assert!(
508 L == size2class_len(small_max, min_block),
509 "L must equal size2class_len(max_class, min_block)"
510 );
511 let mut out = [0u8; L];
512 let mut k = 0;
513 // `class_idx` persists across `k` (monotone-pointer): both `need` and the
514 // table are non-decreasing, so the answer for an earlier bucket is a valid
515 // start for the next — O(buckets + classes) total.
516 let mut class_idx = 0;
517 while k < L {
518 // The largest size mapping to bucket k via (size-1)>>shift is
519 // (k+1)*min_block. Clamp to small_max so the top bucket (only ever
520 // indexed by a size > small_max, which `class_for` rejects first) stays
521 // in-range and resolves to the last class (a harmless sentinel).
522 //
523 // `checked_mul` folds overflow into that same clamp: if the product
524 // does not fit `usize`, its true value certainly exceeds `small_max`
525 // (which does fit), so `small_max` is exactly the answer an
526 // unwrapped multiply would have given.
527 let need = match (k + 1).checked_mul(min_block) {
528 Some(v) if v < small_max => v,
529 _ => small_max,
530 };
531 while class_idx < N {
532 if table[class_idx] >= need {
533 break;
534 }
535 class_idx += 1;
536 }
537 out[k] = class_idx as u8;
538 k += 1;
539 }
540 out
541}
542
543/// A const-built size-class scheme: the sorted class table, its derived O(1)
544/// `size→class` lookup, and the policy constants needed to classify a request.
545///
546/// - `N` — the number of classes (`geo_count + extras.len()`).
547/// - `L` — the `size2class` length ([`size2class_len`]`(max_class, min_block)`).
548///
549/// Construct one at compile time with [`SizeClasses::build`]. All query methods
550/// are `const` pure arithmetic — no allocation, and no panics on the lookup
551/// path FOR IN-CONTRACT INPUTS: `need = max(size, align) >= 1` (so `size ==
552/// 0` alone is fine whenever `align >= 1` -- see
553/// [`class_for`](Self::class_for)'s own doc for the precise domain), a
554/// power-of-two `align`, and an `idx` obtained from
555/// [`class_for`](Self::class_for) rather than picked independently — an
556/// out-of-range `idx` does panic, see [`block_size`](Self::block_size).
557///
558/// Deliberately not `Copy`: duplicating a realistic scheme is ~16 KiB (see
559/// [`size2class_len`]'s `# Memory cost` for the breakdown), so call
560/// `.clone()` explicitly. Intended use is a `static` referenced in place;
561/// no method needs ownership. (Design rationale: the CHANGELOG.)
562///
563/// `Debug` prints a short summary, not the raw tables -- inspect those with
564/// [`table`](Self::table) / [`size2class`](Self::size2class).
565#[derive(Clone)]
566pub struct SizeClasses<const N: usize, const L: usize> {
567 // LAYOUT NOTE (default `repr(Rust)`): `class_for`'s fast path hot-loads
568 // `min_block_shift` and the LOW end of `size2class` (small sizes index the
569 // first buckets), and the default field-reordering heuristic
570 // (largest-align-first) happens to place the align-4 `min_block_shift`
571 // immediately before the align-1 `size2class`, so both loads share a cache
572 // line. That adjacency follows from the heuristic, NOT from the declaration
573 // order below; if `#[repr(C)]` is ever added here, declaration order becomes
574 // memory order, so the fields would have to be reordered to
575 // `min_block_shift, size2class, table, huge_threshold` to preserve it.
576 table: [usize; N],
577 size2class: [u8; L],
578 // `min_block`, `small_align_max`, and `1 << min_block_shift` are the same
579 // value by construction (see `build` below) -- storing only the shift
580 // removes one hot-path field load (`small_align_max`, read once in
581 // `class_for`'s fast-path check) at the cost of re-deriving `1 << shift`
582 // there instead; `min_block` was already accessor-only (never read
583 // directly in `class_for`). `min_block()`/`small_align_max()` re-derive it.
584 min_block_shift: u32,
585 huge_threshold: usize,
586}
587
588/// The error [`SizeClasses::try_class_for`] returns when `align` is not a
589/// power of two (the [`core::alloc::Layout`] contract
590/// [`SizeClasses::class_for`] assumes but -- on its own hot path -- only
591/// `debug_assert!`s). Carries the offending value for diagnostics.
592///
593/// A plain tuple struct, not `#[non_exhaustive]`: match the offending value
594/// directly as `Err(InvalidAlign(n))`.
595#[derive(Debug, Clone, Copy, PartialEq, Eq)]
596pub struct InvalidAlign(pub usize);
597
598impl core::fmt::Display for InvalidAlign {
599 fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
600 write!(
601 f,
602 "align ({}) must be a power of two (the Layout contract)",
603 self.0
604 )
605 }
606}
607
608impl core::error::Error for InvalidAlign {}
609
610impl<const N: usize, const L: usize> core::fmt::Debug for SizeClasses<N, L> {
611 fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
612 f.debug_struct("SizeClasses")
613 .field("N", &N)
614 .field("L", &L)
615 .field("min_block", &self.min_block())
616 .field("small_max", &self.small_max())
617 .field("huge_threshold", &self.huge_threshold)
618 .finish_non_exhaustive()
619 }
620}
621
622impl<const N: usize, const L: usize> SizeClasses<N, L> {
623 /// Build a scheme from [`Params`] at compile time. `N` and `L` must match
624 /// the params (see [`build_table`] / [`build_size2class`] for the exact
625 /// obligations); a mismatch panics identically in `const` evaluation
626 /// (compile error) and at runtime.
627 ///
628 /// Intended placement is a `static`, not a `const`: both const-evaluate
629 /// this for free, but a `const` item re-materializes its value at every
630 /// use site, duplicating the embedded tables, while a `static` is one
631 /// fixed-address copy referenced in place. Nothing stops calling it at
632 /// *runtime* instead of either: doing so materializes the whole return
633 /// value -- at least `L` bytes, several KiB for a realistic scheme -- by
634 /// value on the caller's stack, which matters on a small-stack `no_std`
635 /// target.
636 ///
637 /// `small_align_max` — the alignment ceiling of the O(1) fast path — is set
638 /// to `min_block`: every class size is a multiple of `min_block`, so the
639 /// stride trivially satisfies divisibility for any `align <= min_block`
640 /// (see [`class_for`](Self::class_for)'s `# Preconditions` for the
641 /// separate base-address requirement). Larger alignments take the
642 /// divisibility-jump slow path in
643 /// [`class_for`](Self::class_for).
644 #[must_use]
645 pub const fn build(params: Params) -> Self {
646 let table = build_table::<N>(params);
647 let size2class = build_size2class::<N, L>(&table, params.min_block);
648 let small_max = table[N - 1];
649 // `build_table` already guarantees every table entry is a multiple
650 // of `min_block` -- this cannot fail through the public API. Kept as
651 // a cheap internal sanity check because `class_for`'s index-space
652 // guard (see its own comment) depends on this equality holding.
653 debug_assert!(
654 small_max.is_multiple_of(params.min_block),
655 "SizeClasses::build: small_max must be a multiple of min_block"
656 );
657 Self {
658 table,
659 size2class,
660 min_block_shift: params.min_block.trailing_zeros(),
661 huge_threshold: params.huge_threshold,
662 }
663 }
664
665 /// The class table (strictly increasing, each entry a multiple of
666 /// `min_block`). The single source of truth for the scheme's geometry.
667 #[must_use]
668 #[inline]
669 pub const fn table(&self) -> &[usize; N] {
670 &self.table
671 }
672
673 /// The derived O(1) `size→class` lookup, as built by [`build_size2class`]
674 /// -- see that function's doc for the indexing formula and the `L - 1`
675 /// top-bucket clamp.
676 ///
677 /// LOW-LEVEL: unlike [`class_for`](Self::class_for), this accessor does
678 /// not itself validate a raw caller's `size`. The documented formula is
679 /// `size2class()[(size - 1) >> min_block_shift()]`, which has TWO
680 /// preconditions this array does not enforce:
681 ///
682 /// - **`size >= 1`** — `size - 1` underflows for `size == 0`; guard it
683 /// with `size.checked_sub(1)` if `size` may be `0`.
684 /// - **`size <= small_max()`** for a genuine classification. Do NOT
685 /// derive this bound as a byte size (`L * min_block()` is NOT
686 /// guaranteed to fit `usize` for every valid scheme). Compare `size` to
687 /// [`small_max`](Self::small_max) directly, or reason about the INDEX
688 /// instead — beyond `small_max()` the raw index is NOT uniformly
689 /// clamped: `idx == L - 1` is in-bounds and returns the clamped
690 /// sentinel (a false "fits" instead of the `None`
691 /// [`class_for`](Self::class_for) would give), while `idx >= L` is
692 /// genuinely out-of-bounds and panics.
693 ///
694 /// [`class_for`](Self::class_for) avoids both pitfalls (its `need =
695 /// max(size, align)` is always `>= 1`, and it rejects a too-large `need`
696 /// before indexing) and additionally applies the `align` predicate this
697 /// raw LUT ignores. Prefer it unless you specifically need the raw LUT.
698 ///
699 /// **This shape is a deliberate, but not permanently promised, choice.**
700 /// The LUT is today one flat `u8` per `min_block`-sized bucket over the
701 /// *whole* size range (see [`size2class_len`]'s `# Memory cost`) — the
702 /// simplest shape that stays O(1) for arbitrary `extras`, but a
703 /// memory-hungry one for a scheme with a large `max_class`. `L` being a
704 /// public const generic means a future layout change (e.g. a hybrid:
705 /// an exact small-size LUT below some threshold, a computed answer
706 /// above it) would very likely require a breaking release regardless.
707 #[must_use]
708 #[inline]
709 pub const fn size2class(&self) -> &[u8; L] {
710 &self.size2class
711 }
712
713 /// The minimum block size / fundamental alignment (`min_block`).
714 /// Derived from [`min_block_shift`](Self::min_block_shift) (`1 <<
715 /// min_block_shift`) rather than stored separately -- the two are equal
716 /// by construction (see [`build`](Self::build)).
717 #[must_use]
718 #[inline]
719 pub const fn min_block(&self) -> usize {
720 1usize << self.min_block_shift
721 }
722
723 /// `log2(min_block)` — the shift turning a byte size into a
724 /// `min_block`-unit index.
725 #[must_use]
726 #[inline]
727 pub const fn min_block_shift(&self) -> u32 {
728 self.min_block_shift
729 }
730
731 /// The alignment ceiling of the O(1) fast path (equal to `min_block`) --
732 /// not the ceiling on alignments [`class_for`](Self::class_for) can
733 /// serve at all; larger alignments take its slow path instead.
734 #[must_use]
735 #[inline]
736 pub const fn small_align_max(&self) -> usize {
737 1usize << self.min_block_shift
738 }
739
740 /// The largest class (`table[N - 1]`). A request larger than this — or with
741 /// an alignment larger than this — takes the caller's large path.
742 #[must_use]
743 #[inline]
744 pub const fn small_max(&self) -> usize {
745 self.table[N - 1]
746 }
747
748 /// The number of classes (`N`).
749 #[must_use]
750 #[inline]
751 pub const fn count(&self) -> usize {
752 N
753 }
754
755 /// The block size of class `idx`.
756 ///
757 /// # Panics
758 ///
759 /// Panics if `idx >= N` — the caller only ever passes indices returned by
760 /// [`class_for`](Self::class_for).
761 #[must_use]
762 #[inline]
763 pub const fn block_size(&self, idx: usize) -> usize {
764 self.table[idx]
765 }
766
767 /// The caller's [`Params::huge_threshold`] policy value, as built. The
768 /// only `Params` field with a dedicated read-back accessor here, for a
769 /// caller that needs to report or log the threshold without keeping its
770 /// own separate copy of it.
771 #[must_use]
772 #[inline]
773 pub const fn huge_threshold(&self) -> usize {
774 self.huge_threshold
775 }
776
777 /// Whether a `size` request is "huge" per the caller's
778 /// [`Params::huge_threshold`] policy.
779 #[must_use]
780 #[inline]
781 pub const fn is_huge(&self, size: usize) -> bool {
782 size >= self.huge_threshold
783 }
784
785 /// Resolve `(size, align)` to a class index, or `None` for the caller's
786 /// large path.
787 ///
788 /// A class fits iff its `block_size >= max(size, align)` AND
789 /// `block_size % align == 0`. Returns the index of the smallest such class.
790 ///
791 /// The divisibility conjunct is a STRIDE property, not an address
792 /// guarantee — see `# Preconditions` below for what it does and does not
793 /// establish about block addresses.
794 ///
795 /// **Fast path (`align <= min_block`):** every class SIZE is a multiple of
796 /// `min_block`, which does two things: the stride divisibility check is
797 /// trivially satisfied, **and** the LUT's bucket-top answer is the
798 /// smallest *fitting* class (no class value can lie strictly between
799 /// `need` and its bucket's top) — one O(1) lookup (same base-alignment
800 /// precondition as the slow path, below).
801 ///
802 /// **Slow path (`align > min_block`, a power of two):** seed at the lookup
803 /// entry covering `max(size, align)`, then jump forward over non-divisible
804 /// classes — from a non-divisible class of block size `b`, the next class
805 /// that could be a multiple of `align` is the one covering the smallest
806 /// multiple of `align` strictly greater than `b` (a bitmask round-up plus
807 /// one lookup). Provably equivalent to a step-by-1 walk, never more
808 /// iterations, fewer whenever the jump skips at least one class.
809 ///
810 /// The useful domain is `size >= 1`; more precisely, what must hold is
811 /// `need = max(size, align) >= 1`, so `size == 0` alone is fine whenever
812 /// `align >= 1` -- `(need - 1) >> shift` never underflows in that case.
813 /// Consumers commonly clamp `size` up to `min_block` before calling (the
814 /// classifier has no smaller class to offer below it anyway); this
815 /// function does not require that clamp.
816 ///
817 /// # Preconditions
818 ///
819 /// `align` **must be a power of two** — the same `Layout` contract the
820 /// standard allocator API requires. An `align` taken from
821 /// [`core::alloc::Layout`] satisfies this by construction; one computed
822 /// by hand may not.
823 ///
824 /// A violation trips a `debug_assert!` whenever `cfg(debug_assertions)` is
825 /// on (both this and the `overflow-checks` knob below track the profile
826 /// `size-classes` itself is compiled with, which normally tracks the
827 /// consumer's). With `debug_assertions` off, the behavior for ANY
828 /// non-power-of-two `align` -- including `0` -- is UNSPECIFIED: an
829 /// incorrect `Some`/`None` (the fast path skips the divisibility check
830 /// entirely; the slow path's bitmask round-up and its
831 /// `block & (align - 1) == 0` test both assume a
832 /// power of two and can overshoot, under-return, or wrongly accept a
833 /// non-fitting class), never memory unsafety or a corrupt table. The
834 /// one non-power-of-two input with a SPECIFIED outcome is the
835 /// `align == 0, size == 0` corner, which does NOT panic, but only with
836 /// `overflow-checks` ALSO off (a separate Cargo knob from
837 /// `debug_assertions`): `need - 1` underflows to `usize::MAX`, landing on
838 /// the same early `None` any other out-of-range request takes (see
839 /// `class_for`'s own index-space guard comment for the proof); with
840 /// `overflow-checks` on, that subtraction panics instead. Prefer
841 /// [`try_class_for`](Self::try_class_for), which rejects `align == 0`
842 /// before any of this arithmetic runs in every profile, over relying on
843 /// this fallback behavior.
844 ///
845 /// **The carve base must also be `align`-aligned.** For blocks carved at
846 /// `base + k * block_size`, `block_size % align == 0` gives `address(k) %
847 /// align == base % align` for every `k`: the stride PRESERVES whatever
848 /// alignment the carve base already has (so no per-block padding is ever
849 /// needed) — it cannot CREATE alignment the base lacks. This crate
850 /// computes over sizes only and never sees an address, so it CANNOT
851 /// check this — unlike the power-of-two contract above, it is not even
852 /// `debug_assert`-able here. The caller must place block `0` of the run
853 /// serving a returned class at an address `base` with `base % align ==
854 /// 0` for every `align` it resolves through this scheme (the address
855 /// that matters is block `0`'s, not the span's OS reservation base, if
856 /// the two differ). Carving every run from a base whose power-of-two
857 /// alignment is `>=` the largest `align` the scheme will ever serve
858 /// satisfies this for every smaller `align` too.
859 ///
860 /// A violation cannot corrupt this crate's own scheme or cause UB INSIDE
861 /// IT (pure arithmetic over sizes, no addresses touched) — it yields
862 /// blocks whose SIZE is `align`-divisible but whose ADDRESSES are all
863 /// congruent to the same `base % align != 0`. But an allocator built on
864 /// top of this crate that returns such a misaligned pointer for a
865 /// request with that `align` violates ITS OWN `Layout` contract with its
866 /// caller — the downstream consequence is safety-critical even though
867 /// this crate cannot detect or cause it directly.
868 #[must_use]
869 #[inline]
870 pub const fn class_for(&self, size: usize, align: usize) -> Option<usize> {
871 debug_assert!(
872 align.is_power_of_two(),
873 "class_for: align must be a power of two (the Layout contract)"
874 );
875 let need = if size > align { size } else { align };
876 // Index-space guard, not `need > self.small_max()`: `small_max()` is
877 // always `(L - 1) * min_block` (every
878 // `build_table` entry is a `min_block` multiple -- see `build`'s own
879 // invariant assert above), so `seed_idx >= L - 1 <=> need >
880 // small_max()` exactly, and `L - 1` is a compile-time constant where
881 // `small_max()` reads `self.table[N - 1]` at runtime. This lets the
882 // compiler prove `seed_idx < L` and drop the bounds check
883 // `self.size2class[seed_idx]` would otherwise need, instead of
884 // paying that check on top of the one just performed here.
885 let seed_idx = (need - 1) >> self.min_block_shift;
886 if seed_idx >= L - 1 {
887 return None;
888 }
889 let seed = self.size2class[seed_idx] as usize;
890 if align <= (1usize << self.min_block_shift) {
891 return Some(seed);
892 }
893 // Slow path: `align > small_align_max` is a power of two (the `Layout`
894 // contract). Walk forward, JUMPING over non-divisible classes via the
895 // lookup rather than stepping one class at a time.
896 //
897 // Termination: the smallest multiple of `align` strictly greater
898 // than `block` is itself `> block`, so the looked-up class index is
899 // strictly greater than `i` (the table is strictly increasing), so
900 // `i` advances every iteration.
901 let mut i = seed;
902 while i < N {
903 let block = self.table[i];
904 // `align` is a power of two here, so the mask is a
905 // division-free `is_multiple_of`.
906 if block & (align - 1) == 0 {
907 return Some(i);
908 }
909 // `block | (align - 1)` is one below the smallest multiple of
910 // `align` strictly greater than `block` (align is a power of
911 // two, so that's what `+ 1` would round up to) -- exactly the
912 // value the bucket index below wants, so there is no reason to
913 // add 1 and then immediately subtract it again. Same
914 // index-space guard as the seed above; if no next multiple
915 // exists (`block | (align - 1) == usize::MAX`), the shifted
916 // index is `usize::MAX >> shift >= small_max >> shift == L - 1`
917 // (the same identity the seed guard relies on), so the guard
918 // below returns `None` on its own -- no separate overflow check
919 // needed.
920 let next_idx = (block | (align - 1)) >> self.min_block_shift;
921 if next_idx >= L - 1 {
922 return None;
923 }
924 i = self.size2class[next_idx] as usize;
925 }
926 // Unreachable in practice: `self.size2class[..] <= N - 1` always (see
927 // `build_size2class`'s own invariant), so the loop always returns
928 // from inside the body at `i == N - 1` at the latest. Still needed
929 // as the type-level fallthrough -- and `while i < N` is what lets
930 // the compiler elide the `self.table[i]` bounds check above.
931 None
932 }
933
934 /// The checked twin of [`class_for`](Self::class_for): validates `align`
935 /// instead of assuming it (`Err(`[`InvalidAlign`]`)` for a non-power-of-two
936 /// `align`, including `0`), then delegates. Same result on every
937 /// already-valid input; the only behavior difference is on the inputs
938 /// `class_for`'s own `# Preconditions` already document as
939 /// contract-violating. Does strictly more work than `class_for` (the
940 /// added power-of-two check).
941 ///
942 /// **Never panics, for any `(size, align)` pair** — this is the
943 /// substantive reason to prefer it over `class_for` for an `align` that
944 /// is not already known-valid: a non-power-of-two `align` (including
945 /// `0`) is rejected before any arithmetic runs, so `need = max(size,
946 /// align)` is always `>= 1` past that point; the seed index
947 /// `(need - 1) >> min_block_shift` is compared against the compile-time
948 /// bound `L - 1` **before** any indexing, so both the seed and every
949 /// slow-path re-seed stay strictly inside `size2class()`, and the
950 /// slow-path jump loop is bounded exactly as `class_for`'s own doc
951 /// proves.
952 ///
953 /// Use this one unless `align` is already known-valid by construction
954 /// (e.g. taken directly from a [`core::alloc::Layout`]) -- `class_for`
955 /// stays the zero-validation hot-path variant for that case, matching
956 /// [`Layout::from_size_align`](core::alloc::Layout::from_size_align) (checked) versus
957 /// [`Layout::from_size_align_unchecked`](core::alloc::Layout::from_size_align_unchecked) (trusted) in `core::alloc`.
958 #[must_use = "this returns a Result, not just a class index -- the Err case must be handled"]
959 #[inline]
960 pub const fn try_class_for(
961 &self,
962 size: usize,
963 align: usize,
964 ) -> Result<Option<usize>, InvalidAlign> {
965 if !align.is_power_of_two() {
966 return Err(InvalidAlign(align));
967 }
968 Ok(self.class_for(size, align))
969 }
970}