frust_engine/cache/images.rs
1//! Image residency: which decoded images own atlas space, and for how long.
2//!
3//! A widget hands the display list the *same* decoded image every frame — the
4//! `peniko::ImageData` in a [`Command::Image`](frust_scene::Command::Image) is a
5//! reference-counted handle cloned per frame, never a re-decode — so the pixels
6//! behind it are stable across frames and belong on the GPU once rather than
7//! per draw. [`ImageResidency`] is what makes that "once" true: it keys an
8//! image on its blob's process-unique id ([`peniko::Blob::id`]) and hands back
9//! the same [`ImageId`] and the same atlas rectangle for every later frame that
10//! draws it.
11//!
12//! Allocation itself is pure — a rectangle packer over
13//! [`vello_common::multi_atlas::MultiAtlasManager`], no device in the loop — so
14//! everything here is host-testable. The two GPU-side halves it *describes*
15//! (writing a newly allocated region's texels, clearing an evicted one's) are
16//! carried out by [`crate::gpu::atlas`] against the plan this module produces:
17//! [`ImageResidency::evictions`] first, [`ImageResidency::uploads`] second. That
18//! order is a contract, not a preference — a rectangle freed this frame can be
19//! re-allocated in the same frame, so clearing after uploading would erase the
20//! image that just moved in.
21//!
22//! ## Residency is committed only once the plan is serviced
23//!
24//! The cache decides residency long before anything reaches a queue: a
25//! compiled frame can still be refused afterwards (a layer shape the scheduler
26//! will not serve, a page past the pool's ceiling, coverage or paints outgrowing
27//! their textures), and a frame refused after compiling is never drawn and never
28//! uploaded. So the plan is *pending* rather than taken: [`ImageResidency::plan`]
29//! hands out a copy, [`begin_frame`](ImageResidency::begin_frame) leaves it
30//! alone, and only [`acknowledge_plan`](ImageResidency::acknowledge_plan) —
31//! called by a consumer that has actually written the regions into a live atlas
32//! — clears it.
33//!
34//! That is what makes the entry map and the atlas's real contents agree. Without
35//! it, a refused frame leaves an image *recorded as resident* whose texels were
36//! never written: every later frame resolves the entry, schedules no upload
37//! (residency is a hit), and either drops the draw for the life of the process
38//! or samples a rectangle nothing ever wrote. Re-emitting the same plan until it
39//! is acknowledged costs a vector copy per frame in the steady state — where the
40//! plan is empty — and is what makes "an image drawn on a thousand frames
41//! uploads once" true of the *atlas* rather than only of the cache.
42//!
43//! ## An image larger than the atlas is minified, not dropped
44//!
45//! A source whose own extent will not fit one atlas layer is downsampled to fit
46//! ([`fit_extent`], [`minify`]) instead of being refused. A photograph decoded
47//! at its capture resolution is routinely several times the atlas budget while
48//! its destination rectangle is a thumbnail, so refusing it drops a draw the
49//! display list plainly describes; and every texel past the destination's own
50//! scale is one the sampler would have thrown away anyway.
51//!
52//! The filter is a plain box average over the premultiplied texels, hand-written
53//! rather than pulled in: the workspace's version pins are law, an image decoder
54//! is not a dependency this crate carries, and a box average is exactly right
55//! for the minification direction (every output texel is the mean of the source
56//! texels it covers). The aspect ratio is preserved, so the paint transform's
57//! two axes stay in step; what changes is the resident rectangle's extent, which
58//! [`ResidentImage::natural`] records alongside it so the consumer can rescale
59//! the encoded paint's own transform into the smaller rectangle.
60//!
61//! ## Two things are refused rather than attempted
62//!
63//! 1. **A format or size `vello_common` panics on.**
64//! [`vello_common::paint::ImageSource::from_peniko_image_data`] asserts on a
65//! dimension past `u16::MAX` and `unimplemented!()`s on a format outside
66//! `Rgba8`/`Bgra8`, and a pixel buffer whose length disagrees with the
67//! declared extent trips [`Pixmap::from_parts_with_opacity`]'s own assertion.
68//! All three are checked here, *before* the call, so an oversized or
69//! malformed image is an [`ImageSkip`] the frame path reports rather than a
70//! panic it takes (E17). These, plus an atlas with no room left in it, are
71//! now the only things [`ImageResidency::skipped`] counts: an image that is
72//! merely *big* is minified and drawn.
73//! 2. **Anything at all, when `FRUST_ENGINE_NO_ATLAS` is set.** The kill switch
74//! ([`crate::config::atlas_disabled`]) makes every resolution answer
75//! [`ImageSkip::AtlasDisabled`], so no atlas is allocated, nothing is
76//! uploaded, and image draws fall back to painting nothing — the switch's
77//! whole point being to take the atlas out of a frame under diagnosis.
78//!
79//! ## Residency is bounded by age, and by pressure
80//!
81//! An entry unseen for [`MAX_UNSEEN_FRAMES`] consecutive frames is deallocated
82//! and its rectangle reported for clearing — the same age-based reap
83//! `frust-render`'s shader-effect cache uses, and for the same reason: a screen
84//! that stops drawing an image should give its texels back promptly, while an
85//! image drawn every other frame must never be mistaken for gone.
86//!
87//! Age alone is not a bound on *population*, though: it says nothing about how
88//! many distinct images a screen draws inside one age window. A list scrolling
89//! faster than [`MAX_UNSEEN_FRAMES`] streams more of them past the viewport
90//! than the atlas has rectangles for, and every one past the last free
91//! rectangle used to be an [`ImageSkip::NoAtlasSpace`] — a draw that painted
92//! nothing, on a screen whose solid fills all still landed, which reads as a
93//! hole rather than as a cache miss. So an allocation the packer refuses now
94//! *evicts* instead of giving up: [`ImageResidency::resolve`] works out which
95//! least-recently-seen rectangles would make room, frees exactly those, and
96//! each one whose rectangle was taken re-uploads on the next frame that draws
97//! it. A working set larger than the atlas therefore degrades to re-uploads —
98//! bandwidth — rather than to missing content, and
99//! [`ImageResidency::pressure_evictions`] counts how often that trade was made.
100//!
101//! ## An eviction is planned on paper before anything is freed
102//!
103//! An eviction that does not produce the allocation is pure loss. The draw is
104//! skipped anyway, so the frame pays a clear per freed rectangle *and* the
105//! re-upload of every displaced image *and* the hole it was trying to avoid.
106//! And the condition recurs: a request the atlas can never satisfy would strip
107//! and rebuild the residency on every frame that asks for it, which on a mobile
108//! budget is megabytes of texture traffic each way, per frame, buying nothing.
109//!
110//! So the pressure path commits nothing speculatively. `allocate_under_pressure`
111//! walks the candidates in least-recently-seen order over a *model* of the
112//! layers — the rectangles this residency knows are occupied, and each layer's
113//! free area as the packer itself reports it — and stops at the first prefix
114//! that would admit the request. Only then is anything released, and only the
115//! candidates on the layer the fit was found in. A request no bounded set of
116//! candidates can satisfy therefore stays the cheap [`ImageSkip::NoAtlasSpace`]
117//! it was before eviction existed, with the residency untouched.
118//!
119//! The model is deliberately *optimistic*: it can see this residency's own
120//! rectangles but not the glyph pages sharing the same allocator (see below),
121//! so it never refuses a request eviction could have satisfied on a given
122//! layer — a placement that clears the real occupants clears the model's
123//! subset of them too. The reverse can happen: the packer may still refuse a
124//! plan the model approved, which costs the planned evictions and no
125//! allocation. That case is bounded by the same plan budget rather than
126//! open-ended, and it is the one residual the planner cannot rule out without
127//! knowing where every glyph page sits.
128//!
129//! The bounded search itself is scoped *per layer*: each atlas layer gets its
130//! own candidate-count and area allowance rather than the whole residency
131//! sharing one pool, so a request only one layer could ever satisfy is not
132//! starved by candidates on layers that were never going to host it — each
133//! layer is either cleared within its own allowance or ruled out on its own
134//! terms, never on account of what another layer's candidates already cost.
135//!
136//! The one entry never freed on either route is one *this frame* has already
137//! resolved. The plan clears before it uploads, so handing back a rectangle the
138//! frame is about to sample would erase exactly the image the eviction was
139//! meant to make room for. A working set that outgrows the atlas inside a
140//! single frame is therefore still a skip, and honestly so: nothing is
141//! evictable. The same guard covers the *extent-mismatch* release in
142//! [`ImageResidency::resolve`], where a blob redrawn at a new extent gives its
143//! old rectangle back: on a frame that already sampled that rectangle the
144//! second draw is skipped instead, as
145//! [`ImageSkip::SameFrameExtentConflict`] rather than as a capacity refusal —
146//! the atlas may hold nothing at all beside the first extent's own rectangle.
147//!
148//! ## One allocator for images *and* glyphs
149//!
150//! The [`ImageCache`] this module holds is the process's only atlas allocator:
151//! the glyph policy (`crate::text::atlas_policy`) owns no cache of its own and
152//! allocates its slots through [`ImageResidency::allocator_mut`].
153//!
154//! That is a correctness requirement, not tidiness. An [`ImageId`] is a *slot
155//! index into one cache* — `ImageCache::allocate` hands out the next free index
156//! of its own `slots` vector — and the strip shader has exactly one atlas
157//! texture array to sample from. Two caches over the same page geometry would
158//! therefore mint the same small integers for unrelated occupants and pack them
159//! into overlapping rectangles of the same layers, so a glyph and an image would
160//! address each other's texels by construction. Sharing one cache makes both
161//! collisions unrepresentable: one id space, one packer, one set of live
162//! rectangles.
163//!
164//! It is also what upstream does — `vello_hybrid`'s `Resources` holds a single
165//! `image_cache` and hands it to its glyph atlas at every call — and it is what
166//! keeps the budget honest: a glyph page and an image page come out of the same
167//! [`AtlasBudget::max_atlases`] allowance, and [`ImageResidency::layers`] counts
168//! whichever of the two created a layer.
169//!
170//! The one rule a borrower must keep is ownership of its own handles: each side
171//! deallocates only the ids it allocated. Nothing enforces that beyond the two
172//! call sites, and it is the reason [`ImageResidency::allocator_mut`] is
173//! documented as a contract rather than a plain accessor.
174
175use std::collections::{HashMap, HashSet};
176use std::sync::Arc;
177#[cfg(feature = "perf-trace")]
178use std::sync::atomic::{AtomicBool, Ordering};
179
180use frust_gpu::{DownlevelProfile, TierCaps};
181use peniko::color::PremulRgba8;
182use peniko::{ImageData, ImageFormat};
183use thiserror::Error;
184use vello_common::image_cache::ImageCache;
185use vello_common::multi_atlas::{AllocationStrategy, AtlasConfig};
186use vello_common::paint::{ImageId, ImageSource};
187use vello_common::pixmap::Pixmap;
188
189use crate::config;
190
191/// Consecutive frames an image may go undrawn before its atlas rectangle is
192/// reclaimed.
193///
194/// 60 frames is half a second at 120Hz and a whole one at 60Hz: long enough
195/// that an image drawn intermittently (an every-other-frame animation, a brief
196/// scene-diff hiccup) never loses its residency, short enough that navigating
197/// away from an image-heavy screen returns its texels within a frame budget's
198/// worth of frames rather than at surface teardown.
199pub const MAX_UNSEEN_FRAMES: u64 = 60;
200
201/// Transparent padding pixels placed around each image in the atlas.
202///
203/// Zero, deliberately. Padding exists to stop a filtered sample from reading a
204/// neighbour's texels, and the strip shader's atlas samplers cannot do that:
205/// both the bilinear and the bicubic path clamp every tap into
206/// `[offset, offset + size - 1]` — the image's own rectangle — before it
207/// reaches `textureLoad`. Paying a two-pixel border per image would buy nothing
208/// and cost atlas space that a mobile budget does not have.
209pub const ATLAS_PADDING: u16 = 0;
210
211// The clear skip in `uncovered_clears` compares reported (padded) eviction
212// rectangles with raw upload rectangles; a non-zero padding would make them
213// differ and silently turn the skip off, or worse, mis-pair rectangles.
214const _: () = assert!(
215 ATLAS_PADDING == 0,
216 "uncovered_clears matches padded evictions against raw uploads, which only holds with zero padding"
217);
218
219/// The largest image edge that can be made resident at all.
220///
221/// `vello_common`'s image cache addresses an offset and an extent in `u16`, and
222/// its `ImageSource` conversion asserts on anything larger, so this is the
223/// ceiling the pre-check enforces rather than a policy of ours.
224pub const MAX_IMAGE_DIMENSION: u32 = u16::MAX as u32;
225
226/// The most atlas layers the shader's encoded-image record can name.
227///
228/// `GpuEncodedImage::image_params` gives `atlas_index` eight bits, so a layer
229/// past 255 could not be addressed even if the adapter allowed it.
230pub const MAX_ATLAS_LAYERS: usize = 256;
231
232/// Why one image could not be made resident.
233///
234/// Every variant is a *skip*, never a frame failure: the draw paints nothing
235/// and the rest of the frame proceeds. They are distinguished because the
236/// remedies differ — a too-large image is an application decision, an exhausted
237/// atlas is a budget one, and a disabled atlas is the operator's own doing.
238#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
239pub enum ImageSkip {
240 /// `FRUST_ENGINE_NO_ATLAS` is set; the atlas is out of the frame entirely.
241 #[error("image residency is disabled by FRUST_ENGINE_NO_ATLAS")]
242 AtlasDisabled,
243 /// The image declares a format the atlas does not store.
244 #[error("unsupported image format {0:?}: only Rgba8 and Bgra8 are uploaded")]
245 UnsupportedFormat(ImageFormat),
246 /// The image has no area, so there is nothing to make resident.
247 #[error("degenerate image extent {width}x{height}")]
248 DegenerateExtent {
249 /// Declared width in pixels.
250 width: u32,
251 /// Declared height in pixels.
252 height: u32,
253 },
254 /// The image is larger than [`MAX_IMAGE_DIMENSION`] on at least one axis.
255 #[error("image {width}x{height} exceeds the {max}-pixel residency ceiling")]
256 TooLarge {
257 /// Declared width in pixels.
258 width: u32,
259 /// Declared height in pixels.
260 height: u32,
261 /// The per-axis ceiling that refused it.
262 max: u32,
263 },
264 /// The pixel buffer's length disagrees with the declared extent and format.
265 #[error("image pixel buffer is {actual} bytes; {width}x{height} needs exactly {expected}")]
266 MalformedPixels {
267 /// Declared width in pixels.
268 width: u32,
269 /// Declared height in pixels.
270 height: u32,
271 /// Bytes the declared extent and format require.
272 expected: usize,
273 /// Bytes the blob actually holds.
274 actual: usize,
275 },
276 /// The paint transform cannot be inverted, so there is no device-to-image
277 /// mapping for the shader to sample through.
278 #[error("image paint transform is singular or non-finite")]
279 SingularTransform,
280 /// The image fits the ceiling but not the configured atlas budget, and no
281 /// resident image could be evicted to make room for it — either the atlas
282 /// holds nothing but images this same frame drew, or what it holds beside
283 /// them still leaves no rectangle this one fits in.
284 #[error("image {width}x{height} does not fit the {atlas_width}x{atlas_height} atlas budget")]
285 NoAtlasSpace {
286 /// Declared width in pixels.
287 width: u32,
288 /// Declared height in pixels.
289 height: u32,
290 /// Configured atlas width.
291 atlas_width: u32,
292 /// Configured atlas height.
293 atlas_height: u32,
294 },
295 /// The same blob was already resolved earlier in this same frame at a
296 /// different extent.
297 ///
298 /// Not an atlas-capacity refusal: the atlas may be nearly empty. The plan
299 /// clears before it uploads (see the module doc), so releasing this
300 /// blob's rectangle to make room for its own second extent would schedule
301 /// a clear over texels this frame already sampled — the first extent's
302 /// draw keeps its texels and the second is skipped until the next frame
303 /// instead.
304 #[error(
305 "image {width}x{height} already resolved this frame at a different extent; the first \
306 extent's texels are kept until the next frame"
307 )]
308 SameFrameExtentConflict {
309 /// The refused extent's width, after [`fit_extent`] the same way
310 /// [`NoAtlasSpace`](Self::NoAtlasSpace) reports it.
311 width: u32,
312 /// The refused extent's height, after [`fit_extent`].
313 height: u32,
314 },
315}
316
317/// The atlas geometry a tier of adapter is given.
318///
319/// Never `vello_common`'s own [`AtlasConfig`] default: that is 4096 square
320/// across eight layers, 64 MiB of `Rgba8Unorm` per layer and half a gigabyte in
321/// total, which is a desktop-only figure that a phone would pay for the first
322/// image it drew.
323#[derive(Debug, Clone, Copy, PartialEq, Eq)]
324pub struct AtlasBudget {
325 /// Extent of each atlas layer, in pixels.
326 pub atlas_size: (u32, u32),
327 /// Maximum number of layers the array may grow to.
328 pub max_atlases: usize,
329}
330
331impl AtlasBudget {
332 /// The budget a tile-based or downlevel-clamped adapter gets: 4 MiB a
333 /// layer, 16 MiB in total.
334 pub const MOBILE: Self = Self {
335 atlas_size: (1024, 1024),
336 max_atlases: 4,
337 };
338
339 /// The budget a full-profile immediate-mode adapter gets: 16 MiB a layer,
340 /// 128 MiB in total.
341 pub const DESKTOP: Self = Self {
342 atlas_size: (2048, 2048),
343 max_atlases: 8,
344 };
345
346 /// The large-mobile budget's preferred shape: four 2048-square layers,
347 /// 64 MiB in total — a quarter as many layers as
348 /// [`MOBILE_LARGE_DEEP`](Self::MOBILE_LARGE_DEEP) for the same texels, so
349 /// the array grows (and copies itself) a quarter as often.
350 pub const MOBILE_LARGE_WIDE: Self = Self {
351 atlas_size: (2048, 2048),
352 max_atlases: 4,
353 };
354
355 /// The large-mobile budget's shape on an adapter whose texture edge stops
356 /// short of 2048: sixteen 1024-square layers, the same 64 MiB.
357 pub const MOBILE_LARGE_DEEP: Self = Self {
358 atlas_size: (1024, 1024),
359 max_atlases: 16,
360 };
361
362 /// The large-mobile shape `caps`' adapter can create unclamped, or `None`
363 /// when it can create neither — the tier then falls back to
364 /// [`MOBILE`](Self::MOBILE) rather than to a large budget its ceilings
365 /// would have narrowed below the plain one's.
366 #[must_use]
367 pub fn mobile_large(caps: &TierCaps) -> Option<Self> {
368 [Self::MOBILE_LARGE_WIDE, Self::MOBILE_LARGE_DEEP]
369 .into_iter()
370 .find(|shape| shape.clamped(caps) == *shape)
371 }
372
373 /// The unclamped budget `tier` stands for on `caps`' adapter.
374 #[must_use]
375 pub fn for_tier(tier: AtlasTier, caps: &TierCaps) -> Self {
376 match tier {
377 AtlasTier::Mobile => Self::MOBILE,
378 AtlasTier::MobileLarge => Self::mobile_large(caps).unwrap_or(Self::MOBILE),
379 AtlasTier::Desktop => Self::DESKTOP,
380 }
381 }
382
383 /// The budget for `caps`' adapter, clamped to what it can actually create.
384 ///
385 /// The tier choice is [`choose_tier`] over the adapter and the device's
386 /// own memory ([`device_memory_bytes`]); the clamps that follow are the
387 /// adapter's own texture-edge and array-layer ceilings plus the eight-bit
388 /// [`MAX_ATLAS_LAYERS`] the encoded-image record can address.
389 ///
390 /// `FRUST_ENGINE_ATLAS_SIZE` overrides the tier's extent, and the override
391 /// is **budgeted, not unbudgeted**: it passes through
392 /// [`within_total_bytes`](Self::within_total_bytes) against the tier's own
393 /// [`total_bytes`](Self::total_bytes) before the adapter clamps. So the knob
394 /// redistributes a tier's memory between extent and depth — one large layer
395 /// instead of eight small ones — and cannot spend more of it than the tier
396 /// itself would. Clamped to the adapter last, because an override is a
397 /// request and a request the adapter cannot honour is still narrowed to what
398 /// it can.
399 ///
400 /// The resolved budget is reported once per process through
401 /// `atlas_tier_line`, because which tier a given phone takes is otherwise
402 /// an inference from behaviour rather than an observation: the adapter
403 /// name, the driver's own capability answers and the memory reading are
404 /// all device-reported, and a device answering differently from the one
405 /// it resembles is exactly the case a benchmark capture has to be able to
406 /// name. That line is `perf-trace`-only, like every other `frust-perf`
407 /// line the engine emits — a build without the feature resolves the same
408 /// budget and writes nothing.
409 #[must_use]
410 pub fn for_caps(caps: &TierCaps) -> Self {
411 let choice = choose_tier(caps, device_memory_bytes());
412 let resolved = Self::for_choice(choice, caps);
413
414 #[cfg(feature = "perf-trace")]
415 ATLAS_TIER_LOG.emit(caps, choice, resolved);
416 resolved
417 }
418
419 /// The budget `choice` resolves to on `caps`' adapter — [`for_caps`]
420 /// without reading the device's memory or writing the tier line, so the
421 /// table's outcome is testable for any adapter and memory size.
422 ///
423 /// [`for_caps`]: Self::for_caps
424 #[must_use]
425 pub fn for_choice(choice: TierChoice, caps: &TierCaps) -> Self {
426 let tier = Self::for_tier(choice.tier, caps);
427 match config::atlas_size() {
428 None => tier.clamped(caps),
429 Some(atlas_size) => Self {
430 atlas_size,
431 max_atlases: tier.max_atlases,
432 }
433 .within_total_bytes(tier.total_bytes())
434 .clamped(caps),
435 }
436 }
437
438 /// This budget narrowed so a fully grown array costs at most `total_bytes`.
439 ///
440 /// The extent goes first, scaled uniformly until one layer fits inside the
441 /// whole allowance; the layer count then takes whatever multiple of that
442 /// layer is left, never below one. Scaling the extent rather than only the
443 /// depth is what makes the ceiling hold at all: a single 16384-square layer
444 /// is a gigabyte on its own, so a clamp that could only reduce the layer
445 /// *count* would have nothing left to reduce.
446 ///
447 /// Both axes are floored after the uniform scale, which can only take the
448 /// product below the allowance — except where an extreme aspect ratio floors
449 /// one axis to nothing and it is raised back to a texel, so each axis is
450 /// then held against what the other leaves.
451 #[must_use]
452 pub fn within_total_bytes(self, total_bytes: u64) -> Self {
453 // A single texel is the smallest atlas that exists, so an allowance
454 // below one is treated as one rather than producing a zero extent the
455 // packer could not address.
456 let allowance = total_bytes.max(ATLAS_FORMAT_BYTES);
457 let max_texels = allowance / ATLAS_FORMAT_BYTES;
458
459 let (width, height) = self.atlas_size;
460 let texels = u64::from(width).saturating_mul(u64::from(height));
461 let atlas_size = if texels <= max_texels {
462 (width, height)
463 } else {
464 // `as` on a float saturates rather than wrapping, and both factors
465 // are finite and positive, so the narrowing is a plain floor.
466 let scale = (max_texels as f64 / texels.max(1) as f64).sqrt();
467 let scaled_w = ((f64::from(width) * scale) as u32).max(1);
468 let scaled_h = ((f64::from(height) * scale) as u32).max(1);
469 let fitted_w = fit_axis(scaled_w, max_texels / u64::from(scaled_h));
470 let fitted_h = fit_axis(scaled_h, max_texels / u64::from(fitted_w));
471 (fitted_w, fitted_h)
472 };
473
474 let layer_bytes = u64::from(atlas_size.0)
475 .saturating_mul(u64::from(atlas_size.1))
476 .saturating_mul(ATLAS_FORMAT_BYTES)
477 .max(1);
478 let layers = usize::try_from(allowance / layer_bytes).unwrap_or(MAX_ATLAS_LAYERS);
479
480 Self {
481 atlas_size,
482 max_atlases: self.max_atlases.min(layers).max(1),
483 }
484 }
485
486 /// Whether `region` lies inside an atlas array of `layers` layers at this
487 /// budget's per-layer extent.
488 ///
489 /// The pure counterpart of [`crate::gpu::atlas::AtlasArray::contains`],
490 /// answerable with no device: a consumer that records where an image lives
491 /// before the array exists checks the rectangle here, so it can never come
492 /// to name a region the array will refuse to write and then sample it as
493 /// whatever the texture happened to hold.
494 #[must_use]
495 pub fn contains(self, region: AtlasRegion, layers: u32) -> bool {
496 !region.is_empty()
497 && region.layer < layers
498 && region.offset[0].saturating_add(region.size[0]) <= self.atlas_size.0
499 && region.offset[1].saturating_add(region.size[1]) <= self.atlas_size.1
500 }
501
502 /// This budget narrowed to what `caps`' adapter can create.
503 #[must_use]
504 pub fn clamped(self, caps: &TierCaps) -> Self {
505 let edge = caps.max_texture_dimension_2d.max(1);
506 let layers = usize::try_from(caps.max_texture_array_layers).unwrap_or(MAX_ATLAS_LAYERS);
507
508 Self {
509 atlas_size: (
510 self.atlas_size.0.clamp(1, edge),
511 self.atlas_size.1.clamp(1, edge),
512 ),
513 max_atlases: self.max_atlases.clamp(1, layers.min(MAX_ATLAS_LAYERS)),
514 }
515 }
516
517 /// The `vello_common` configuration this budget describes.
518 ///
519 /// `initial_atlas_count` is zero so the first layer is created by the first
520 /// allocation that needs one: an application drawing no images pays for no
521 /// atlas at all, and [`vello_common::multi_atlas::MultiAtlasManager::new`]'s
522 /// own `expect` on eager creation can never be reached.
523 #[must_use]
524 pub fn config(self) -> AtlasConfig {
525 AtlasConfig {
526 initial_atlas_count: 0,
527 max_atlases: self.max_atlases,
528 atlas_size: self.atlas_size,
529 auto_grow: true,
530 allocation_strategy: AllocationStrategy::FirstFit,
531 }
532 }
533
534 /// The bytes one fully populated layer costs at [`ATLAS_FORMAT_BYTES`].
535 #[must_use]
536 pub fn layer_bytes(self) -> u64 {
537 u64::from(self.atlas_size.0)
538 .saturating_mul(u64::from(self.atlas_size.1))
539 .saturating_mul(ATLAS_FORMAT_BYTES)
540 }
541
542 /// The bytes every layer of this budget costs once fully grown.
543 #[must_use]
544 pub fn total_bytes(self) -> u64 {
545 self.layer_bytes()
546 .saturating_mul(self.max_atlases.try_into().unwrap_or(u64::MAX))
547 }
548}
549
550/// Bytes per texel of the atlas array (`Rgba8Unorm`).
551pub const ATLAS_FORMAT_BYTES: u64 = 4;
552
553/// `evictions` less every rectangle one of `uploads` covers exactly.
554///
555/// Exact cover only — same layer, offset and extent: an upload writes every
556/// texel of its own rectangle (its pixels are exactly the region's extent), so
557/// a clear of that same rectangle ordered ahead of it is overwritten in full.
558/// A rectangle covered only partly, or only by several uploads together, keeps
559/// its clear: whatever texels no upload reaches must still read as zero to the
560/// next tenant. Order is preserved, so the clears stay in the order the
561/// residency reported them.
562///
563/// The exact-cover match compares the reported eviction rectangle with the
564/// upload's allocated rectangle. Evictions are reported padded by
565/// [`ATLAS_PADDING`] while uploads carry the raw allocation, so the two only
566/// agree while the padding is zero; the assertion below ties the skip to that.
567#[must_use]
568pub fn uncovered_clears(evictions: &[AtlasRegion], uploads: &[ImageUpload]) -> Vec<AtlasRegion> {
569 if evictions.is_empty() || uploads.is_empty() {
570 return evictions.to_vec();
571 }
572 let written: HashSet<AtlasRegion> = uploads.iter().map(|upload| upload.region).collect();
573 evictions
574 .iter()
575 .copied()
576 .filter(|region| !written.contains(region))
577 .collect()
578}
579
580/// Whether `caps`' adapter reads as a mobile part by its capability answers
581/// alone.
582///
583/// Two signals, either of which is enough:
584///
585/// - its usable limits are clamped to the GLES-3.0/WebGL2 downlevel defaults,
586/// which is the profile every GL-backed target reports; or
587/// - it reports `transient_saves_memory`, wgpu's own answer to "is this a
588/// tile-based renderer keeping attachments on-chip" — true on the mobile
589/// TBDR parts and false on desktop immediate-mode ones.
590///
591/// This is the device-class read text hinting keys on, and the atlas tier's
592/// fallback for an adapter [`ADAPTER_FAMILIES`] does not name. It is not the
593/// atlas tier itself: some mobile drivers answer `transient_saves_memory`
594/// false (Adreno 618/620 do), which this read alone would put on the desktop
595/// budget — see [`choose_tier`].
596#[must_use]
597pub fn is_mobile_tier(caps: &TierCaps) -> bool {
598 caps.downlevel_profile != DownlevelProfile::Full || caps.transient_saves_memory
599}
600
601/// Which class of part an adapter's name places it in.
602#[derive(Debug, Clone, Copy, PartialEq, Eq)]
603pub enum AdapterFamily {
604 /// A phone/tablet GPU family: the atlas competes with the rest of a
605 /// shared-memory device's budget.
606 Mobile,
607 /// A discrete or desktop-integrated GPU family.
608 Desktop,
609}
610
611/// The adapter-name table the atlas tier is read from: a lowercase fragment
612/// and the family an adapter whose name contains it belongs to, first match
613/// winning.
614///
615/// A name table rather than capability answers alone because the capability
616/// answers disagree with the hardware: Adreno 618 and 620 report a full
617/// profile and `transient_saves_memory` false — the desktop answers — while
618/// sharing their device's memory with everything else on it. An adapter the
619/// table does not name keeps the capability read ([`is_mobile_tier`]), so a
620/// family missing here costs the old behaviour rather than a wrong one.
621///
622/// Apple is listed whole: Apple-silicon Macs already took the mobile budget
623/// through `transient_saves_memory`, so naming them changes nothing there.
624pub const ADAPTER_FAMILIES: &[(&str, AdapterFamily)] = &[
625 ("adreno", AdapterFamily::Mobile),
626 ("mali", AdapterFamily::Mobile),
627 ("immortalis", AdapterFamily::Mobile),
628 ("powervr", AdapterFamily::Mobile),
629 ("xclipse", AdapterFamily::Mobile),
630 ("maleoon", AdapterFamily::Mobile),
631 ("apple", AdapterFamily::Mobile),
632 ("tegra", AdapterFamily::Mobile),
633 ("nvidia", AdapterFamily::Desktop),
634 ("geforce", AdapterFamily::Desktop),
635 ("quadro", AdapterFamily::Desktop),
636 ("radeon", AdapterFamily::Desktop),
637 ("amd", AdapterFamily::Desktop),
638 ("intel", AdapterFamily::Desktop),
639];
640
641/// The family [`ADAPTER_FAMILIES`] places `adapter_name` in, matched
642/// case-insensitively; `None` for a name the table does not know.
643#[must_use]
644pub fn adapter_family(adapter_name: &str) -> Option<AdapterFamily> {
645 let name = adapter_name.to_ascii_lowercase();
646 ADAPTER_FAMILIES
647 .iter()
648 .find(|(fragment, _)| name.contains(fragment))
649 .map(|(_, family)| *family)
650}
651
652/// The atlas budget tiers.
653#[derive(Debug, Clone, Copy, PartialEq, Eq)]
654pub enum AtlasTier {
655 /// [`AtlasBudget::MOBILE`].
656 Mobile,
657 /// [`AtlasBudget::mobile_large`]: a mobile family part on a device with at
658 /// least [`MOBILE_LARGE_MIN_MEMORY`] of memory.
659 MobileLarge,
660 /// [`AtlasBudget::DESKTOP`].
661 Desktop,
662}
663
664impl AtlasTier {
665 /// The tier's name as the tier line spells it.
666 #[must_use]
667 pub fn label(self) -> &'static str {
668 match self {
669 Self::Mobile => "mobile",
670 Self::MobileLarge => "mobile_large",
671 Self::Desktop => "desktop",
672 }
673 }
674}
675
676/// Which rule of [`choose_tier`] decided a tier.
677#[derive(Debug, Clone, Copy, PartialEq, Eq)]
678pub enum TierRule {
679 /// The adapter's limits are clamped to the WebGL2 downlevel defaults.
680 Downlevel,
681 /// [`ADAPTER_FAMILIES`] named the adapter.
682 Table,
683 /// The table did not; [`is_mobile_tier`]'s capability read decided.
684 Heuristic,
685}
686
687impl TierRule {
688 /// The rule's name as the tier line spells it.
689 #[must_use]
690 pub fn label(self) -> &'static str {
691 match self {
692 Self::Downlevel => "downlevel",
693 Self::Table => "table",
694 Self::Heuristic => "heuristic",
695 }
696 }
697}
698
699/// The resolved tier, with what decided it.
700#[derive(Debug, Clone, Copy, PartialEq, Eq)]
701pub struct TierChoice {
702 /// The tier.
703 pub tier: AtlasTier,
704 /// The rule that chose it.
705 pub rule: TierRule,
706 /// The device memory the choice was made against, when it was known.
707 pub device_memory: Option<u64>,
708}
709
710/// The device memory from which a mobile family part takes the large budget:
711/// 4 GiB.
712///
713/// The large budget is 64 MiB of atlas, a sixteenth of that; below it the
714/// plain 16 MiB budget stays.
715pub const MOBILE_LARGE_MIN_MEMORY: u64 = 4 << 30;
716
717/// The atlas tier for `caps`' adapter on a device with `device_memory` bytes
718/// of memory (`None` when unknown).
719///
720/// In order:
721///
722/// 1. A downlevel (WebGL2/GLES) profile is [`AtlasTier::Mobile`], whatever the
723/// name: its limits are the browser ceiling, and the budget follows them.
724/// 2. An adapter [`ADAPTER_FAMILIES`] names takes its family's tier — a
725/// desktop family [`AtlasTier::Desktop`]; a mobile family
726/// [`AtlasTier::MobileLarge`] when the device reports at least
727/// [`MOBILE_LARGE_MIN_MEMORY`] and the adapter can create a large shape
728/// unclamped ([`AtlasBudget::mobile_large`]), else [`AtlasTier::Mobile`].
729/// Unknown memory is the small tier: the large one is earned, not assumed.
730/// 3. Any other adapter keeps [`is_mobile_tier`]'s capability read, mobile or
731/// desktop, exactly as before the table existed.
732#[must_use]
733pub fn choose_tier(caps: &TierCaps, device_memory: Option<u64>) -> TierChoice {
734 let (tier, rule) = if caps.downlevel_profile != DownlevelProfile::Full {
735 (AtlasTier::Mobile, TierRule::Downlevel)
736 } else {
737 match adapter_family(&caps.adapter_name) {
738 Some(AdapterFamily::Desktop) => (AtlasTier::Desktop, TierRule::Table),
739 Some(AdapterFamily::Mobile) => {
740 let large = device_memory.is_some_and(|bytes| bytes >= MOBILE_LARGE_MIN_MEMORY)
741 && AtlasBudget::mobile_large(caps).is_some();
742 let tier = if large {
743 AtlasTier::MobileLarge
744 } else {
745 AtlasTier::Mobile
746 };
747 (tier, TierRule::Table)
748 }
749 None if is_mobile_tier(caps) => (AtlasTier::Mobile, TierRule::Heuristic),
750 None => (AtlasTier::Desktop, TierRule::Heuristic),
751 }
752 };
753 TierChoice {
754 tier,
755 rule,
756 device_memory,
757 }
758}
759
760/// The device's total memory in bytes, when this platform reports it.
761///
762/// Read once per process from `/proc/meminfo`'s `MemTotal` on Linux and
763/// Android — the kernel's own figure, readable without a permission on both —
764/// and `None` everywhere else, which keeps every other platform on the tier
765/// it took before the large one existed.
766#[must_use]
767pub fn device_memory_bytes() -> Option<u64> {
768 #[cfg(any(target_os = "linux", target_os = "android"))]
769 {
770 use std::sync::OnceLock;
771 static MEMORY: OnceLock<Option<u64>> = OnceLock::new();
772 *MEMORY.get_or_init(|| {
773 std::fs::read_to_string("/proc/meminfo")
774 .ok()
775 .and_then(|text| parse_mem_total(&text))
776 })
777 }
778 #[cfg(not(any(target_os = "linux", target_os = "android")))]
779 {
780 None
781 }
782}
783
784/// `MemTotal` from a `/proc/meminfo` text, in bytes; `None` when the line is
785/// missing or not the `<n> kB` the kernel writes.
786#[must_use]
787pub fn parse_mem_total(meminfo: &str) -> Option<u64> {
788 let line = meminfo
789 .lines()
790 .find_map(|line| line.strip_prefix("MemTotal:"))?;
791 let mut fields = line.split_whitespace();
792 let kib: u64 = fields.next()?.parse().ok()?;
793 match fields.next() {
794 Some("kB") => kib.checked_mul(1024),
795 _ => None,
796 }
797}
798
799/// The process's latch for the resolved-tier line, held by
800/// [`AtlasBudget::for_caps`].
801#[cfg(feature = "perf-trace")]
802static ATLAS_TIER_LOG: TierLogOnce = TierLogOnce::new();
803
804/// A one-shot latch for the resolved-tier line.
805///
806/// Once per *process* rather than once per call: a budget is resolved once per
807/// surface, and the question the line answers — which tier this adapter took —
808/// is about the device, so one line per run answers it completely while a line
809/// per surface would repeat it.
810///
811/// A latch value rather than a bare [`std::sync::Once`] because the rule it
812/// keeps is worth testing: [`emit`](Self::emit) answers whether *this* call was
813/// the one that wrote the line, so a test constructs a latch of its own and
814/// pins "the first call writes and no later one does" without installing a
815/// process-global logger.
816///
817/// Compiled only under `perf-trace`, with the line it writes: the release-lean
818/// gate asserts a shipping binary carries no `frust-perf` bytes at all, and a
819/// `#[cfg]` is what makes that the compiler's answer rather than the
820/// optimizer's.
821#[cfg(feature = "perf-trace")]
822#[derive(Debug, Default)]
823pub struct TierLogOnce {
824 emitted: AtomicBool,
825}
826
827#[cfg(feature = "perf-trace")]
828impl TierLogOnce {
829 /// A latch that has not emitted yet.
830 #[must_use]
831 pub const fn new() -> Self {
832 Self {
833 emitted: AtomicBool::new(false),
834 }
835 }
836
837 /// Log [`atlas_tier_line`] for `caps`, `choice` and `budget` if this latch
838 /// has not fired, answering whether this call was the one that did.
839 pub fn emit(&self, caps: &TierCaps, choice: TierChoice, budget: AtlasBudget) -> bool {
840 if self.emitted.swap(true, Ordering::Relaxed) {
841 return false;
842 }
843
844 log::info!("{}", atlas_tier_line(caps, choice, budget));
845 true
846 }
847}
848
849/// The one line a run records the resolved atlas tier as.
850///
851/// `perf-trace`-only, with [`TierLogOnce`] — see its own doc for why the gate
852/// is a `#[cfg]` rather than a branch.
853///
854/// `frust-perf`-prefixed deliberately: a benchmark capture keeps every line
855/// carrying that prefix and drops the rest, so this is what lets a run's own
856/// log say which budget the device was given rather than leaving it to be
857/// inferred from whether images went missing. `rule` names which of
858/// [`choose_tier`]'s rules decided, and `memory_mib` the device memory it was
859/// decided against (`unknown` where the platform does not report it). The
860/// adapter name is last because it is the one field that can contain spaces,
861/// which keeps every `key=value` field ahead of it parseable by splitting on
862/// whitespace.
863#[cfg(feature = "perf-trace")]
864#[must_use]
865pub fn atlas_tier_line(caps: &TierCaps, choice: TierChoice, budget: AtlasBudget) -> String {
866 let memory = choice
867 .device_memory
868 .map_or_else(|| "unknown".to_string(), |bytes| (bytes >> 20).to_string());
869
870 format!(
871 "frust-perf atlas tier={} rule={} budget={}x{}x{} downlevel={:?} \
872 transient_saves_memory={} memory_mib={memory} adapter={}",
873 choice.tier.label(),
874 choice.rule.label(),
875 budget.atlas_size.0,
876 budget.atlas_size.1,
877 budget.max_atlases,
878 caps.downlevel_profile,
879 caps.transient_saves_memory,
880 caps.adapter_name,
881 )
882}
883
884/// Where one image's texels live in the atlas array.
885///
886/// Hashable because the pending-clear list carries a set index beside it: the
887/// same rectangle can reach [`ImageResidency::evictions`] twice before a single
888/// clear has been serviced, and the dedup that keeps it reported once must not
889/// be a linear scan of a list that grows with the churn rate.
890#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
891pub struct AtlasRegion {
892 /// Array layer holding the region.
893 pub layer: u32,
894 /// Pixel offset of the region's top-left corner within that layer.
895 pub offset: [u32; 2],
896 /// Region extent in pixels.
897 pub size: [u32; 2],
898}
899
900impl AtlasRegion {
901 /// The bytes this region's texels occupy at [`ATLAS_FORMAT_BYTES`].
902 #[must_use]
903 pub fn byte_len(self) -> usize {
904 (self.size[0] as usize)
905 .saturating_mul(self.size[1] as usize)
906 .saturating_mul(ATLAS_FORMAT_BYTES as usize)
907 }
908
909 /// The row stride a texel copy of this region uses.
910 #[must_use]
911 pub fn bytes_per_row(self) -> u32 {
912 self.size[0].saturating_mul(ATLAS_FORMAT_BYTES as u32)
913 }
914
915 /// Whether this region has any texels at all.
916 #[must_use]
917 pub fn is_empty(self) -> bool {
918 self.size[0] == 0 || self.size[1] == 0
919 }
920}
921
922/// A resident image: the handle a paint names it by, and where its texels are.
923#[derive(Debug, Clone, Copy, PartialEq, Eq)]
924pub struct ResidentImage {
925 /// The handle an [`ImageSource::OpaqueId`] carries.
926 pub id: ImageId,
927 /// The atlas rectangle holding the image's own pixels, padding excluded.
928 ///
929 /// Equal to [`natural`](Self::natural) unless the source was minified to
930 /// fit the atlas, in which case it is the smaller rectangle actually
931 /// uploaded.
932 pub region: AtlasRegion,
933 /// The source's own declared extent, before any minification.
934 ///
935 /// Carried because it is the extent a paint transform was composed against
936 /// — `compile::paint::encode_image_command` derives its natural-to-dest
937 /// mapping from the `ImageData`'s declared size, which it reads before this
938 /// residency is consulted. A consumer that ends up sampling a *minified*
939 /// rectangle has to fold [`minify_scale`](Self::minify_scale) into that
940 /// transform, and this is the half of the ratio the atlas rectangle does
941 /// not carry.
942 pub natural: [u32; 2],
943 /// Transparent padding pixels around `region` in the atlas.
944 pub padding: u32,
945 /// Whether the image's premultiplied pixels include any non-opaque texel.
946 pub may_have_transparency: bool,
947}
948
949impl ResidentImage {
950 /// This image as the paint-side source a `vello_common` encoding carries.
951 #[must_use]
952 pub fn source(&self) -> ImageSource {
953 ImageSource::opaque_id_with_transparency_hint(self.id, self.may_have_transparency)
954 }
955
956 /// The per-axis factor mapping a natural-space texel coordinate onto the
957 /// resident rectangle, or `None` when the source was stored at full size.
958 ///
959 /// `None` rather than `Some((1.0, 1.0))` so a caller can skip the rescale
960 /// entirely on the overwhelmingly common path, and so "was this image
961 /// minified?" is one question rather than a float comparison.
962 #[must_use]
963 pub fn minify_scale(&self) -> Option<(f32, f32)> {
964 if self.region.size == self.natural {
965 return None;
966 }
967 let natural_w = self.natural[0].max(1) as f32;
968 let natural_h = self.natural[1].max(1) as f32;
969 Some((
970 self.region.size[0] as f32 / natural_w,
971 self.region.size[1] as f32 / natural_h,
972 ))
973 }
974}
975
976/// One region's texels, waiting to be written into the atlas array.
977///
978/// The pixels are the premultiplied `Rgba8Unorm` the atlas stores, produced
979/// once at allocation — an image made resident on frame 1 and drawn on frames
980/// 1..1000 is converted exactly once.
981#[derive(Debug, Clone)]
982pub struct ImageUpload {
983 /// The handle the paint naming these pixels carries.
984 ///
985 /// Carried so a consumer rebuilding its own view of residency can key this
986 /// upload directly rather than pairing the frame's uploads positionally
987 /// against its encoded paints. A plan re-emitted until it is acknowledged
988 /// breaks that pairing (an upload can outlive the frame whose draw order
989 /// produced it), and an upload that names itself needs no ordering to be
990 /// read correctly.
991 pub id: ImageId,
992 /// Where the texels go.
993 pub region: AtlasRegion,
994 /// The source's own declared extent, before any minification — see
995 /// [`ResidentImage::natural`], whose value this is.
996 ///
997 /// Carried on the upload because a consumer rebuilding its own view of
998 /// residency from a compiled frame has the upload and nothing else, and
999 /// `region` alone cannot say whether it holds a minified copy.
1000 pub natural: [u32; 2],
1001 /// The premultiplied pixels, row-major and exactly `region`'s extent.
1002 pub pixels: Arc<Pixmap>,
1003 /// Whether those pixels include any non-opaque texel — the same hint
1004 /// [`ResidentImage::may_have_transparency`] carries, so a consumer can
1005 /// rebuild the whole residency record from this upload alone.
1006 pub may_have_transparency: bool,
1007}
1008
1009/// One cache entry: the handle, its rectangle, and when it was last drawn.
1010#[derive(Debug, Clone, Copy)]
1011struct Entry {
1012 id: ImageId,
1013 region: AtlasRegion,
1014 /// The source extent this entry was allocated for, which is what a later
1015 /// frame's `ImageData` is matched against — `region` may be the minified
1016 /// rectangle and so says nothing about the source.
1017 natural: [u32; 2],
1018 may_have_transparency: bool,
1019 last_seen: u64,
1020}
1021
1022/// The images resident in the atlas array, keyed by blob identity.
1023///
1024/// One per compiler (and so one per surface). Its frame clock is advanced by
1025/// [`begin_frame`](Self::begin_frame), which is also where the age-based reap
1026/// runs — before any of the frame's own resolutions, so a rectangle freed this
1027/// frame is available to this frame's allocations and its clear is ordered
1028/// ahead of their uploads.
1029///
1030/// It also owns the atlas allocator the *glyph* policy packs into — see this
1031/// module's doc and [`allocator_mut`](Self::allocator_mut).
1032#[derive(Debug)]
1033pub struct ImageResidency {
1034 cache: ImageCache,
1035 budget: AtlasBudget,
1036 entries: HashMap<u64, Entry>,
1037 frame: u64,
1038 disabled: bool,
1039 uploads: Vec<ImageUpload>,
1040 evictions: Vec<AtlasRegion>,
1041 /// The pending clears' own set index: exactly the rectangles `evictions`
1042 /// holds, kept so [`release`](Self::release)'s "report a rectangle once"
1043 /// dedup is a hash lookup rather than a scan of a list whose length grows
1044 /// with the churn rate.
1045 eviction_set: HashSet<AtlasRegion>,
1046 skipped: u64,
1047 /// Rectangles taken back from a resident image because the packer had no
1048 /// room for a new one, over this residency's lifetime.
1049 pressure_evictions: u64,
1050 /// The same count for the current frame alone, zeroed by
1051 /// [`begin_frame`](Self::begin_frame) — what a per-frame report reads, since
1052 /// the lifetime total says nothing about whether the atlas is churning
1053 /// *now*.
1054 frame_pressure_evictions: u64,
1055 minified: u64,
1056 /// Blob keys whose minification has already been reported, so a source that
1057 /// is redrawn — or reaped and made resident again — logs once rather than
1058 /// once per residency.
1059 reported_minify: HashSet<u64>,
1060 /// Scratch for the reap's key list, kept so a frame that evicts allocates
1061 /// nothing.
1062 reaped: Vec<u64>,
1063 /// Scratch for the pressure planner's view of the layers — one rectangle
1064 /// per resident entry, grouped by layer — kept for the same reason: the
1065 /// frame that has to evict is the one least able to afford an allocation.
1066 pressure_rects: Vec<PressureRect>,
1067 /// Scratch for the candidate order over [`Self::pressure_rects`], least
1068 /// recently seen first. Built and sorted once per refused allocation, never
1069 /// once per eviction.
1070 pressure_order: Vec<usize>,
1071 /// Scratch for each layer's modelled free area, indexed by layer.
1072 pressure_free_area: Vec<u64>,
1073 /// Scratch for the distinct layers [`Self::pressure_order`] names, in the
1074 /// order their stalest candidate appears — the per-layer walk's own visit
1075 /// order, kept for the same no-allocation-while-evicting reason as the
1076 /// other pressure scratch.
1077 pressure_layers: Vec<u32>,
1078 /// Scratch for the keys an accepted plan releases.
1079 pressure_plan: Vec<u64>,
1080}
1081
1082impl ImageResidency {
1083 /// A residency over `budget`'s atlas geometry.
1084 ///
1085 /// `FRUST_ENGINE_NO_ATLAS` is consulted here, once, rather than per
1086 /// resolution: the switch is process-global and cached, so re-reading it on
1087 /// a hot path would buy nothing but a lock. A residency constructed with
1088 /// the switch set is exactly [`disabled`](Self::disabled).
1089 #[must_use]
1090 pub fn new(budget: AtlasBudget) -> Self {
1091 Self::with_enabled(budget, !config::atlas_disabled())
1092 }
1093
1094 /// A residency that refuses every image, allocating no atlas at all.
1095 ///
1096 /// What `FRUST_ENGINE_NO_ATLAS` selects, and what a caller wanting the same
1097 /// effect programmatically constructs. Every [`resolve`](Self::resolve)
1098 /// answers [`ImageSkip::AtlasDisabled`], so image draws are dropped with a
1099 /// logged reason rather than painted from an atlas that does not exist.
1100 #[must_use]
1101 pub fn disabled(budget: AtlasBudget) -> Self {
1102 Self::with_enabled(budget, false)
1103 }
1104
1105 fn with_enabled(budget: AtlasBudget, enabled: bool) -> Self {
1106 Self {
1107 cache: ImageCache::new_with_config(budget.config()),
1108 budget,
1109 entries: HashMap::new(),
1110 frame: 0,
1111 disabled: !enabled,
1112 uploads: Vec::new(),
1113 evictions: Vec::new(),
1114 eviction_set: HashSet::new(),
1115 skipped: 0,
1116 pressure_evictions: 0,
1117 frame_pressure_evictions: 0,
1118 minified: 0,
1119 reported_minify: HashSet::new(),
1120 reaped: Vec::new(),
1121 pressure_rects: Vec::new(),
1122 pressure_order: Vec::new(),
1123 pressure_free_area: Vec::new(),
1124 pressure_layers: Vec::new(),
1125 pressure_plan: Vec::new(),
1126 }
1127 }
1128
1129 /// A residency sized for `caps`' adapter.
1130 #[must_use]
1131 pub fn for_caps(caps: &TierCaps) -> Self {
1132 Self::new(AtlasBudget::for_caps(caps))
1133 }
1134
1135 /// The atlas geometry this residency allocates within.
1136 #[must_use]
1137 pub fn budget(&self) -> AtlasBudget {
1138 self.budget
1139 }
1140
1141 /// The single atlas allocator, for reading.
1142 ///
1143 /// Enough to resolve a handle either side allocated
1144 /// ([`ImageCache::get`]) or to count the layers created so far, without
1145 /// the mutable borrow allocation needs.
1146 #[must_use]
1147 pub fn allocator(&self) -> &ImageCache {
1148 &self.cache
1149 }
1150
1151 /// The single atlas allocator, for allocating through.
1152 ///
1153 /// This is the seam the glyph policy takes its slots from — see this
1154 /// module's doc for why one cache rather than two is a correctness
1155 /// requirement. `ImageId`s and rectangles handed out through here and
1156 /// through [`resolve`](Self::resolve) come from the same packer and the
1157 /// same id space, so a glyph and an image can never be given the same
1158 /// handle or overlapping texels.
1159 ///
1160 /// **The contract a borrower keeps:** allocate what you like, and
1161 /// deallocate *only handles you allocated yourself*. This residency's entry
1162 /// map records the rectangles it allocated and nothing re-checks them;
1163 /// freeing one of them from outside would leave an entry claiming a
1164 /// rectangle the packer has since handed to someone else. Nothing here
1165 /// enforces that, because the type the two sides must share is
1166 /// `vello_common`'s own and cannot carry an ownership tag — the glyph
1167 /// policy deallocates strictly inside `glifo`'s eviction, over the handles
1168 /// in its own entry map.
1169 #[must_use]
1170 pub fn allocator_mut(&mut self) -> &mut ImageCache {
1171 &mut self.cache
1172 }
1173
1174 /// Whether this residency refuses every image.
1175 #[must_use]
1176 pub fn is_disabled(&self) -> bool {
1177 self.disabled
1178 }
1179
1180 /// How many images are currently resident.
1181 #[must_use]
1182 pub fn entry_count(&self) -> usize {
1183 self.entries.len()
1184 }
1185
1186 /// The frame clock entries age against.
1187 #[must_use]
1188 pub fn frame(&self) -> u64 {
1189 self.frame
1190 }
1191
1192 /// How many atlas layers have been created so far.
1193 ///
1194 /// The array texture must be at least this deep before the frame's uploads
1195 /// are written.
1196 ///
1197 /// Read off the shared allocator rather than accumulated from this
1198 /// residency's own allocations, so a layer the *glyph* policy created
1199 /// counts too: the two classes pack into one array, and an array sized to
1200 /// the image half alone would refuse to hold the glyph half's pages. The
1201 /// count only ever grows — `MultiAtlasManager` appends layers and never
1202 /// drops one — so a depth reported on an earlier frame stays valid.
1203 #[must_use]
1204 pub fn layers(&self) -> u32 {
1205 u32::try_from(self.cache.atlas_count()).unwrap_or(u32::MAX)
1206 }
1207
1208 /// How many resolutions have been refused over this residency's lifetime.
1209 ///
1210 /// Counts only images that could not be uploaded at all — a format or
1211 /// extent the conversion refuses, a disabled atlas, or an atlas with no
1212 /// room left *and nothing evictable to make room with*. A source merely
1213 /// larger than one atlas layer is minified to fit, and one that displaced
1214 /// another image is counted by
1215 /// [`pressure_evictions`](Self::pressure_evictions) instead; neither
1216 /// reaches this counter.
1217 #[must_use]
1218 pub fn skipped(&self) -> u64 {
1219 self.skipped
1220 }
1221
1222 /// How many resident images have been evicted to make room for another
1223 /// over this residency's lifetime.
1224 ///
1225 /// The price of never refusing a draw the atlas could hold *something*
1226 /// for: each one is an image that will re-upload the next frame it is
1227 /// drawn on. A steady screen leaves this at zero; a scroll through more
1228 /// distinct images than the atlas has rectangles for grows it once per
1229 /// displaced image, which is the signal that the budget — not the
1230 /// mechanism — is what a frame is paying for.
1231 #[must_use]
1232 pub fn pressure_evictions(&self) -> u64 {
1233 self.pressure_evictions
1234 }
1235
1236 /// How many resident images the *current* frame has evicted to make room.
1237 ///
1238 /// Zeroed by [`begin_frame`](Self::begin_frame), so a per-frame report reads
1239 /// this rather than differencing
1240 /// [`pressure_evictions`](Self::pressure_evictions) itself.
1241 #[must_use]
1242 pub fn frame_pressure_evictions(&self) -> u64 {
1243 self.frame_pressure_evictions
1244 }
1245
1246 /// How many sources have been downsampled to fit the atlas over this
1247 /// residency's lifetime.
1248 ///
1249 /// Observational, and the counter that makes "an image too large for the
1250 /// atlas is minified, not dropped" measurable rather than asserted.
1251 #[must_use]
1252 pub fn minified(&self) -> u64 {
1253 self.minified
1254 }
1255
1256 /// The regions whose texels must be cleared before the pending uploads.
1257 ///
1258 /// Pending rather than per-frame: an entry freed on a frame nobody
1259 /// serviced is still waiting to be cleared on the next one. Both kinds of
1260 /// eviction land here — the age-based reap at the head of a frame and a
1261 /// rectangle taken back under pressure mid-frame — so the consumer's
1262 /// clear-then-upload contract covers the second without knowing it exists.
1263 #[must_use]
1264 pub fn evictions(&self) -> &[AtlasRegion] {
1265 &self.evictions
1266 }
1267
1268 /// The regions whose texels must be written after the pending evictions.
1269 ///
1270 /// Pending rather than per-frame: an image made resident on a frame that was
1271 /// refused is still waiting to be written on the next one.
1272 #[must_use]
1273 pub fn uploads(&self) -> &[ImageUpload] {
1274 &self.uploads
1275 }
1276
1277 /// A copy of the pending clear-then-write plan, evictions first.
1278 ///
1279 /// A *copy*, not a drain: the plan stays pending until
1280 /// [`acknowledge_plan`](Self::acknowledge_plan) says it was serviced against
1281 /// a live atlas, so a frame refused after compiling re-emits the same plan
1282 /// on the next frame rather than losing it (see the module doc). The upload
1283 /// pixels are behind an `Arc`, so a re-emitted plan copies handles rather
1284 /// than texels, and the steady-state plan is empty on both counts.
1285 ///
1286 /// The clears are [`evictions`](Self::evictions) less every rectangle a
1287 /// pending upload covers exactly (see [`uncovered_clears`]): that upload
1288 /// writes every texel the clear would have zeroed, after it, in the same
1289 /// plan — so the clear is pure cost, a rectangle's worth of zeroes per
1290 /// churned image. Dropped from the plan rather than from `evictions`, so
1291 /// the pending set and its acknowledgement are unchanged; a consumer
1292 /// forgetting the evicted tenant by rectangle (the renderer's registry)
1293 /// still needs the full eviction set: [`evictions`](Self::evictions) names
1294 /// every rectangle to forget, whether or not its clear was skipped.
1295 #[must_use]
1296 pub fn plan(&self) -> (Vec<AtlasRegion>, Vec<ImageUpload>) {
1297 (
1298 uncovered_clears(&self.evictions, &self.uploads),
1299 self.uploads.clone(),
1300 )
1301 }
1302
1303 /// Whether any part of the pending plan is still unserviced.
1304 #[must_use]
1305 pub fn has_pending_plan(&self) -> bool {
1306 !self.evictions.is_empty() || !self.uploads.is_empty()
1307 }
1308
1309 /// Record that the pending plan reached the atlas array, clearing it.
1310 ///
1311 /// Call this only once every region it names has actually been written or
1312 /// cleared. Calling it on a frame that was refused, or whose writes the
1313 /// array declined, is exactly the defect the pending plan exists to prevent:
1314 /// the entry map would go on claiming an image is resident whose texels are
1315 /// whatever the atlas texture happened to hold.
1316 pub fn acknowledge_plan(&mut self) {
1317 self.evictions.clear();
1318 self.eviction_set.clear();
1319 self.uploads.clear();
1320 }
1321
1322 /// Advance the frame clock and reclaim everything unseen for
1323 /// [`MAX_UNSEEN_FRAMES`].
1324 ///
1325 /// Call once at the head of a frame, before any [`resolve`](Self::resolve).
1326 /// An unacknowledged plan deliberately survives this: the frame that was to
1327 /// service it may have been refused, and dropping it here is what would turn
1328 /// a refused frame into a permanently unwritten atlas region.
1329 pub fn begin_frame(&mut self) {
1330 self.frame = self.frame.saturating_add(1);
1331 self.frame_pressure_evictions = 0;
1332 self.reap();
1333 }
1334
1335 /// The atlas rectangle for `data`'s pixels, allocating and scheduling an
1336 /// upload on the first frame that asks for it.
1337 ///
1338 /// # Errors
1339 ///
1340 /// Returns the [`ImageSkip`] naming why the image was refused. Every
1341 /// refusal happens before any `vello_common` call that could assert on the
1342 /// same condition, so an image the renderer cannot hold is a skipped draw
1343 /// rather than a panicked frame (E17).
1344 pub fn resolve(&mut self, data: &ImageData) -> Result<ResidentImage, ImageSkip> {
1345 let result = self.resolve_inner(data);
1346 if result.is_err() {
1347 self.skipped = self.skipped.saturating_add(1);
1348 }
1349 result
1350 }
1351
1352 fn resolve_inner(&mut self, data: &ImageData) -> Result<ResidentImage, ImageSkip> {
1353 if self.disabled {
1354 return Err(ImageSkip::AtlasDisabled);
1355 }
1356
1357 check_supported(data)?;
1358
1359 let key = data.data.id();
1360 if let Some(entry) = self.entries.get_mut(&key)
1361 && entry.natural == [data.width, data.height]
1362 {
1363 entry.last_seen = self.frame;
1364 return Ok(ResidentImage {
1365 id: entry.id,
1366 region: entry.region,
1367 natural: entry.natural,
1368 padding: u32::from(ATLAS_PADDING),
1369 may_have_transparency: entry.may_have_transparency,
1370 });
1371 }
1372
1373 // A blob id recorded at a different extent is a caller that rebuilt the
1374 // metadata around the same buffer. Its old rectangle describes the
1375 // wrong pixels, so it is released before the new one is taken rather
1376 // than left to age out holding both — unless *this* frame has already
1377 // resolved it at the other extent. The plan clears before it uploads,
1378 // so freeing that rectangle would schedule a clear over texels a draw
1379 // this same frame already recorded, which is exactly what the pressure
1380 // path refuses to do. The second extent is skipped instead, on the same
1381 // terms as a frame whose own working set outgrows the atlas — but this
1382 // is a same-frame ordering conflict on one blob key, not an atlas
1383 // capacity shortage, so it is reported as its own reason rather than
1384 // folded into [`ImageSkip::NoAtlasSpace`].
1385 if self
1386 .entries
1387 .get(&key)
1388 .is_some_and(|entry| entry.last_seen == self.frame)
1389 {
1390 let (width, height) = fit_extent([data.width, data.height], self.budget.atlas_size);
1391 return Err(ImageSkip::SameFrameExtentConflict { width, height });
1392 }
1393 self.release(key);
1394
1395 // Checked above, so this cannot assert; the conversion is also the one
1396 // place the straight-alpha premultiply happens, and it happens once per
1397 // image rather than once per frame.
1398 let ImageSource::Pixmap(pixels) = ImageSource::from_peniko_image_data(data) else {
1399 // `from_peniko_image_data` only ever yields a pixmap; an id-backed
1400 // source would mean a `vello_common` change, which is a skip rather
1401 // than an unreachable.
1402 return Err(ImageSkip::UnsupportedFormat(data.format));
1403 };
1404
1405 let natural = [data.width, data.height];
1406 // Minified before the allocation rather than after it: the packer is
1407 // asked for the rectangle that is actually going to be uploaded, so an
1408 // over-ceiling source never takes (and then has to give back) space at
1409 // its declared extent.
1410 let (width, height) = fit_extent(natural, self.budget.atlas_size);
1411 let pixels = if [width, height] == natural {
1412 pixels
1413 } else {
1414 self.note_minify(key, natural, [width, height]);
1415 Arc::new(minify(&pixels, width, height))
1416 };
1417
1418 let no_space = ImageSkip::NoAtlasSpace {
1419 width,
1420 height,
1421 atlas_width: self.budget.atlas_size.0,
1422 atlas_height: self.budget.atlas_size.1,
1423 };
1424
1425 let id = match self.cache.allocate(width, height, ATLAS_PADDING) {
1426 Ok(id) => id,
1427 // The atlas is full. Take a rectangle back from an image no longer
1428 // on screen rather than dropping this draw — see
1429 // [`allocate_under_pressure`](Self::allocate_under_pressure), and
1430 // the module doc for why a hole is worse than a re-upload.
1431 Err(_) => self
1432 .allocate_under_pressure(width, height)
1433 .ok_or(no_space)?,
1434 };
1435
1436 let Some(resource) = self.cache.get(id) else {
1437 // A successful allocation always populates its slot; refusing here
1438 // keeps the frame path free of the `expect` the reference renderer
1439 // takes at this same point.
1440 return Err(no_space);
1441 };
1442
1443 let region = AtlasRegion {
1444 layer: resource.atlas_id.as_u32(),
1445 offset: resource.offsets(),
1446 size: resource.size(),
1447 };
1448 let may_have_transparency = pixels.may_have_transparency();
1449
1450 self.entries.insert(
1451 key,
1452 Entry {
1453 id,
1454 region,
1455 natural,
1456 may_have_transparency,
1457 last_seen: self.frame,
1458 },
1459 );
1460 self.uploads.push(ImageUpload {
1461 id,
1462 region,
1463 natural,
1464 pixels,
1465 may_have_transparency,
1466 });
1467
1468 Ok(ResidentImage {
1469 id,
1470 region,
1471 natural,
1472 padding: u32::from(ATLAS_PADDING),
1473 may_have_transparency,
1474 })
1475 }
1476
1477 /// Allocate `width` x `height` by taking rectangles back from resident
1478 /// images, least recently seen first, once the packer has refused.
1479 ///
1480 /// Answers the handle that finally fit, or `None` when no bounded set of
1481 /// candidates would have made room — which is what
1482 /// [`ImageSkip::NoAtlasSpace`] then reports, with the residency exactly as
1483 /// it was found.
1484 ///
1485 /// **Nothing is released until a plan that fits has been found.** An
1486 /// eviction that does not produce the allocation buys nothing and costs a
1487 /// clear plus a re-upload per rectangle, every frame the condition recurs
1488 /// (see the module doc). So the search runs over a *model* of the layers
1489 /// and commits only when the model says the request would then be placed:
1490 ///
1491 /// 1. **Candidates.** Only an entry last seen on an earlier frame. The
1492 /// frame path services [`evictions`](Self::evictions) *before*
1493 /// [`uploads`](Self::uploads), so freeing a rectangle this frame has
1494 /// already resolved would schedule a clear over texels the same frame is
1495 /// about to sample — painting the very hole this eviction exists to
1496 /// prevent. A frame whose own working set outgrows the atlas therefore
1497 /// still skips, because everything in the atlas belongs to it.
1498 /// 2. **Order.** `(last_seen, slot, key)`, least recently seen first, built
1499 /// and sorted *once* per refused allocation rather than once per
1500 /// eviction. The slot index breaks a tie between two entries last drawn
1501 /// on the same frame, so which image is displaced is a property of the
1502 /// residency rather than of the entry map's iteration order. Because the
1503 /// order is by age, an entry drawn on the *immediately preceding* frame
1504 /// is reached only after every staler one has already been planned —
1505 /// which is what keeps a large image redrawn every frame from thrashing
1506 /// a working set that is merely idle. The candidates are then walked one
1507 /// *layer* at a time, layers visited in that same staleness order (the
1508 /// layer whose stalest candidate sorts earliest goes first).
1509 /// 3. **Fit.** After each candidate joins the plan, the layer it sat on is
1510 /// re-tested: the request must fit inside the freed area that layer now
1511 /// models, and must clear every rectangle still occupying it
1512 /// ([`layer_admits`]). The first prefix that passes ends the search.
1513 /// 4. **Budget, per layer.** Each layer's own walk stops after
1514 /// [`MAX_PRESSURE_EVICTION_CANDIDATES`] of *that layer's* candidates, or
1515 /// once *that layer's* plan has freed [`PRESSURE_EVICTION_AREA_BUDGET`]
1516 /// times the requested area — a fresh allowance per layer rather than one
1517 /// pool the whole residency shares, so candidates on a layer too
1518 /// fragmented to ever admit the request cannot spend the allowance a
1519 /// later layer needed to prove it could. A layer that ends on its own
1520 /// budget contributes nothing and the walk moves to the next layer.
1521 /// 5. **Commit.** Only the planned rectangles *on the layer the fit was
1522 /// found in* are released — a candidate the walk passed over on another
1523 /// layer contributed nothing and is left resident.
1524 ///
1525 /// The model is optimistic about occupants it cannot see: the glyph policy
1526 /// packs through the same allocator (see the module doc) and its pages are
1527 /// not in this entry map, so [`layer_admits`] tests a subset of the real
1528 /// obstacles. That direction is the safe one — a placement clearing every
1529 /// real occupant clears the subset too, so a request eviction could satisfy
1530 /// on a *given layer* is never refused for that reason. The converse is
1531 /// possible: the packer may refuse a plan the model approved, which costs
1532 /// the plan's evictions and no allocation. It is bounded by the same
1533 /// per-layer budget as the plan itself, and it is the one residual the
1534 /// planner cannot rule out without knowing where every glyph page sits.
1535 fn allocate_under_pressure(&mut self, width: u32, height: u32) -> Option<ImageId> {
1536 // The overwhelmingly common refusal — a frame that filled the atlas
1537 // with its own draws — costs one pass over the entry map and no model
1538 // at all.
1539 if !self
1540 .entries
1541 .values()
1542 .any(|entry| entry.last_seen < self.frame)
1543 {
1544 return None;
1545 }
1546
1547 // Taken from the residency's own scratch, so the frame least able to
1548 // afford an allocation does not make one to evict with.
1549 let mut rects = std::mem::take(&mut self.pressure_rects);
1550 let mut order = std::mem::take(&mut self.pressure_order);
1551 let mut free_area = std::mem::take(&mut self.pressure_free_area);
1552 let mut layers = std::mem::take(&mut self.pressure_layers);
1553 let mut plan = std::mem::take(&mut self.pressure_plan);
1554
1555 self.model_layers(&mut rects, &mut order, &mut free_area);
1556 let fit = plan_pressure_eviction(
1557 width,
1558 height,
1559 self.budget.atlas_size,
1560 &mut rects,
1561 &order,
1562 &mut free_area,
1563 &mut layers,
1564 );
1565
1566 plan.clear();
1567 if let Some(layer) = fit {
1568 plan.extend(
1569 rects
1570 .iter()
1571 .filter(|rect| rect.freed && rect.layer == layer)
1572 .map(|rect| rect.key),
1573 );
1574 }
1575
1576 let allocated = if plan.is_empty() {
1577 None
1578 } else {
1579 for key in &plan {
1580 self.release(*key);
1581 self.pressure_evictions = self.pressure_evictions.saturating_add(1);
1582 self.frame_pressure_evictions = self.frame_pressure_evictions.saturating_add(1);
1583 }
1584 self.cache.allocate(width, height, ATLAS_PADDING).ok()
1585 };
1586
1587 rects.clear();
1588 order.clear();
1589 free_area.clear();
1590 layers.clear();
1591 plan.clear();
1592 self.pressure_rects = rects;
1593 self.pressure_order = order;
1594 self.pressure_free_area = free_area;
1595 self.pressure_layers = layers;
1596 self.pressure_plan = plan;
1597 allocated
1598 }
1599
1600 /// Fill the planner's scratch with this residency's own view of the atlas.
1601 ///
1602 /// `rects` takes one entry per resident image, grouped by layer so a fit
1603 /// test reads one layer's slice; `order` takes the candidate indices, least
1604 /// recently seen first across the *whole* residency — [`plan_pressure_eviction`]
1605 /// is what regroups it per layer for its own bounded walk; `free_area` takes
1606 /// each layer's free area *as the packer reports it*, which is the one
1607 /// figure that accounts for the glyph pages this map cannot see.
1608 fn model_layers(
1609 &self,
1610 rects: &mut Vec<PressureRect>,
1611 order: &mut Vec<usize>,
1612 free_area: &mut Vec<u64>,
1613 ) {
1614 rects.clear();
1615 rects.extend(self.entries.iter().map(|(key, entry)| {
1616 let region = padded(entry.region, ATLAS_PADDING);
1617 PressureRect {
1618 key: *key,
1619 layer: region.layer,
1620 x0: region.offset[0],
1621 y0: region.offset[1],
1622 x1: region.offset[0].saturating_add(region.size[0]),
1623 y1: region.offset[1].saturating_add(region.size[1]),
1624 area: u64::from(region.size[0]).saturating_mul(u64::from(region.size[1])),
1625 last_seen: entry.last_seen,
1626 slot: entry.id.as_u32(),
1627 freed: false,
1628 }
1629 }));
1630 rects.sort_unstable_by_key(|rect| rect.layer);
1631
1632 order.clear();
1633 order.extend(
1634 rects
1635 .iter()
1636 .enumerate()
1637 .filter_map(|(index, rect)| (rect.last_seen < self.frame).then_some(index)),
1638 );
1639 order.sort_unstable_by_key(|index| {
1640 let rect = &rects[*index];
1641 (rect.last_seen, rect.slot, rect.key)
1642 });
1643
1644 free_area.clear();
1645 free_area.resize(self.cache.atlas_count(), 0);
1646 for (id, stats) in self.cache.atlas_manager().atlas_stats() {
1647 if let Some(free) = free_area.get_mut(id.as_u32() as usize) {
1648 *free = u64::from(stats.total_area.saturating_sub(stats.allocated_area));
1649 }
1650 }
1651 }
1652
1653 /// Count and report one source downsampled to fit the atlas.
1654 ///
1655 /// Once per blob id, at warning level: a minified image is a silent quality
1656 /// decision the application may not have intended (a full-resolution photo
1657 /// where a thumbnail was wanted), so it says so — but a scene redrawing it
1658 /// every frame must not repeat the message.
1659 fn note_minify(&mut self, key: u64, natural: [u32; 2], fitted: [u32; 2]) {
1660 self.minified = self.minified.saturating_add(1);
1661 if !self.reported_minify.insert(key) {
1662 return;
1663 }
1664 log::warn!(
1665 "image {}x{} is larger than the {}x{} atlas budget; uploading a {}x{} box-filtered \
1666 copy instead (reported once per image)",
1667 natural[0],
1668 natural[1],
1669 self.budget.atlas_size.0,
1670 self.budget.atlas_size.1,
1671 fitted[0],
1672 fitted[1],
1673 );
1674 }
1675
1676 /// Deallocate every entry unseen for [`MAX_UNSEEN_FRAMES`], recording each
1677 /// one's rectangle for clearing.
1678 ///
1679 /// The age half of the two bounds; the population half is
1680 /// [`allocate_under_pressure`](Self::allocate_under_pressure), which runs
1681 /// only when an allocation has already failed. Keeping them separate is
1682 /// what makes an idle screen give its texels back promptly whether or not
1683 /// anything is asking for them.
1684 fn reap(&mut self) {
1685 let mut reaped = std::mem::take(&mut self.reaped);
1686 reaped.clear();
1687 reaped.extend(self.entries.iter().filter_map(|(key, entry)| {
1688 (self.frame.saturating_sub(entry.last_seen) > MAX_UNSEEN_FRAMES).then_some(*key)
1689 }));
1690
1691 for key in reaped.drain(..) {
1692 self.release(key);
1693 }
1694 self.reaped = reaped;
1695 }
1696
1697 /// Drop `key`'s entry, freeing its atlas rectangle and recording it for
1698 /// clearing.
1699 ///
1700 /// The padded rectangle is what is cleared: an allocation reserves its
1701 /// padding too, so leaving the border behind would leave stale texels a
1702 /// later allocation could sample through.
1703 ///
1704 /// An upload still pending for that rectangle goes with the entry. It
1705 /// describes pixels no live entry claims any more, and writing them after
1706 /// the clear would put an evicted image back into a rectangle the packer has
1707 /// already handed to something else.
1708 fn release(&mut self, key: u64) {
1709 let Some(entry) = self.entries.remove(&key) else {
1710 return;
1711 };
1712 // The handle came from this cache's own `allocate`, so the atlas it
1713 // names exists and the deallocation cannot fail.
1714 self.cache.deallocate(entry.id);
1715 self.uploads.retain(|upload| upload.id != entry.id);
1716
1717 // The plan survives an unacknowledged frame, so the same rectangle can
1718 // reach here twice before a single clear has been issued for it; one
1719 // clear is what the second one would write anyway.
1720 let region = padded(entry.region, ATLAS_PADDING);
1721 if self.eviction_set.insert(region) {
1722 self.evictions.push(region);
1723 }
1724 }
1725}
1726
1727/// The most rectangles one refused allocation may plan to take back from a
1728/// single atlas layer.
1729///
1730/// A bound on the *search*, not only on the damage: a layer whose walk
1731/// reaches it without finding a fit contributes nothing to the plan, so the
1732/// cost of one hopeless layer stays bounded rather than eating into the
1733/// allowance a later, satisfiable layer needs to prove itself. Sixty-four is
1734/// far past what any satisfiable request has needed — a fit is normally found
1735/// in one, since the rectangle handed back is usually the size of the one
1736/// being asked for — while still being a small fraction of what a
1737/// mobile-tier layer can hold. Applied once per layer a refused allocation
1738/// considers, never once across every layer together — see
1739/// [`plan_pressure_eviction`].
1740const MAX_PRESSURE_EVICTION_CANDIDATES: usize = 64;
1741
1742/// How much area one refused allocation may plan to free from a single atlas
1743/// layer, as a multiple of the area it asked for.
1744///
1745/// The other half of the bound, and the one that scales with the request: a
1746/// layer that has already given back four times the texels the request wants
1747/// and still has nowhere to put them is fighting fragmentation, not scarcity,
1748/// and freeing more of that same layer will not fix it. Applied per layer,
1749/// the same way [`MAX_PRESSURE_EVICTION_CANDIDATES`] is.
1750const PRESSURE_EVICTION_AREA_BUDGET: u64 = 4;
1751
1752/// One resident rectangle as the pressure planner sees it.
1753///
1754/// A flat copy rather than a borrow of the entry map, because the plan is drawn
1755/// up while the map is still intact and the release that follows needs it
1756/// mutably. `x1`/`y1` are exclusive, and every field is the *padded* allocation
1757/// — the rectangle the packer actually holds — rather than the image's own.
1758#[derive(Debug, Clone, Copy)]
1759struct PressureRect {
1760 /// Blob key of the entry holding this rectangle.
1761 key: u64,
1762 /// Array layer the rectangle sits in.
1763 layer: u32,
1764 /// Left edge, inclusive.
1765 x0: u32,
1766 /// Top edge, inclusive.
1767 y0: u32,
1768 /// Right edge, exclusive.
1769 x1: u32,
1770 /// Bottom edge, exclusive.
1771 y1: u32,
1772 /// Texels the rectangle covers.
1773 area: u64,
1774 /// The entry's frame stamp, which is what makes it a candidate or not.
1775 last_seen: u64,
1776 /// The entry's slot index, which breaks a tie between two entries last
1777 /// drawn on the same frame.
1778 slot: u32,
1779 /// Whether the plan under construction has given this rectangle back.
1780 freed: bool,
1781}
1782
1783/// The layer a bounded prefix of `order` would make room in, or `None`.
1784///
1785/// Frees nothing: it marks `rects` and grows `free_area` in the model only, so
1786/// a request no prefix satisfies leaves the residency untouched. `order` is
1787/// least-recently-seen-first across the *whole* residency; this regroups it
1788/// one layer at a time — `layers` scratch, cleared and refilled here, in the
1789/// order each layer's stalest candidate appears in `order` — and gives each
1790/// layer its own [`MAX_PRESSURE_EVICTION_CANDIDATES`]/
1791/// [`PRESSURE_EVICTION_AREA_BUDGET`] allowance rather than pooling one across
1792/// every layer. Without that split, a layer with many candidates that could
1793/// never admit the request (too fragmented, or blocked by an occupant no
1794/// eviction touches) spends the whole budget before a later, satisfiable
1795/// layer is ever tried — the request is then refused even though a bounded
1796/// eviction on that later layer would have placed it. See
1797/// [`ImageResidency::allocate_under_pressure`] for the whole shape and for why
1798/// the answer is a layer rather than a bool — only the planned rectangles on
1799/// the fitting layer are worth releasing.
1800fn plan_pressure_eviction(
1801 width: u32,
1802 height: u32,
1803 atlas_size: (u32, u32),
1804 rects: &mut [PressureRect],
1805 order: &[usize],
1806 free_area: &mut [u64],
1807 layers: &mut Vec<u32>,
1808) -> Option<u32> {
1809 let padding = u32::from(ATLAS_PADDING).saturating_mul(2);
1810 let request_w = width.saturating_add(padding);
1811 let request_h = height.saturating_add(padding);
1812 let request_area = u64::from(request_w).saturating_mul(u64::from(request_h));
1813 let area_budget = request_area.saturating_mul(PRESSURE_EVICTION_AREA_BUDGET);
1814
1815 layers.clear();
1816 for &index in order {
1817 let Some(rect) = rects.get(index) else {
1818 continue;
1819 };
1820 if !layers.contains(&rect.layer) {
1821 layers.push(rect.layer);
1822 }
1823 }
1824
1825 for &layer in layers.iter() {
1826 let mut layer_freed_area = 0_u64;
1827 let mut step = 0_usize;
1828
1829 for &index in order {
1830 let on_this_layer = rects.get(index).is_some_and(|rect| rect.layer == layer);
1831 if !on_this_layer {
1832 continue;
1833 }
1834 if step >= MAX_PRESSURE_EVICTION_CANDIDATES || layer_freed_area > area_budget {
1835 break;
1836 }
1837 step += 1;
1838
1839 let area = {
1840 let Some(rect) = rects.get_mut(index) else {
1841 continue;
1842 };
1843 rect.freed = true;
1844 rect.area
1845 };
1846 layer_freed_area = layer_freed_area.saturating_add(area);
1847
1848 let Some(free) = free_area.get_mut(layer as usize) else {
1849 continue;
1850 };
1851 *free = free.saturating_add(area);
1852 // Area first because it is exact — it comes off the packer's own
1853 // accounting, so it counts the glyph pages `layer_admits` cannot see —
1854 // and because it is the cheaper of the two questions.
1855 if *free >= request_area && layer_admits(rects, layer, request_w, request_h, atlas_size)
1856 {
1857 return Some(layer);
1858 }
1859 }
1860 }
1861
1862 None
1863}
1864
1865/// Whether a `width` x `height` rectangle could still be placed in `layer` once
1866/// every rectangle the plan has marked freed is out of the way.
1867///
1868/// A *necessary* condition rather than an exact packing answer. Any placement
1869/// has to avoid each occupant whole, so it must lie entirely to one side of it:
1870/// if some occupant leaves less than the requested extent on all four sides,
1871/// there is nowhere for the rectangle to go and the plan cannot succeed. A plan
1872/// this refuses is therefore certainly hopeless; one it admits may still be
1873/// refused by the packer, which is the direction that costs a bounded number of
1874/// evictions rather than a wrong picture (see
1875/// [`ImageResidency::allocate_under_pressure`]).
1876///
1877/// `rects` is grouped by layer, so this reads one layer's slice rather than the
1878/// whole residency.
1879fn layer_admits(
1880 rects: &[PressureRect],
1881 layer: u32,
1882 width: u32,
1883 height: u32,
1884 atlas_size: (u32, u32),
1885) -> bool {
1886 if width > atlas_size.0 || height > atlas_size.1 {
1887 return false;
1888 }
1889
1890 let start = rects.partition_point(|rect| rect.layer < layer);
1891 let end = rects.partition_point(|rect| rect.layer <= layer);
1892 rects[start..end]
1893 .iter()
1894 .filter(|rect| !rect.freed)
1895 .all(|rect| {
1896 width <= rect.x0
1897 || width <= atlas_size.0.saturating_sub(rect.x1)
1898 || height <= rect.y0
1899 || height <= atlas_size.1.saturating_sub(rect.y1)
1900 })
1901}
1902
1903/// The extent `natural` is stored at inside an `atlas`-sized layer: itself when
1904/// it already fits, and otherwise the largest rectangle of the same aspect ratio
1905/// that does.
1906///
1907/// The scale is taken as the smaller of the two axis ratios and applied to both,
1908/// so the two axes of the paint transform stay in step — a per-axis fit would
1909/// stretch the image. Each axis is floored (never rounding *up* past the layer)
1910/// and then held at one texel: a source that scales below a whole texel still
1911/// has to have somewhere to be, and a zero-extent allocation is not a rectangle
1912/// the packer or the shader can address.
1913#[must_use]
1914pub fn fit_extent(natural: [u32; 2], atlas: (u32, u32)) -> (u32, u32) {
1915 let (width, height) = (natural[0], natural[1]);
1916 if width <= atlas.0 && height <= atlas.1 {
1917 return (width, height);
1918 }
1919
1920 let scale = (f64::from(atlas.0) / f64::from(width.max(1)))
1921 .min(f64::from(atlas.1) / f64::from(height.max(1)));
1922 let axis = |value: u32, ceiling: u32| {
1923 let scaled = (f64::from(value) * scale).floor();
1924 // `as` on a non-finite or out-of-range float saturates in Rust, so the
1925 // clamps below are the whole check rather than a second line of defence.
1926 (scaled as u32).clamp(1, ceiling.max(1))
1927 };
1928
1929 (axis(width, atlas.0), axis(height, atlas.1))
1930}
1931
1932/// `source` box-filtered down to `width` x `height`.
1933///
1934/// Every output texel is the unweighted mean of the half-open source rectangle
1935/// it covers, computed on the premultiplied bytes the atlas stores — which is
1936/// the space the samples are composited in, so averaging there is what keeps a
1937/// partially transparent source from bleeding its colour outward.
1938///
1939/// Minification only: a request at or above the source's own extent copies the
1940/// texels straight across rather than inventing any, since a box filter has no
1941/// magnification arm and the caller never asks for one ([`fit_extent`] only ever
1942/// shrinks).
1943#[must_use]
1944pub fn minify(source: &Pixmap, width: u32, height: u32) -> Pixmap {
1945 let source_w = u32::from(source.width());
1946 let source_h = u32::from(source.height());
1947 let width = width.clamp(1, source_w.max(1));
1948 let height = height.clamp(1, source_h.max(1));
1949
1950 let texels = source.data();
1951 let mut out = Vec::with_capacity((width as usize).saturating_mul(height as usize));
1952 let mut may_have_transparency = false;
1953
1954 for y in 0..height {
1955 // Half-open source rows, derived from the output row rather than
1956 // accumulated, so rounding never leaves a gap or an overlap between
1957 // consecutive rows.
1958 let y0 = (u64::from(y) * u64::from(source_h) / u64::from(height)) as u32;
1959 let y1 = ((u64::from(y) + 1) * u64::from(source_h) / u64::from(height)) as u32;
1960 let y1 = y1.max(y0.saturating_add(1)).min(source_h);
1961
1962 for x in 0..width {
1963 let x0 = (u64::from(x) * u64::from(source_w) / u64::from(width)) as u32;
1964 let x1 = ((u64::from(x) + 1) * u64::from(source_w) / u64::from(width)) as u32;
1965 let x1 = x1.max(x0.saturating_add(1)).min(source_w);
1966
1967 let mut sum = [0_u64; 4];
1968 let mut count = 0_u64;
1969 for row in y0..y1 {
1970 let base = (row as usize).saturating_mul(source_w as usize);
1971 for column in x0..x1 {
1972 let Some(texel) = texels.get(base.saturating_add(column as usize)) else {
1973 continue;
1974 };
1975 sum[0] += u64::from(texel.r);
1976 sum[1] += u64::from(texel.g);
1977 sum[2] += u64::from(texel.b);
1978 sum[3] += u64::from(texel.a);
1979 count += 1;
1980 }
1981 }
1982
1983 // A count of zero is unreachable — both ranges are non-empty by
1984 // construction — and flooring it at one rather than branching keeps
1985 // the divide-by-zero panic off the frame path (E17) while answering
1986 // a transparent texel if it ever were reachable, since the sums
1987 // would then be zero too.
1988 let divisor = count.max(1);
1989 // Round to nearest rather than truncating: a uniform source has to
1990 // come back bit-identical, which truncating an exact integer mean
1991 // already gives, but a near-uniform one must not drift a level
1992 // darker.
1993 let mean = |channel: usize| {
1994 ((sum[channel] + divisor / 2) / divisor).min(u64::from(u8::MAX)) as u8
1995 };
1996
1997 let texel = PremulRgba8 {
1998 r: mean(0),
1999 g: mean(1),
2000 b: mean(2),
2001 a: mean(3),
2002 };
2003 may_have_transparency |= texel.a != u8::MAX;
2004 out.push(texel);
2005 }
2006 }
2007
2008 // Both extents were clamped into `1..=u16::MAX` above (a source pixmap's own
2009 // extents are `u16`), so neither conversion can lose information.
2010 Pixmap::from_parts_with_opacity(
2011 out,
2012 width.min(u32::from(u16::MAX)) as u16,
2013 height.min(u32::from(u16::MAX)) as u16,
2014 may_have_transparency,
2015 )
2016}
2017
2018/// `axis` held at or below `ceiling`, and never below the one texel an
2019/// addressable rectangle needs.
2020fn fit_axis(axis: u32, ceiling: u64) -> u32 {
2021 axis.min(u32::try_from(ceiling).unwrap_or(u32::MAX)).max(1)
2022}
2023
2024/// `region` grown by `padding` on every side, clamped at the layer origin.
2025fn padded(region: AtlasRegion, padding: u16) -> AtlasRegion {
2026 let padding = u32::from(padding);
2027 let pad_x = padding.min(region.offset[0]);
2028 let pad_y = padding.min(region.offset[1]);
2029
2030 AtlasRegion {
2031 layer: region.layer,
2032 offset: [region.offset[0] - pad_x, region.offset[1] - pad_y],
2033 size: [
2034 region.size[0].saturating_add(pad_x + padding),
2035 region.size[1].saturating_add(pad_y + padding),
2036 ],
2037 }
2038}
2039
2040/// Refuse an image `vello_common`'s own conversion would assert on.
2041///
2042/// The three conditions are exactly the three assertions downstream: the
2043/// format `unimplemented!()`, the `u16::MAX` dimension check, and
2044/// `Pixmap::from_parts_with_opacity`'s length equality. Checking them here is
2045/// what keeps the panic off the frame path.
2046fn check_supported(data: &ImageData) -> Result<(), ImageSkip> {
2047 match data.format {
2048 ImageFormat::Rgba8 | ImageFormat::Bgra8 => {}
2049 format => return Err(ImageSkip::UnsupportedFormat(format)),
2050 }
2051
2052 if data.width == 0 || data.height == 0 {
2053 return Err(ImageSkip::DegenerateExtent {
2054 width: data.width,
2055 height: data.height,
2056 });
2057 }
2058
2059 if data.width > MAX_IMAGE_DIMENSION || data.height > MAX_IMAGE_DIMENSION {
2060 return Err(ImageSkip::TooLarge {
2061 width: data.width,
2062 height: data.height,
2063 max: MAX_IMAGE_DIMENSION,
2064 });
2065 }
2066
2067 let actual = data.data.data().len();
2068 let expected = data
2069 .format
2070 .size_in_bytes(data.width, data.height)
2071 .unwrap_or(usize::MAX);
2072 if actual != expected {
2073 return Err(ImageSkip::MalformedPixels {
2074 width: data.width,
2075 height: data.height,
2076 expected,
2077 actual,
2078 });
2079 }
2080
2081 Ok(())
2082}
2083
2084#[cfg(test)]
2085mod tests {
2086 use super::*;
2087 use peniko::{Blob, ImageAlphaType};
2088
2089 fn image(width: u32, height: u32) -> ImageData {
2090 let len = (width as usize) * (height as usize) * 4;
2091 ImageData {
2092 data: Blob::new(Arc::new(vec![255_u8; len])),
2093 format: ImageFormat::Rgba8,
2094 alpha_type: ImageAlphaType::Alpha,
2095 width,
2096 height,
2097 }
2098 }
2099
2100 fn residency() -> ImageResidency {
2101 ImageResidency::new(AtlasBudget {
2102 atlas_size: (64, 64),
2103 max_atlases: 2,
2104 })
2105 }
2106
2107 #[test]
2108 fn a_mobile_budget_is_never_the_vello_default() {
2109 let default = AtlasConfig::default();
2110 for budget in [AtlasBudget::MOBILE, AtlasBudget::DESKTOP] {
2111 assert_ne!(budget.atlas_size, default.atlas_size);
2112 assert!(budget.total_bytes() < 4096 * 4096 * ATLAS_FORMAT_BYTES * 8);
2113 }
2114 assert_eq!(AtlasBudget::MOBILE.total_bytes(), 16 << 20);
2115 assert_eq!(AtlasBudget::DESKTOP.total_bytes(), 128 << 20);
2116 }
2117
2118 #[test]
2119 fn a_downlevel_adapter_takes_the_mobile_budget() {
2120 let webgl2 = TierCaps::fake(DownlevelProfile::WebGl2);
2121 assert!(is_mobile_tier(&webgl2));
2122
2123 let mut tiler = TierCaps::fake(DownlevelProfile::Full);
2124 assert!(!is_mobile_tier(&tiler));
2125 tiler.transient_saves_memory = true;
2126 assert!(is_mobile_tier(&tiler));
2127 }
2128
2129 fn named(name: &str) -> TierCaps {
2130 let mut caps = TierCaps::fake(DownlevelProfile::Full);
2131 caps.adapter_name = name.to_string();
2132 caps
2133 }
2134
2135 #[test]
2136 fn the_adapter_table_names_the_mobile_and_desktop_families() {
2137 for name in [
2138 "Adreno (TM) 618",
2139 "Adreno (TM) 660",
2140 "Mali-G78 MP14",
2141 "Immortalis-G715",
2142 "PowerVR B-Series BXM-8-256",
2143 "Samsung Xclipse 920",
2144 "Maleoon 910",
2145 "Apple A15 GPU",
2146 "NVIDIA Tegra X1",
2147 ] {
2148 assert_eq!(adapter_family(name), Some(AdapterFamily::Mobile), "{name}");
2149 }
2150 for name in [
2151 "NVIDIA T400",
2152 "NVIDIA GeForce RTX 4090",
2153 "Quadro P1000",
2154 "AMD Radeon RX 7900 XTX",
2155 "Intel(R) UHD Graphics 770",
2156 ] {
2157 assert_eq!(adapter_family(name), Some(AdapterFamily::Desktop), "{name}");
2158 }
2159 for name in ["llvmpipe (LLVM 19.1.7, 256 bits)", "fake-full", ""] {
2160 assert_eq!(adapter_family(name), None, "{name}");
2161 }
2162 }
2163
2164 #[test]
2165 fn the_tier_table_overrides_a_mobile_driver_that_answers_like_a_desktop() {
2166 const GIB: u64 = 1 << 30;
2167 // Adreno 618/620 answer full profile and no transient savings — the
2168 // desktop answers — and used to take the desktop budget on them.
2169 let adreno = named("Adreno (TM) 618");
2170 assert!(!is_mobile_tier(&adreno));
2171
2172 let small = choose_tier(&adreno, Some(3 * GIB));
2173 assert_eq!(
2174 (small.tier, small.rule),
2175 (AtlasTier::Mobile, TierRule::Table)
2176 );
2177 assert_eq!(AtlasBudget::for_choice(small, &adreno), AtlasBudget::MOBILE);
2178
2179 let unknown = choose_tier(&adreno, None);
2180 assert_eq!(
2181 unknown.tier,
2182 AtlasTier::Mobile,
2183 "unknown memory earns nothing"
2184 );
2185
2186 let large = choose_tier(&adreno, Some(6 * GIB));
2187 assert_eq!(
2188 (large.tier, large.rule),
2189 (AtlasTier::MobileLarge, TierRule::Table)
2190 );
2191 assert_eq!(
2192 AtlasBudget::for_choice(large, &adreno),
2193 AtlasBudget::MOBILE_LARGE_WIDE
2194 );
2195 assert_eq!(
2196 AtlasBudget::for_choice(choose_tier(&adreno, Some(MOBILE_LARGE_MIN_MEMORY)), &adreno),
2197 AtlasBudget::MOBILE_LARGE_WIDE,
2198 "the threshold is inclusive"
2199 );
2200 assert_eq!(AtlasBudget::MOBILE_LARGE_WIDE.total_bytes(), 64 << 20);
2201 assert_eq!(AtlasBudget::MOBILE_LARGE_DEEP.total_bytes(), 64 << 20);
2202
2203 // A desktop family stays on the desktop budget whatever it answers.
2204 let mut nvidia = named("NVIDIA T400");
2205 nvidia.transient_saves_memory = true;
2206 let desktop = choose_tier(&nvidia, Some(GIB));
2207 assert_eq!(
2208 (desktop.tier, desktop.rule),
2209 (AtlasTier::Desktop, TierRule::Table)
2210 );
2211 }
2212
2213 #[test]
2214 fn the_large_mobile_shape_is_the_one_the_adapter_can_create() {
2215 let mut caps = named("Mali-G78 MP14");
2216 assert_eq!(
2217 AtlasBudget::mobile_large(&caps),
2218 Some(AtlasBudget::MOBILE_LARGE_WIDE)
2219 );
2220
2221 // An edge short of 2048 takes the deep shape instead.
2222 caps.max_texture_dimension_2d = 1024;
2223 assert_eq!(
2224 AtlasBudget::mobile_large(&caps),
2225 Some(AtlasBudget::MOBILE_LARGE_DEEP)
2226 );
2227
2228 // Neither shape fits: the tier is the plain mobile one, never a large
2229 // budget clamped below it.
2230 caps.max_texture_array_layers = 8;
2231 assert_eq!(AtlasBudget::mobile_large(&caps), None);
2232 let choice = choose_tier(&caps, Some(16 << 30));
2233 assert_eq!(choice.tier, AtlasTier::Mobile);
2234 assert_eq!(AtlasBudget::for_choice(choice, &caps), AtlasBudget::MOBILE);
2235 }
2236
2237 #[test]
2238 fn an_unnamed_adapter_keeps_the_capability_read_and_downlevel_wins_over_the_table() {
2239 let unnamed = TierCaps::fake(DownlevelProfile::Full);
2240 let choice = choose_tier(&unnamed, Some(64 << 30));
2241 assert_eq!(
2242 (choice.tier, choice.rule),
2243 (AtlasTier::Desktop, TierRule::Heuristic)
2244 );
2245
2246 let mut tiler = unnamed.clone();
2247 tiler.transient_saves_memory = true;
2248 let choice = choose_tier(&tiler, Some(64 << 30));
2249 assert_eq!(
2250 (choice.tier, choice.rule),
2251 (AtlasTier::Mobile, TierRule::Heuristic),
2252 "the heuristic never reaches the large tier"
2253 );
2254
2255 let mut gl_nvidia = TierCaps::fake(DownlevelProfile::WebGl2);
2256 gl_nvidia.adapter_name = "ANGLE (NVIDIA, NVIDIA T400)".to_string();
2257 let choice = choose_tier(&gl_nvidia, Some(64 << 30));
2258 assert_eq!(
2259 (choice.tier, choice.rule),
2260 (AtlasTier::Mobile, TierRule::Downlevel)
2261 );
2262 }
2263
2264 #[test]
2265 fn mem_total_is_read_in_bytes_from_meminfo() {
2266 let meminfo = "MemTotal: 5768520 kB\nMemFree: 123456 kB\n";
2267 assert_eq!(parse_mem_total(meminfo), Some(5_768_520 * 1024));
2268 assert_eq!(parse_mem_total("MemFree: 1 kB\n"), None);
2269 assert_eq!(parse_mem_total("MemTotal: lots\n"), None);
2270 assert_eq!(parse_mem_total("MemTotal: 1024 MB\n"), None);
2271 }
2272
2273 #[test]
2274 fn a_budget_is_clamped_to_the_adapters_own_ceilings() {
2275 let mut caps = TierCaps::fake(DownlevelProfile::Full);
2276 caps.max_texture_dimension_2d = 1024;
2277 caps.max_texture_array_layers = 2;
2278
2279 let budget = AtlasBudget::DESKTOP.clamped(&caps);
2280 assert_eq!(budget.atlas_size, (1024, 1024));
2281 assert_eq!(budget.max_atlases, 2);
2282
2283 // The eight-bit `atlas_index` field is a ceiling of its own.
2284 caps.max_texture_array_layers = 4096;
2285 let budget = AtlasBudget {
2286 atlas_size: (256, 256),
2287 max_atlases: 4096,
2288 }
2289 .clamped(&caps);
2290 assert_eq!(budget.max_atlases, MAX_ATLAS_LAYERS);
2291 }
2292
2293 #[test]
2294 fn the_first_layer_is_created_lazily() {
2295 let residency = residency();
2296 assert_eq!(residency.layers(), 0);
2297 assert_eq!(residency.entry_count(), 0);
2298 }
2299
2300 #[test]
2301 fn one_image_over_many_frames_uploads_exactly_once() {
2302 let mut residency = residency();
2303 let data = image(8, 8);
2304 let mut uploads = 0;
2305
2306 for _ in 0..60 {
2307 residency.begin_frame();
2308 residency.resolve(&data).expect("8x8 fits a 64x64 atlas");
2309 uploads += residency.uploads().len();
2310 // Every frame here is a frame that reached the atlas.
2311 residency.acknowledge_plan();
2312 }
2313
2314 assert_eq!(uploads, 1, "residency survives every frame that draws it");
2315 assert_eq!(residency.entry_count(), 1);
2316 assert_eq!(residency.layers(), 1);
2317 }
2318
2319 #[test]
2320 fn an_unseen_image_is_reaped_and_its_padded_region_cleared() {
2321 let mut residency = residency();
2322 let data = image(8, 8);
2323
2324 residency.begin_frame();
2325 let resident = residency.resolve(&data).expect("fits");
2326 residency.acknowledge_plan();
2327
2328 for _ in 0..MAX_UNSEEN_FRAMES {
2329 residency.begin_frame();
2330 assert!(residency.evictions().is_empty(), "still inside the window");
2331 }
2332
2333 residency.begin_frame();
2334 assert_eq!(residency.entry_count(), 0);
2335 assert_eq!(
2336 residency.evictions(),
2337 &[padded(resident.region, ATLAS_PADDING)]
2338 );
2339
2340 // The freed rectangle is available again, and the reallocation
2341 // schedules a fresh upload.
2342 residency.resolve(&data).expect("fits");
2343 assert_eq!(residency.uploads().len(), 1);
2344 }
2345
2346 #[test]
2347 fn an_image_larger_than_the_atlas_is_minified_to_fit_rather_than_skipped() {
2348 let mut residency = residency();
2349 residency.begin_frame();
2350
2351 // Twice the layer's width at a 16:1 aspect ratio: the width sets the
2352 // scale, and the height follows it rather than being fitted on its own.
2353 let resident = residency.resolve(&image(128, 8)).expect("minified to fit");
2354
2355 assert_eq!(resident.region.size, [64, 4]);
2356 assert_eq!(resident.natural, [128, 8]);
2357 assert_eq!(resident.minify_scale(), Some((0.5, 0.5)));
2358 assert_eq!(residency.minified(), 1);
2359 assert_eq!(residency.skipped(), 0, "a big image is not a skipped one");
2360 assert_eq!(residency.uploads().len(), 1);
2361 let upload = &residency.uploads()[0];
2362 assert_eq!(
2363 (upload.pixels.width(), upload.pixels.height()),
2364 (64, 4),
2365 "the upload carries exactly the region's texels"
2366 );
2367 }
2368
2369 #[test]
2370 fn an_atlas_with_no_room_left_is_still_a_skip() {
2371 // Minification fits an image to a LAYER, not to the space left in one:
2372 // a budget already full refuses the next allocation as it always did.
2373 let mut residency = ImageResidency::new(AtlasBudget {
2374 atlas_size: (64, 64),
2375 max_atlases: 1,
2376 });
2377 residency.begin_frame();
2378 residency
2379 .resolve(&image(64, 64))
2380 .expect("fills the one layer");
2381
2382 let skip = residency
2383 .resolve(&image(32, 32))
2384 .expect_err("nothing is left to allocate from");
2385 assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
2386 assert_eq!(residency.skipped(), 1);
2387 assert_eq!(residency.minified(), 0);
2388 }
2389
2390 #[test]
2391 fn a_minified_image_stays_resident_across_frames_at_its_declared_extent() {
2392 // Residency is keyed on the SOURCE extent, not the resident one: an
2393 // entry matched against its own minified rectangle would miss every
2394 // frame and re-upload the image on each of them.
2395 let mut residency = residency();
2396 let data = image(128, 8);
2397
2398 for _ in 0..8 {
2399 residency.begin_frame();
2400 let resident = residency.resolve(&data).expect("minified to fit");
2401 assert_eq!(resident.natural, [128, 8]);
2402 residency.acknowledge_plan();
2403 }
2404
2405 assert_eq!(residency.entry_count(), 1);
2406 assert_eq!(
2407 residency.minified(),
2408 1,
2409 "downsampled once, not once a frame"
2410 );
2411 assert!(
2412 residency.uploads().is_empty(),
2413 "no re-upload after the first"
2414 );
2415 }
2416
2417 #[test]
2418 fn a_fit_extent_preserves_the_aspect_ratio_and_never_grows() {
2419 let atlas = (64, 64);
2420 // Already inside the layer: unchanged, both axes.
2421 assert_eq!(fit_extent([64, 64], atlas), (64, 64));
2422 assert_eq!(fit_extent([8, 4], atlas), (8, 4));
2423
2424 // The `adv-huge-image` shape, at this budget's scale.
2425 assert_eq!(fit_extent([5000, 5000], (2048, 2048)), (2048, 2048));
2426
2427 // The tighter axis sets the scale for both.
2428 assert_eq!(fit_extent([128, 8], atlas), (64, 4));
2429 assert_eq!(fit_extent([8, 128], atlas), (4, 64));
2430 assert_eq!(fit_extent([1000, 10], atlas), (64, 1));
2431
2432 // A non-square layer is fitted per axis and still uniformly scaled.
2433 assert_eq!(fit_extent([400, 400], (100, 50)), (50, 50));
2434 }
2435
2436 #[test]
2437 fn an_extreme_aspect_ratio_still_fits_at_one_texel_rather_than_none() {
2438 // 10000:1 into a 64-square layer scales the short axis to 0.0064 texels.
2439 // A zero extent is not a rectangle the packer or the shader can
2440 // address, so it is held at one.
2441 let fitted = fit_extent([10_000, 1], (64, 64));
2442 assert_eq!(fitted, (64, 1));
2443
2444 let mut residency = residency();
2445 residency.begin_frame();
2446 let resident = residency
2447 .resolve(&image(10_000, 1))
2448 .expect("a sliver still becomes a rectangle");
2449 assert_eq!(resident.region.size, [64, 1]);
2450 }
2451
2452 #[test]
2453 fn a_uniform_source_minifies_to_exactly_its_own_colour() {
2454 // What `adv-huge-image` rests on: a uniform source comes back bit
2455 // identical whatever the scale, so the case's `Exact` probe is a
2456 // property of the filter rather than a tolerance.
2457 let texel = PremulRgba8 {
2458 r: 255,
2459 g: 0,
2460 b: 255,
2461 a: 255,
2462 };
2463 let source = Pixmap::from_parts_with_opacity(vec![texel; 100 * 100], 100, 100, false);
2464
2465 let small = minify(&source, 7, 3);
2466 assert_eq!((small.width(), small.height()), (7, 3));
2467 assert!(small.data().iter().all(|out| *out == texel));
2468 assert!(
2469 !small.may_have_transparency(),
2470 "an opaque source stays opaque"
2471 );
2472 }
2473
2474 #[test]
2475 fn a_box_filter_averages_the_source_texels_each_output_covers() {
2476 // A 2x1 source of black and white halves down to one texel, which must
2477 // be their mean rather than either end.
2478 let black = PremulRgba8 {
2479 r: 0,
2480 g: 0,
2481 b: 0,
2482 a: 255,
2483 };
2484 let white = PremulRgba8 {
2485 r: 255,
2486 g: 255,
2487 b: 255,
2488 a: 255,
2489 };
2490 let source = Pixmap::from_parts_with_opacity(vec![black, white], 2, 1, false);
2491
2492 let small = minify(&source, 1, 1);
2493 assert_eq!((small.width(), small.height()), (1, 1));
2494 // 255 / 2 rounded to nearest.
2495 assert_eq!(small.data()[0].r, 128);
2496 assert_eq!(small.data()[0].a, 255);
2497
2498 // Transparency is recomputed from the result, not inherited: averaging
2499 // an opaque and a fully transparent texel produces a partial one.
2500 let clear = PremulRgba8 {
2501 r: 0,
2502 g: 0,
2503 b: 0,
2504 a: 0,
2505 };
2506 let mixed = Pixmap::from_parts_with_opacity(vec![white, clear], 2, 1, true);
2507 let small = minify(&mixed, 1, 1);
2508 assert_eq!(small.data()[0].a, 128);
2509 assert!(small.may_have_transparency());
2510 }
2511
2512 #[test]
2513 fn minify_never_magnifies() {
2514 let texel = PremulRgba8 {
2515 r: 1,
2516 g: 2,
2517 b: 3,
2518 a: 255,
2519 };
2520 let source = Pixmap::from_parts_with_opacity(vec![texel; 4], 2, 2, false);
2521
2522 // A request above the source's own extent is clamped to it rather than
2523 // inventing texels the box filter has no arm for.
2524 let same = minify(&source, 8, 8);
2525 assert_eq!((same.width(), same.height()), (2, 2));
2526 assert!(same.data().iter().all(|out| *out == texel));
2527 }
2528
2529 #[test]
2530 fn a_malformed_or_degenerate_image_is_refused_before_the_conversion() {
2531 let mut short = image(4, 4);
2532 short.data = Blob::new(Arc::new(vec![0_u8; 8]));
2533 assert!(matches!(
2534 check_supported(&short),
2535 Err(ImageSkip::MalformedPixels { .. })
2536 ));
2537
2538 assert!(matches!(
2539 check_supported(&image(0, 4)),
2540 Err(ImageSkip::DegenerateExtent { .. })
2541 ));
2542
2543 let mut huge = image(1, 1);
2544 huge.width = MAX_IMAGE_DIMENSION + 1;
2545 assert!(matches!(
2546 check_supported(&huge),
2547 Err(ImageSkip::TooLarge { .. })
2548 ));
2549 }
2550
2551 #[test]
2552 fn two_images_share_a_layer_and_hold_distinct_rectangles() {
2553 let mut residency = residency();
2554 residency.begin_frame();
2555
2556 let first = residency.resolve(&image(8, 8)).expect("fits");
2557 let second = residency.resolve(&image(16, 16)).expect("fits");
2558
2559 assert_ne!(first.id, second.id);
2560 assert_eq!(first.region.layer, second.region.layer);
2561 assert_ne!(first.region.offset, second.region.offset);
2562 assert_eq!(residency.uploads().len(), 2);
2563 assert_eq!(residency.layers(), 1);
2564 }
2565
2566 #[test]
2567 fn a_second_layer_is_created_when_the_first_is_full() {
2568 let mut residency = residency();
2569 residency.begin_frame();
2570
2571 // Two 64x48 images cannot share one 64x64 layer.
2572 let first = residency.resolve(&image(64, 48)).expect("fits a layer");
2573 let second = residency.resolve(&image(64, 48)).expect("fits the next");
2574
2575 assert_eq!(first.region.layer, 0);
2576 assert_eq!(second.region.layer, 1);
2577 assert_eq!(residency.layers(), 2);
2578 }
2579
2580 #[test]
2581 fn a_slot_taken_through_the_shared_allocator_is_a_distinct_id_on_the_same_budget() {
2582 // The allocator half of the glyph/image seam, stated without the glyph
2583 // policy: whatever else packs into this cache draws from the same id
2584 // space and the same layer allowance, and its pages are visible in the
2585 // depth the array is built to.
2586 let mut residency = residency();
2587 residency.begin_frame();
2588 let resident = residency.resolve(&image(64, 48)).expect("fills a layer");
2589
2590 // Big enough that it cannot share the layer the image took, so it must
2591 // create the second one rather than report the first.
2592 let borrowed = residency
2593 .allocator_mut()
2594 .allocate(64, 48, ATLAS_PADDING)
2595 .expect("the second layer is free");
2596
2597 assert_ne!(
2598 resident.id, borrowed,
2599 "one cache never hands the same slot index to two live occupants"
2600 );
2601 assert_eq!(
2602 residency.layers(),
2603 2,
2604 "a layer created by the other class still has to be in the array"
2605 );
2606 assert_eq!(
2607 residency
2608 .allocator()
2609 .get(resident.id)
2610 .map(|resource| resource.size()),
2611 Some(resident.region.size),
2612 "and neither allocation disturbed the other's rectangle"
2613 );
2614 }
2615
2616 #[test]
2617 fn a_disabled_residency_refuses_every_image_and_touches_no_atlas() {
2618 let mut residency = ImageResidency::disabled(AtlasBudget::MOBILE);
2619 assert!(residency.is_disabled());
2620
2621 residency.begin_frame();
2622 let skip = residency
2623 .resolve(&image(8, 8))
2624 .expect_err("the kill switch refuses every image");
2625
2626 assert_eq!(skip, ImageSkip::AtlasDisabled);
2627 assert_eq!(residency.entry_count(), 0);
2628 assert_eq!(residency.layers(), 0, "no atlas layer is ever created");
2629 assert!(residency.uploads().is_empty());
2630 assert_eq!(residency.skipped(), 1);
2631 }
2632
2633 #[test]
2634 fn an_enabled_residency_is_what_an_unset_kill_switch_produces() {
2635 // The default test environment leaves `FRUST_ENGINE_NO_ATLAS` unset, so
2636 // this pins that `new` consults the switch rather than hardcoding
2637 // either answer; the disabled half is pinned above through the
2638 // constructor the switch selects.
2639 assert!(!ImageResidency::new(AtlasBudget::MOBILE).is_disabled());
2640 }
2641
2642 #[test]
2643 fn reading_the_plan_leaves_it_pending_and_acknowledging_it_clears_it() {
2644 let mut residency = residency();
2645 residency.begin_frame();
2646 residency.resolve(&image(8, 8)).expect("fits");
2647
2648 let (evictions, uploads) = residency.plan();
2649 assert!(evictions.is_empty());
2650 assert_eq!(uploads.len(), 1);
2651 assert!(
2652 residency.has_pending_plan(),
2653 "reading the plan is not servicing it"
2654 );
2655
2656 residency.acknowledge_plan();
2657 assert!(!residency.has_pending_plan());
2658 assert!(residency.uploads().is_empty());
2659 assert!(residency.evictions().is_empty());
2660 }
2661
2662 #[test]
2663 fn a_clear_is_dropped_only_for_a_rectangle_an_upload_covers_exactly() {
2664 let region = |layer, x, y, w, h| AtlasRegion {
2665 layer,
2666 offset: [x, y],
2667 size: [w, h],
2668 };
2669 let upload = |region| ImageUpload {
2670 id: ImageId::new(0),
2671 region,
2672 natural: region.size,
2673 pixels: Arc::new(Pixmap::new(
2674 u16::try_from(region.size[0]).expect("small"),
2675 u16::try_from(region.size[1]).expect("small"),
2676 )),
2677 may_have_transparency: false,
2678 };
2679
2680 let exact = region(0, 0, 0, 16, 16);
2681 let other_layer = region(1, 0, 0, 16, 16);
2682 let partial = region(0, 16, 0, 16, 16);
2683 let evictions = [exact, other_layer, partial];
2684 let uploads = [upload(exact), upload(region(0, 16, 0, 16, 8))];
2685
2686 assert_eq!(
2687 uncovered_clears(&evictions, &uploads),
2688 vec![other_layer, partial],
2689 "only the exactly covered clear goes, the rest keep their order"
2690 );
2691 assert_eq!(uncovered_clears(&evictions, &[]), evictions.to_vec());
2692 assert!(uncovered_clears(&[], &uploads).is_empty());
2693 }
2694
2695 #[test]
2696 fn the_clear_skip_is_tied_to_zero_padding() {
2697 // Evictions are reported as `padded(region, ATLAS_PADDING)`, uploads
2698 // carry the raw allocation; the exact-cover match needs them equal.
2699 let raw = AtlasRegion {
2700 layer: 0,
2701 offset: [4, 8],
2702 size: [16, 16],
2703 };
2704 assert_eq!(padded(raw, ATLAS_PADDING), raw);
2705 }
2706
2707 #[test]
2708 fn a_pressure_eviction_into_the_same_rectangle_plans_no_clear() {
2709 // One slot: the second image can only take the first one's rectangle.
2710 let mut residency = ImageResidency::new(AtlasBudget {
2711 atlas_size: (16, 16),
2712 max_atlases: 1,
2713 });
2714 residency.begin_frame();
2715 let first = residency.resolve(&image(16, 16)).expect("fits");
2716 residency.acknowledge_plan();
2717
2718 residency.begin_frame();
2719 let second = residency
2720 .resolve(&image(16, 16))
2721 .expect("displaces the first");
2722 assert_eq!(second.region, first.region);
2723 assert_eq!(
2724 residency.evictions(),
2725 &[first.region],
2726 "the eviction itself is still pending"
2727 );
2728
2729 let (clears, uploads) = residency.plan();
2730 assert!(clears.is_empty(), "but the plan issues no clear for it");
2731 assert_eq!(uploads.len(), 1);
2732
2733 residency.acknowledge_plan();
2734 assert!(!residency.has_pending_plan());
2735 }
2736
2737 #[test]
2738 fn an_unacknowledged_upload_is_re_emitted_until_it_is_serviced() {
2739 // The invariant the whole pending plan exists for: a frame refused
2740 // after compiling never wrote these texels, so the entry map claiming
2741 // the image is resident has to stay answerable by a later frame's plan.
2742 let mut residency = residency();
2743 let data = image(8, 8);
2744
2745 residency.begin_frame();
2746 let first = residency.resolve(&data).expect("fits");
2747 let region = residency.uploads()[0].region;
2748
2749 for _ in 0..4 {
2750 residency.begin_frame();
2751 let again = residency.resolve(&data).expect("still resident");
2752 assert_eq!(again.region, first.region, "residency does not move");
2753 assert_eq!(
2754 residency.uploads().len(),
2755 1,
2756 "the same upload is re-offered, never duplicated"
2757 );
2758 assert_eq!(residency.uploads()[0].region, region);
2759 }
2760
2761 residency.acknowledge_plan();
2762 residency.begin_frame();
2763 residency.resolve(&data).expect("still resident");
2764 assert!(
2765 residency.uploads().is_empty(),
2766 "a serviced upload is never offered again"
2767 );
2768 }
2769
2770 #[test]
2771 fn reaping_an_unacknowledged_entry_withdraws_its_upload_with_it() {
2772 // Otherwise the plan would clear the rectangle and then write the
2773 // evicted image straight back into it, over whatever the packer handed
2774 // that space to next.
2775 let mut residency = residency();
2776 let data = image(8, 8);
2777
2778 residency.begin_frame();
2779 let resident = residency.resolve(&data).expect("fits");
2780 assert_eq!(residency.uploads().len(), 1);
2781
2782 for _ in 0..=MAX_UNSEEN_FRAMES {
2783 residency.begin_frame();
2784 }
2785
2786 assert_eq!(residency.entry_count(), 0);
2787 assert!(
2788 residency.uploads().is_empty(),
2789 "the reaped entry's unserviced upload goes with it"
2790 );
2791 assert_eq!(
2792 residency.evictions(),
2793 &[padded(resident.region, ATLAS_PADDING)]
2794 );
2795 }
2796
2797 #[test]
2798 fn every_pending_upload_lies_inside_the_atlas_its_layer_count_asks_for() {
2799 // What keeps a consumer from recording where an image lives and then
2800 // sampling a rectangle the array refused to write: the plan can only
2801 // ever name regions the budget and the reported depth already admit.
2802 let mut residency = residency();
2803 residency.begin_frame();
2804 residency.resolve(&image(64, 48)).expect("fills a layer");
2805 residency.resolve(&image(64, 48)).expect("takes the next");
2806
2807 let budget = residency.budget();
2808 let layers = residency.layers();
2809 assert_eq!(layers, 2);
2810 for upload in residency.uploads() {
2811 assert!(
2812 budget.contains(upload.region, layers),
2813 "{:?} is outside a {layers}-layer {budget:?}",
2814 upload.region
2815 );
2816 }
2817 }
2818
2819 /// A budget holding exactly two 64-square images: one per layer, with no
2820 /// room to pack a third anywhere, so "the atlas is full" is reachable in
2821 /// two allocations and needs no fragmentation reasoning.
2822 fn two_slot_residency() -> ImageResidency {
2823 ImageResidency::new(AtlasBudget {
2824 atlas_size: (64, 64),
2825 max_atlases: 2,
2826 })
2827 }
2828
2829 #[test]
2830 fn a_working_set_larger_than_the_atlas_re_uploads_rather_than_skipping() {
2831 // The scrolling-list shape: more distinct images pass the viewport
2832 // inside one age window than the atlas has rectangles for. Every draw
2833 // must still resolve — the price is a re-upload, never a hole.
2834 let mut residency = two_slot_residency();
2835 let images: Vec<ImageData> = (0..12).map(|_| image(64, 64)).collect();
2836
2837 for data in &images {
2838 residency.begin_frame();
2839 residency
2840 .resolve(data)
2841 .expect("a full atlas gives a rectangle back rather than refusing");
2842 residency.acknowledge_plan();
2843 }
2844
2845 assert_eq!(residency.skipped(), 0, "no draw was ever dropped");
2846 assert_eq!(
2847 residency.pressure_evictions(),
2848 10,
2849 "the ten images past the two slots each displaced one"
2850 );
2851 assert_eq!(residency.entry_count(), 2, "and residency stays bounded");
2852 }
2853
2854 #[test]
2855 fn an_image_resolved_this_frame_is_never_the_one_evicted() {
2856 // The ordering contract's own boundary: the plan clears before it
2857 // uploads, so a rectangle this frame already resolved cannot be handed
2858 // back inside it. A frame whose own working set outgrows the atlas is
2859 // still a skip, and nothing is scheduled for clearing.
2860 let mut residency = two_slot_residency();
2861 residency.begin_frame();
2862 residency.resolve(&image(64, 64)).expect("takes layer 0");
2863 residency.resolve(&image(64, 64)).expect("takes layer 1");
2864
2865 let skip = residency
2866 .resolve(&image(64, 64))
2867 .expect_err("nothing this frame drew is evictable");
2868
2869 assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
2870 assert_eq!(residency.skipped(), 1);
2871 assert_eq!(residency.pressure_evictions(), 0);
2872 assert_eq!(residency.entry_count(), 2, "both occupants stayed put");
2873 assert!(
2874 residency.evictions().is_empty(),
2875 "no region a frame is about to sample was scheduled for clearing"
2876 );
2877 }
2878
2879 #[test]
2880 fn the_least_recently_seen_image_is_the_one_displaced() {
2881 let mut residency = two_slot_residency();
2882 let old = image(64, 64);
2883 let refreshed = image(64, 64);
2884
2885 residency.begin_frame();
2886 let refreshed_region = residency.resolve(&refreshed).expect("takes a layer").region;
2887 residency.begin_frame();
2888 let old_region = residency.resolve(&old).expect("takes the other").region;
2889 residency.acknowledge_plan();
2890
2891 // The older entry is drawn again, which makes the *other* one the least
2892 // recently seen despite having been made resident second.
2893 residency.begin_frame();
2894 residency.resolve(&refreshed).expect("still resident");
2895 let arrival = residency.resolve(&image(64, 64)).expect("displaces one");
2896
2897 assert_eq!(residency.pressure_evictions(), 1);
2898 assert_eq!(
2899 arrival.region, old_region,
2900 "the rectangle taken is the least recently seen one"
2901 );
2902 assert_eq!(
2903 residency.resolve(&refreshed).map(|again| again.region),
2904 Ok(refreshed_region),
2905 "the image drawn this frame kept its own"
2906 );
2907 }
2908
2909 #[test]
2910 fn a_pressure_eviction_reports_its_rectangle_once_and_withdraws_its_upload() {
2911 let mut residency = two_slot_residency();
2912 let displaced = image(64, 64);
2913
2914 residency.begin_frame();
2915 let displaced_region = residency.resolve(&displaced).expect("takes a layer").region;
2916 residency.resolve(&image(64, 64)).expect("takes the other");
2917 // Deliberately NOT acknowledged: the displaced image still has an
2918 // upload pending, which must go with its entry rather than be written
2919 // into a rectangle its new occupant now owns.
2920 assert_eq!(residency.uploads().len(), 2);
2921
2922 residency.begin_frame();
2923 let arrival = residency
2924 .resolve(&image(64, 64))
2925 .expect("displaces the oldest");
2926
2927 assert_eq!(
2928 residency.evictions(),
2929 &[padded(displaced_region, ATLAS_PADDING)],
2930 "exactly one rectangle, reported exactly once"
2931 );
2932 assert_eq!(
2933 arrival.region, displaced_region,
2934 "and the freed rectangle is what the new occupant took"
2935 );
2936 assert_eq!(
2937 residency.uploads().len(),
2938 2,
2939 "the displaced image's unserviced upload went with its entry — \
2940 writing it would put an evicted image back over its successor"
2941 );
2942 }
2943
2944 #[test]
2945 fn an_age_reap_and_a_pressure_eviction_are_counted_separately() {
2946 // The two bounds answer different questions: one says a screen moved
2947 // on, the other says the atlas is too small for what is on it.
2948 let mut residency = two_slot_residency();
2949 let data = image(64, 64);
2950
2951 residency.begin_frame();
2952 residency.resolve(&data).expect("fits");
2953 for _ in 0..=MAX_UNSEEN_FRAMES {
2954 residency.begin_frame();
2955 }
2956
2957 assert_eq!(residency.entry_count(), 0, "the age reap ran");
2958 assert_eq!(
2959 residency.pressure_evictions(),
2960 0,
2961 "an age reap is not a pressure eviction"
2962 );
2963 assert_eq!(residency.frame_pressure_evictions(), 0);
2964 }
2965
2966 #[test]
2967 fn the_per_frame_eviction_count_is_this_frames_alone() {
2968 let mut residency = two_slot_residency();
2969 residency.begin_frame();
2970 residency.resolve(&image(64, 64)).expect("takes a layer");
2971 residency.resolve(&image(64, 64)).expect("takes the other");
2972
2973 residency.begin_frame();
2974 residency.resolve(&image(64, 64)).expect("displaces one");
2975 assert_eq!(residency.frame_pressure_evictions(), 1);
2976 assert_eq!(residency.pressure_evictions(), 1);
2977
2978 residency.begin_frame();
2979 assert_eq!(
2980 residency.frame_pressure_evictions(),
2981 0,
2982 "a new frame starts from nothing"
2983 );
2984 assert_eq!(
2985 residency.pressure_evictions(),
2986 1,
2987 "while the lifetime total keeps counting"
2988 );
2989 }
2990
2991 /// A budget of exactly four 32-square cells in one layer, so "the atlas is
2992 /// full" needs no reasoning about how a packer splits free space and a
2993 /// request that fits *no* arrangement of the holes is one rectangle.
2994 fn four_slot_residency() -> ImageResidency {
2995 ImageResidency::new(AtlasBudget {
2996 atlas_size: (64, 64),
2997 max_atlases: 1,
2998 })
2999 }
3000
3001 #[test]
3002 fn a_request_no_candidate_set_can_place_is_refused_before_anything_is_freed() {
3003 // The storm, at the cache seam. One 32-square is this frame's and three
3004 // are last frame's; a 48-square fits none of the holes those three
3005 // would leave while the fourth is resident. Freeing them buys nothing
3006 // — the draw is skipped either way — and costs three clears plus three
3007 // re-uploads, every frame the scene repeats. So the refusal has to come
3008 // before the first rectangle is handed back.
3009 let mut residency = four_slot_residency();
3010 let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3011 residency.begin_frame();
3012 for data in &resident {
3013 residency
3014 .resolve(data)
3015 .expect("the four cells take four images");
3016 }
3017 residency.acknowledge_plan();
3018
3019 residency.begin_frame();
3020 residency.resolve(&resident[0]).expect("still resident");
3021
3022 let skip = residency
3023 .resolve(&image(48, 48))
3024 .expect_err("no arrangement of the three candidates admits a 48-square");
3025
3026 assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
3027 assert_eq!(
3028 residency.pressure_evictions(),
3029 0,
3030 "a request that could not have been placed displaced nothing"
3031 );
3032 assert_eq!(residency.entry_count(), 4, "the residency is untouched");
3033 assert!(
3034 residency.evictions().is_empty(),
3035 "and nothing was scheduled for clearing on the way to failing"
3036 );
3037 assert!(residency.uploads().is_empty());
3038 }
3039
3040 #[test]
3041 fn a_request_larger_than_every_candidate_together_is_refused_on_area_alone() {
3042 // The cheaper half of the same precheck: freeing all three candidates
3043 // leaves three quarters of one layer, and the request wants all of it.
3044 // The area comes off the packer's own accounting, so it counts what the
3045 // glyph half of the allocator holds as well as this residency's own.
3046 let mut residency = four_slot_residency();
3047 let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3048 residency.begin_frame();
3049 for data in &resident {
3050 residency
3051 .resolve(data)
3052 .expect("the four cells take four images");
3053 }
3054 residency.acknowledge_plan();
3055
3056 residency.begin_frame();
3057 residency.resolve(&resident[0]).expect("still resident");
3058 residency
3059 .resolve(&image(64, 64))
3060 .expect_err("a whole layer is more than the three candidates hold");
3061
3062 assert_eq!(residency.pressure_evictions(), 0);
3063 assert_eq!(residency.entry_count(), 4);
3064 assert!(residency.evictions().is_empty());
3065 }
3066
3067 #[test]
3068 fn a_satisfiable_request_under_pressure_still_evicts_the_minimum() {
3069 // The other side of the precheck: it must not turn a working eviction
3070 // into a skip. One 32-square arriving against four resident ones is
3071 // satisfiable by freeing exactly one, and that is what it costs.
3072 let mut residency = four_slot_residency();
3073 let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3074 residency.begin_frame();
3075 for data in &resident {
3076 residency
3077 .resolve(data)
3078 .expect("the four cells take four images");
3079 }
3080 residency.acknowledge_plan();
3081
3082 residency.begin_frame();
3083 residency.resolve(&image(32, 32)).expect("displaces one");
3084
3085 assert_eq!(
3086 residency.pressure_evictions(),
3087 1,
3088 "the minimum, not the lot"
3089 );
3090 assert_eq!(residency.entry_count(), 4);
3091 assert_eq!(residency.evictions().len(), 1);
3092 }
3093
3094 #[test]
3095 fn a_blob_redrawn_at_a_new_extent_never_frees_what_this_frame_resolved() {
3096 // The extent-mismatch release under the pressure path's own rule. The
3097 // plan clears before it uploads, so giving the rectangle back on a
3098 // frame whose first draw already resolved it would zero texels that
3099 // draw is about to sample.
3100 let square = image(16, 8);
3101 let transposed = ImageData {
3102 data: square.data.clone(),
3103 format: square.format,
3104 alpha_type: square.alpha_type,
3105 width: 8,
3106 height: 16,
3107 };
3108
3109 let mut residency = residency();
3110 residency.begin_frame();
3111 let first = residency.resolve(&square).expect("takes a rectangle");
3112 let skip = residency
3113 .resolve(&transposed)
3114 .expect_err("the second extent cannot take the first one's rectangle");
3115
3116 assert!(
3117 matches!(skip, ImageSkip::SameFrameExtentConflict { .. }),
3118 "a same-frame ordering conflict, not an atlas-capacity refusal: {skip:?}"
3119 );
3120 assert!(
3121 residency.evictions().is_empty(),
3122 "no clear over a rectangle this frame already sampled"
3123 );
3124 assert_eq!(residency.entry_count(), 1);
3125 assert_eq!(residency.uploads().len(), 1);
3126
3127 // On a later frame nothing has sampled it yet, so the release is
3128 // ordinary again and the new extent takes its place.
3129 residency.acknowledge_plan();
3130 residency.begin_frame();
3131 let second = residency.resolve(&transposed).expect("the new extent fits");
3132 assert_eq!(second.natural, [8, 16]);
3133 assert_eq!(
3134 residency.evictions(),
3135 &[padded(first.region, ATLAS_PADDING)],
3136 "the rectangle the old extent held is reported for clearing"
3137 );
3138 }
3139
3140 #[test]
3141 fn a_layer_admits_only_what_every_occupant_leaves_room_for() {
3142 // The planner's own necessary condition, in isolation. A placement has
3143 // to lie entirely to one side of each occupant, so a 32-square in one
3144 // corner of a 64-square layer leaves nothing a 48-square can occupy —
3145 // while a 32-square still has three cells to choose from.
3146 let corner = PressureRect {
3147 key: 1,
3148 layer: 0,
3149 x0: 0,
3150 y0: 0,
3151 x1: 32,
3152 y1: 32,
3153 area: 1024,
3154 last_seen: 0,
3155 slot: 0,
3156 freed: false,
3157 };
3158 let rects = [corner];
3159
3160 assert!(!layer_admits(&rects, 0, 48, 48, (64, 64)));
3161 assert!(layer_admits(&rects, 0, 32, 32, (64, 64)));
3162 assert!(layer_admits(&rects, 0, 32, 64, (64, 64)));
3163 // A freed rectangle is not an obstacle, and a layer nothing occupies
3164 // admits anything the layer itself can hold.
3165 assert!(layer_admits(
3166 &[PressureRect {
3167 freed: true,
3168 ..corner
3169 }],
3170 0,
3171 64,
3172 64,
3173 (64, 64)
3174 ));
3175 assert!(!layer_admits(&[], 0, 65, 64, (64, 64)));
3176 // An occupant on another layer says nothing about this one.
3177 assert!(layer_admits(
3178 &[PressureRect { layer: 1, ..corner }],
3179 0,
3180 64,
3181 64,
3182 (64, 64)
3183 ));
3184 }
3185
3186 #[test]
3187 fn a_plan_that_would_outrun_its_budget_frees_nothing() {
3188 // The bound is on the *search*, not only on the damage: a walk that
3189 // reaches the budget without finding a fit answers `None`, and the
3190 // caller has released nothing by then.
3191 let mut rects: Vec<PressureRect> = (0..MAX_PRESSURE_EVICTION_CANDIDATES + 8)
3192 .map(|index| {
3193 let index = u32::try_from(index).expect("fits");
3194 PressureRect {
3195 key: u64::from(index),
3196 layer: 0,
3197 x0: index,
3198 y0: 0,
3199 x1: index + 1,
3200 y1: 1,
3201 area: 1,
3202 last_seen: 0,
3203 slot: index,
3204 freed: false,
3205 }
3206 })
3207 .collect();
3208 let order: Vec<usize> = (0..rects.len()).collect();
3209 let mut free_area = vec![0_u64];
3210 let mut layers = Vec::new();
3211
3212 assert_eq!(
3213 plan_pressure_eviction(
3214 64,
3215 64,
3216 (64, 64),
3217 &mut rects,
3218 &order,
3219 &mut free_area,
3220 &mut layers
3221 ),
3222 None,
3223 "a walk that cannot fit the request inside its budget plans nothing"
3224 );
3225 assert!(
3226 rects.iter().filter(|rect| rect.freed).count() <= MAX_PRESSURE_EVICTION_CANDIDATES + 1,
3227 "and it stopped walking rather than marking the whole residency"
3228 );
3229 }
3230
3231 #[test]
3232 fn a_satisfiable_layer_is_not_starved_by_a_hopeless_ones_candidates() {
3233 // The multi-layer shape the per-layer budget exists for. Layer 0 is
3234 // "this frame's" 64x40 anchor plus more than
3235 // `MAX_PRESSURE_EVICTION_CANDIDATES` one-pixel stale fillers packed
3236 // into the 64x24 strip left over — every one of them stale (older
3237 // than the incoming request's frame) and none of them, individually
3238 // or together, ever able to free a 32-tall rectangle, since the strip
3239 // they sit in is only 24 pixels tall. Layer 1 is four 32-square
3240 // quadrants exactly tiling the layer, three "this frame's" and one
3241 // stale. Layers 2 and 3 are each one "this frame's" 64-square image,
3242 // filling them outright so the incoming request cannot simply grow
3243 // into an unused layer.
3244 //
3245 // A single pooled budget spends its whole allowance walking layer 0's
3246 // fillers — none of which can ever satisfy the request — and never
3247 // reaches layer 1's one satisfiable candidate at all. A per-layer
3248 // budget exhausts layer 0's own allowance the same way, then gives
3249 // layer 1 a fresh one and finds the fit immediately.
3250 let mut residency = ImageResidency::new(AtlasBudget {
3251 atlas_size: (64, 64),
3252 max_atlases: 4,
3253 });
3254
3255 let anchor = image(64, 40);
3256 let fillers: Vec<ImageData> = (0..MAX_PRESSURE_EVICTION_CANDIDATES + 6)
3257 .map(|_| image(1, 1))
3258 .collect();
3259 let quadrants: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3260 let layer2_filler = image(64, 64);
3261 let layer3_filler = image(64, 64);
3262
3263 residency.begin_frame();
3264 residency
3265 .resolve(&anchor)
3266 .expect("the anchor takes layer 0's top");
3267 for filler in &fillers {
3268 residency
3269 .resolve(filler)
3270 .expect("a one-pixel filler always finds room in the leftover strip");
3271 }
3272 for quadrant in &quadrants {
3273 residency
3274 .resolve(quadrant)
3275 .expect("a 32-square quadrant packs layer 1's empty corners");
3276 }
3277 residency
3278 .resolve(&layer2_filler)
3279 .expect("a whole-layer image grows into a fresh layer 2");
3280 residency
3281 .resolve(&layer3_filler)
3282 .expect("a whole-layer image grows into a fresh layer 3");
3283 residency.acknowledge_plan();
3284 assert_eq!(
3285 residency.layers(),
3286 4,
3287 "fixture precondition: all four layers exist"
3288 );
3289
3290 // Refresh everything except the fillers and the first quadrant, so
3291 // only they are stale candidates when the request arrives — the
3292 // fillers sort ahead of the stale quadrant because they were
3293 // resolved first and share the same `last_seen`.
3294 residency.begin_frame();
3295 residency.resolve(&anchor).expect("still resident");
3296 residency.resolve(&quadrants[1]).expect("still resident");
3297 residency.resolve(&quadrants[2]).expect("still resident");
3298 residency.resolve(&quadrants[3]).expect("still resident");
3299 residency.resolve(&layer2_filler).expect("still resident");
3300 residency.resolve(&layer3_filler).expect("still resident");
3301
3302 let arrival = residency
3303 .resolve(&image(32, 32))
3304 .expect("a per-layer budget reaches layer 1's satisfiable candidate");
3305
3306 assert_eq!(
3307 residency.pressure_evictions(),
3308 1,
3309 "exactly the one quadrant needed, not layer 0's hopeless fillers"
3310 );
3311 assert_eq!(residency.evictions().len(), 1);
3312 assert_eq!(
3313 residency.evictions()[0].layer,
3314 arrival.region.layer,
3315 "the eviction and the arrival land on the same, single layer"
3316 );
3317 assert_ne!(
3318 arrival.region.layer, 0,
3319 "the winning layer is not the one full of hopeless fillers"
3320 );
3321 }
3322
3323 /// The tier line's own text, which only a `perf-trace` build has.
3324 #[cfg(feature = "perf-trace")]
3325 #[test]
3326 fn the_resolved_tier_line_names_the_tier_and_is_written_once() {
3327 let mobile = TierCaps::fake(DownlevelProfile::WebGl2);
3328 let choice = choose_tier(&mobile, None);
3329 assert_eq!(
3330 atlas_tier_line(&mobile, choice, AtlasBudget::MOBILE),
3331 "frust-perf atlas tier=mobile rule=downlevel budget=1024x1024x4 downlevel=WebGl2 \
3332 transient_saves_memory=false memory_mib=unknown adapter=fake-webgl2"
3333 );
3334
3335 let mut adreno = TierCaps::fake(DownlevelProfile::Full);
3336 adreno.adapter_name = "Adreno (TM) 660".to_string();
3337 let choice = choose_tier(&adreno, Some(8 << 30));
3338 assert_eq!(
3339 atlas_tier_line(&adreno, choice, AtlasBudget::MOBILE_LARGE_WIDE),
3340 "frust-perf atlas tier=mobile_large rule=table budget=2048x2048x4 downlevel=Full \
3341 transient_saves_memory=false memory_mib=8192 adapter=Adreno (TM) 660"
3342 );
3343
3344 // An adapter the table does not name says which capability answer
3345 // decided it.
3346 let mut tiler = TierCaps::fake(DownlevelProfile::Full);
3347 tiler.transient_saves_memory = true;
3348 let line = atlas_tier_line(&tiler, choose_tier(&tiler, None), AtlasBudget::MOBILE);
3349 assert!(line.contains("tier=mobile rule=heuristic"));
3350 assert!(line.contains("transient_saves_memory=true"));
3351
3352 let latch = TierLogOnce::new();
3353 let choice = choose_tier(&mobile, None);
3354 assert!(
3355 latch.emit(&mobile, choice, AtlasBudget::MOBILE),
3356 "the first call writes the line"
3357 );
3358 assert!(
3359 !latch.emit(&mobile, choice, AtlasBudget::MOBILE),
3360 "and no later one repeats it"
3361 );
3362 }
3363
3364 #[test]
3365 fn an_override_redistributes_a_tiers_memory_but_never_exceeds_it() {
3366 let tier = AtlasBudget::DESKTOP;
3367
3368 // The pathological override the adapter alone would admit: a
3369 // 16384-square layer is a gibibyte on its own, so clamping the layer
3370 // count could not have brought it inside the tier's 128 MiB.
3371 let huge = AtlasBudget {
3372 atlas_size: (16_384, 16_384),
3373 max_atlases: tier.max_atlases,
3374 }
3375 .within_total_bytes(tier.total_bytes());
3376 assert!(huge.total_bytes() <= tier.total_bytes());
3377 assert_eq!(huge.atlas_size.0, huge.atlas_size.1, "aspect preserved");
3378 assert!(huge.max_atlases >= 1);
3379
3380 // A modest override is left exactly as asked for.
3381 let modest = AtlasBudget {
3382 atlas_size: (1024, 1024),
3383 max_atlases: tier.max_atlases,
3384 }
3385 .within_total_bytes(tier.total_bytes());
3386 assert_eq!(modest.atlas_size, (1024, 1024));
3387 assert_eq!(modest.max_atlases, tier.max_atlases);
3388
3389 // An extreme aspect ratio floors one axis to nothing; it is raised back
3390 // to a texel and the other axis gives the room up, so the pair still
3391 // fits and neither axis is zero.
3392 let sliver = AtlasBudget {
3393 atlas_size: (65_535, 4),
3394 max_atlases: 1,
3395 }
3396 .within_total_bytes(64);
3397 assert!(sliver.atlas_size.0 >= 1 && sliver.atlas_size.1 >= 1);
3398 assert!(sliver.total_bytes() <= 64);
3399 }
3400}