Skip to main content

frust_engine/cache/
images.rs

1//! Image residency: which decoded images own atlas space, and for how long.
2//!
3//! A widget hands the display list the *same* decoded image every frame — the
4//! `peniko::ImageData` in a [`Command::Image`](frust_scene::Command::Image) is a
5//! reference-counted handle cloned per frame, never a re-decode — so the pixels
6//! behind it are stable across frames and belong on the GPU once rather than
7//! per draw. [`ImageResidency`] is what makes that "once" true: it keys an
8//! image on its blob's process-unique id ([`peniko::Blob::id`]) and hands back
9//! the same [`ImageId`] and the same atlas rectangle for every later frame that
10//! draws it.
11//!
12//! Allocation itself is pure — a rectangle packer over
13//! [`vello_common::multi_atlas::MultiAtlasManager`], no device in the loop — so
14//! everything here is host-testable. The two GPU-side halves it *describes*
15//! (writing a newly allocated region's texels, clearing an evicted one's) are
16//! carried out by [`crate::gpu::atlas`] against the plan this module produces:
17//! [`ImageResidency::evictions`] first, [`ImageResidency::uploads`] second. That
18//! order is a contract, not a preference — a rectangle freed this frame can be
19//! re-allocated in the same frame, so clearing after uploading would erase the
20//! image that just moved in.
21//!
22//! ## Residency is committed only once the plan is serviced
23//!
24//! The cache decides residency long before anything reaches a queue: a
25//! compiled frame can still be refused afterwards (a layer shape the scheduler
26//! will not serve, a page past the pool's ceiling, coverage or paints outgrowing
27//! their textures), and a frame refused after compiling is never drawn and never
28//! uploaded. So the plan is *pending* rather than taken: [`ImageResidency::plan`]
29//! hands out a copy, [`begin_frame`](ImageResidency::begin_frame) leaves it
30//! alone, and only [`acknowledge_plan`](ImageResidency::acknowledge_plan) —
31//! called by a consumer that has actually written the regions into a live atlas
32//! — clears it.
33//!
34//! That is what makes the entry map and the atlas's real contents agree. Without
35//! it, a refused frame leaves an image *recorded as resident* whose texels were
36//! never written: every later frame resolves the entry, schedules no upload
37//! (residency is a hit), and either drops the draw for the life of the process
38//! or samples a rectangle nothing ever wrote. Re-emitting the same plan until it
39//! is acknowledged costs a vector copy per frame in the steady state — where the
40//! plan is empty — and is what makes "an image drawn on a thousand frames
41//! uploads once" true of the *atlas* rather than only of the cache.
42//!
43//! ## An image larger than the atlas is minified, not dropped
44//!
45//! A source whose own extent will not fit one atlas layer is downsampled to fit
46//! ([`fit_extent`], [`minify`]) instead of being refused. A photograph decoded
47//! at its capture resolution is routinely several times the atlas budget while
48//! its destination rectangle is a thumbnail, so refusing it drops a draw the
49//! display list plainly describes; and every texel past the destination's own
50//! scale is one the sampler would have thrown away anyway.
51//!
52//! The filter is a plain box average over the premultiplied texels, hand-written
53//! rather than pulled in: the workspace's version pins are law, an image decoder
54//! is not a dependency this crate carries, and a box average is exactly right
55//! for the minification direction (every output texel is the mean of the source
56//! texels it covers). The aspect ratio is preserved, so the paint transform's
57//! two axes stay in step; what changes is the resident rectangle's extent, which
58//! [`ResidentImage::natural`] records alongside it so the consumer can rescale
59//! the encoded paint's own transform into the smaller rectangle.
60//!
61//! ## Two things are refused rather than attempted
62//!
63//! 1. **A format or size `vello_common` panics on.**
64//!    [`vello_common::paint::ImageSource::from_peniko_image_data`] asserts on a
65//!    dimension past `u16::MAX` and `unimplemented!()`s on a format outside
66//!    `Rgba8`/`Bgra8`, and a pixel buffer whose length disagrees with the
67//!    declared extent trips [`Pixmap::from_parts_with_opacity`]'s own assertion.
68//!    All three are checked here, *before* the call, so an oversized or
69//!    malformed image is an [`ImageSkip`] the frame path reports rather than a
70//!    panic it takes (E17). These, plus an atlas with no room left in it, are
71//!    now the only things [`ImageResidency::skipped`] counts: an image that is
72//!    merely *big* is minified and drawn.
73//! 2. **Anything at all, when `FRUST_ENGINE_NO_ATLAS` is set.** The kill switch
74//!    ([`crate::config::atlas_disabled`]) makes every resolution answer
75//!    [`ImageSkip::AtlasDisabled`], so no atlas is allocated, nothing is
76//!    uploaded, and image draws fall back to painting nothing — the switch's
77//!    whole point being to take the atlas out of a frame under diagnosis.
78//!
79//! ## Residency is bounded by age, and by pressure
80//!
81//! An entry unseen for [`MAX_UNSEEN_FRAMES`] consecutive frames is deallocated
82//! and its rectangle reported for clearing — the same age-based reap
83//! `frust-render`'s shader-effect cache uses, and for the same reason: a screen
84//! that stops drawing an image should give its texels back promptly, while an
85//! image drawn every other frame must never be mistaken for gone.
86//!
87//! Age alone is not a bound on *population*, though: it says nothing about how
88//! many distinct images a screen draws inside one age window. A list scrolling
89//! faster than [`MAX_UNSEEN_FRAMES`] streams more of them past the viewport
90//! than the atlas has rectangles for, and every one past the last free
91//! rectangle used to be an [`ImageSkip::NoAtlasSpace`] — a draw that painted
92//! nothing, on a screen whose solid fills all still landed, which reads as a
93//! hole rather than as a cache miss. So an allocation the packer refuses now
94//! *evicts* instead of giving up: [`ImageResidency::resolve`] works out which
95//! least-recently-seen rectangles would make room, frees exactly those, and
96//! each one whose rectangle was taken re-uploads on the next frame that draws
97//! it. A working set larger than the atlas therefore degrades to re-uploads —
98//! bandwidth — rather than to missing content, and
99//! [`ImageResidency::pressure_evictions`] counts how often that trade was made.
100//!
101//! ## An eviction is planned on paper before anything is freed
102//!
103//! An eviction that does not produce the allocation is pure loss. The draw is
104//! skipped anyway, so the frame pays a clear per freed rectangle *and* the
105//! re-upload of every displaced image *and* the hole it was trying to avoid.
106//! And the condition recurs: a request the atlas can never satisfy would strip
107//! and rebuild the residency on every frame that asks for it, which on a mobile
108//! budget is megabytes of texture traffic each way, per frame, buying nothing.
109//!
110//! So the pressure path commits nothing speculatively. `allocate_under_pressure`
111//! walks the candidates in least-recently-seen order over a *model* of the
112//! layers — the rectangles this residency knows are occupied, and each layer's
113//! free area as the packer itself reports it — and stops at the first prefix
114//! that would admit the request. Only then is anything released, and only the
115//! candidates on the layer the fit was found in. A request no bounded set of
116//! candidates can satisfy therefore stays the cheap [`ImageSkip::NoAtlasSpace`]
117//! it was before eviction existed, with the residency untouched.
118//!
119//! The model is deliberately *optimistic*: it can see this residency's own
120//! rectangles but not the glyph pages sharing the same allocator (see below),
121//! so it never refuses a request eviction could have satisfied on a given
122//! layer — a placement that clears the real occupants clears the model's
123//! subset of them too. The reverse can happen: the packer may still refuse a
124//! plan the model approved, which costs the planned evictions and no
125//! allocation. That case is bounded by the same plan budget rather than
126//! open-ended, and it is the one residual the planner cannot rule out without
127//! knowing where every glyph page sits.
128//!
129//! The bounded search itself is scoped *per layer*: each atlas layer gets its
130//! own candidate-count and area allowance rather than the whole residency
131//! sharing one pool, so a request only one layer could ever satisfy is not
132//! starved by candidates on layers that were never going to host it — each
133//! layer is either cleared within its own allowance or ruled out on its own
134//! terms, never on account of what another layer's candidates already cost.
135//!
136//! The one entry never freed on either route is one *this frame* has already
137//! resolved. The plan clears before it uploads, so handing back a rectangle the
138//! frame is about to sample would erase exactly the image the eviction was
139//! meant to make room for. A working set that outgrows the atlas inside a
140//! single frame is therefore still a skip, and honestly so: nothing is
141//! evictable. The same guard covers the *extent-mismatch* release in
142//! [`ImageResidency::resolve`], where a blob redrawn at a new extent gives its
143//! old rectangle back: on a frame that already sampled that rectangle the
144//! second draw is skipped instead, as
145//! [`ImageSkip::SameFrameExtentConflict`] rather than as a capacity refusal —
146//! the atlas may hold nothing at all beside the first extent's own rectangle.
147//!
148//! ## One allocator for images *and* glyphs
149//!
150//! The [`ImageCache`] this module holds is the process's only atlas allocator:
151//! the glyph policy (`crate::text::atlas_policy`) owns no cache of its own and
152//! allocates its slots through [`ImageResidency::allocator_mut`].
153//!
154//! That is a correctness requirement, not tidiness. An [`ImageId`] is a *slot
155//! index into one cache* — `ImageCache::allocate` hands out the next free index
156//! of its own `slots` vector — and the strip shader has exactly one atlas
157//! texture array to sample from. Two caches over the same page geometry would
158//! therefore mint the same small integers for unrelated occupants and pack them
159//! into overlapping rectangles of the same layers, so a glyph and an image would
160//! address each other's texels by construction. Sharing one cache makes both
161//! collisions unrepresentable: one id space, one packer, one set of live
162//! rectangles.
163//!
164//! It is also what upstream does — `vello_hybrid`'s `Resources` holds a single
165//! `image_cache` and hands it to its glyph atlas at every call — and it is what
166//! keeps the budget honest: a glyph page and an image page come out of the same
167//! [`AtlasBudget::max_atlases`] allowance, and [`ImageResidency::layers`] counts
168//! whichever of the two created a layer.
169//!
170//! The one rule a borrower must keep is ownership of its own handles: each side
171//! deallocates only the ids it allocated. Nothing enforces that beyond the two
172//! call sites, and it is the reason [`ImageResidency::allocator_mut`] is
173//! documented as a contract rather than a plain accessor.
174
175use std::collections::{HashMap, HashSet};
176use std::sync::Arc;
177#[cfg(feature = "perf-trace")]
178use std::sync::atomic::{AtomicBool, Ordering};
179
180use frust_gpu::{DownlevelProfile, TierCaps};
181use peniko::color::PremulRgba8;
182use peniko::{ImageData, ImageFormat};
183use thiserror::Error;
184use vello_common::image_cache::ImageCache;
185use vello_common::multi_atlas::{AllocationStrategy, AtlasConfig};
186use vello_common::paint::{ImageId, ImageSource};
187use vello_common::pixmap::Pixmap;
188
189use crate::config;
190
191/// Consecutive frames an image may go undrawn before its atlas rectangle is
192/// reclaimed.
193///
194/// 60 frames is half a second at 120Hz and a whole one at 60Hz: long enough
195/// that an image drawn intermittently (an every-other-frame animation, a brief
196/// scene-diff hiccup) never loses its residency, short enough that navigating
197/// away from an image-heavy screen returns its texels within a frame budget's
198/// worth of frames rather than at surface teardown.
199pub const MAX_UNSEEN_FRAMES: u64 = 60;
200
201/// Transparent padding pixels placed around each image in the atlas.
202///
203/// Zero, deliberately. Padding exists to stop a filtered sample from reading a
204/// neighbour's texels, and the strip shader's atlas samplers cannot do that:
205/// both the bilinear and the bicubic path clamp every tap into
206/// `[offset, offset + size - 1]` — the image's own rectangle — before it
207/// reaches `textureLoad`. Paying a two-pixel border per image would buy nothing
208/// and cost atlas space that a mobile budget does not have.
209pub const ATLAS_PADDING: u16 = 0;
210
211// The clear skip in `uncovered_clears` compares reported (padded) eviction
212// rectangles with raw upload rectangles; a non-zero padding would make them
213// differ and silently turn the skip off, or worse, mis-pair rectangles.
214const _: () = assert!(
215    ATLAS_PADDING == 0,
216    "uncovered_clears matches padded evictions against raw uploads, which only holds with zero padding"
217);
218
219/// The largest image edge that can be made resident at all.
220///
221/// `vello_common`'s image cache addresses an offset and an extent in `u16`, and
222/// its `ImageSource` conversion asserts on anything larger, so this is the
223/// ceiling the pre-check enforces rather than a policy of ours.
224pub const MAX_IMAGE_DIMENSION: u32 = u16::MAX as u32;
225
226/// The most atlas layers the shader's encoded-image record can name.
227///
228/// `GpuEncodedImage::image_params` gives `atlas_index` eight bits, so a layer
229/// past 255 could not be addressed even if the adapter allowed it.
230pub const MAX_ATLAS_LAYERS: usize = 256;
231
232/// Why one image could not be made resident.
233///
234/// Every variant is a *skip*, never a frame failure: the draw paints nothing
235/// and the rest of the frame proceeds. They are distinguished because the
236/// remedies differ — a too-large image is an application decision, an exhausted
237/// atlas is a budget one, and a disabled atlas is the operator's own doing.
238#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
239pub enum ImageSkip {
240    /// `FRUST_ENGINE_NO_ATLAS` is set; the atlas is out of the frame entirely.
241    #[error("image residency is disabled by FRUST_ENGINE_NO_ATLAS")]
242    AtlasDisabled,
243    /// The image declares a format the atlas does not store.
244    #[error("unsupported image format {0:?}: only Rgba8 and Bgra8 are uploaded")]
245    UnsupportedFormat(ImageFormat),
246    /// The image has no area, so there is nothing to make resident.
247    #[error("degenerate image extent {width}x{height}")]
248    DegenerateExtent {
249        /// Declared width in pixels.
250        width: u32,
251        /// Declared height in pixels.
252        height: u32,
253    },
254    /// The image is larger than [`MAX_IMAGE_DIMENSION`] on at least one axis.
255    #[error("image {width}x{height} exceeds the {max}-pixel residency ceiling")]
256    TooLarge {
257        /// Declared width in pixels.
258        width: u32,
259        /// Declared height in pixels.
260        height: u32,
261        /// The per-axis ceiling that refused it.
262        max: u32,
263    },
264    /// The pixel buffer's length disagrees with the declared extent and format.
265    #[error("image pixel buffer is {actual} bytes; {width}x{height} needs exactly {expected}")]
266    MalformedPixels {
267        /// Declared width in pixels.
268        width: u32,
269        /// Declared height in pixels.
270        height: u32,
271        /// Bytes the declared extent and format require.
272        expected: usize,
273        /// Bytes the blob actually holds.
274        actual: usize,
275    },
276    /// The paint transform cannot be inverted, so there is no device-to-image
277    /// mapping for the shader to sample through.
278    #[error("image paint transform is singular or non-finite")]
279    SingularTransform,
280    /// The image fits the ceiling but not the configured atlas budget, and no
281    /// resident image could be evicted to make room for it — either the atlas
282    /// holds nothing but images this same frame drew, or what it holds beside
283    /// them still leaves no rectangle this one fits in.
284    #[error("image {width}x{height} does not fit the {atlas_width}x{atlas_height} atlas budget")]
285    NoAtlasSpace {
286        /// Declared width in pixels.
287        width: u32,
288        /// Declared height in pixels.
289        height: u32,
290        /// Configured atlas width.
291        atlas_width: u32,
292        /// Configured atlas height.
293        atlas_height: u32,
294    },
295    /// The same blob was already resolved earlier in this same frame at a
296    /// different extent.
297    ///
298    /// Not an atlas-capacity refusal: the atlas may be nearly empty. The plan
299    /// clears before it uploads (see the module doc), so releasing this
300    /// blob's rectangle to make room for its own second extent would schedule
301    /// a clear over texels this frame already sampled — the first extent's
302    /// draw keeps its texels and the second is skipped until the next frame
303    /// instead.
304    #[error(
305        "image {width}x{height} already resolved this frame at a different extent; the first \
306         extent's texels are kept until the next frame"
307    )]
308    SameFrameExtentConflict {
309        /// The refused extent's width, after [`fit_extent`] the same way
310        /// [`NoAtlasSpace`](Self::NoAtlasSpace) reports it.
311        width: u32,
312        /// The refused extent's height, after [`fit_extent`].
313        height: u32,
314    },
315}
316
317/// The atlas geometry a tier of adapter is given.
318///
319/// Never `vello_common`'s own [`AtlasConfig`] default: that is 4096 square
320/// across eight layers, 64 MiB of `Rgba8Unorm` per layer and half a gigabyte in
321/// total, which is a desktop-only figure that a phone would pay for the first
322/// image it drew.
323#[derive(Debug, Clone, Copy, PartialEq, Eq)]
324pub struct AtlasBudget {
325    /// Extent of each atlas layer, in pixels.
326    pub atlas_size: (u32, u32),
327    /// Maximum number of layers the array may grow to.
328    pub max_atlases: usize,
329}
330
331impl AtlasBudget {
332    /// The budget a tile-based or downlevel-clamped adapter gets: 4 MiB a
333    /// layer, 16 MiB in total.
334    pub const MOBILE: Self = Self {
335        atlas_size: (1024, 1024),
336        max_atlases: 4,
337    };
338
339    /// The budget a full-profile immediate-mode adapter gets: 16 MiB a layer,
340    /// 128 MiB in total.
341    pub const DESKTOP: Self = Self {
342        atlas_size: (2048, 2048),
343        max_atlases: 8,
344    };
345
346    /// The large-mobile budget's preferred shape: four 2048-square layers,
347    /// 64 MiB in total — a quarter as many layers as
348    /// [`MOBILE_LARGE_DEEP`](Self::MOBILE_LARGE_DEEP) for the same texels, so
349    /// the array grows (and copies itself) a quarter as often.
350    pub const MOBILE_LARGE_WIDE: Self = Self {
351        atlas_size: (2048, 2048),
352        max_atlases: 4,
353    };
354
355    /// The large-mobile budget's shape on an adapter whose texture edge stops
356    /// short of 2048: sixteen 1024-square layers, the same 64 MiB.
357    pub const MOBILE_LARGE_DEEP: Self = Self {
358        atlas_size: (1024, 1024),
359        max_atlases: 16,
360    };
361
362    /// The large-mobile shape `caps`' adapter can create unclamped, or `None`
363    /// when it can create neither — the tier then falls back to
364    /// [`MOBILE`](Self::MOBILE) rather than to a large budget its ceilings
365    /// would have narrowed below the plain one's.
366    #[must_use]
367    pub fn mobile_large(caps: &TierCaps) -> Option<Self> {
368        [Self::MOBILE_LARGE_WIDE, Self::MOBILE_LARGE_DEEP]
369            .into_iter()
370            .find(|shape| shape.clamped(caps) == *shape)
371    }
372
373    /// The unclamped budget `tier` stands for on `caps`' adapter.
374    #[must_use]
375    pub fn for_tier(tier: AtlasTier, caps: &TierCaps) -> Self {
376        match tier {
377            AtlasTier::Mobile => Self::MOBILE,
378            AtlasTier::MobileLarge => Self::mobile_large(caps).unwrap_or(Self::MOBILE),
379            AtlasTier::Desktop => Self::DESKTOP,
380        }
381    }
382
383    /// The budget for `caps`' adapter, clamped to what it can actually create.
384    ///
385    /// The tier choice is [`choose_tier`] over the adapter and the device's
386    /// own memory ([`device_memory_bytes`]); the clamps that follow are the
387    /// adapter's own texture-edge and array-layer ceilings plus the eight-bit
388    /// [`MAX_ATLAS_LAYERS`] the encoded-image record can address.
389    ///
390    /// `FRUST_ENGINE_ATLAS_SIZE` overrides the tier's extent, and the override
391    /// is **budgeted, not unbudgeted**: it passes through
392    /// [`within_total_bytes`](Self::within_total_bytes) against the tier's own
393    /// [`total_bytes`](Self::total_bytes) before the adapter clamps. So the knob
394    /// redistributes a tier's memory between extent and depth — one large layer
395    /// instead of eight small ones — and cannot spend more of it than the tier
396    /// itself would. Clamped to the adapter last, because an override is a
397    /// request and a request the adapter cannot honour is still narrowed to what
398    /// it can.
399    ///
400    /// The resolved budget is reported once per process through
401    /// `atlas_tier_line`, because which tier a given phone takes is otherwise
402    /// an inference from behaviour rather than an observation: the adapter
403    /// name, the driver's own capability answers and the memory reading are
404    /// all device-reported, and a device answering differently from the one
405    /// it resembles is exactly the case a benchmark capture has to be able to
406    /// name. That line is `perf-trace`-only, like every other `frust-perf`
407    /// line the engine emits — a build without the feature resolves the same
408    /// budget and writes nothing.
409    #[must_use]
410    pub fn for_caps(caps: &TierCaps) -> Self {
411        let choice = choose_tier(caps, device_memory_bytes());
412        let resolved = Self::for_choice(choice, caps);
413
414        #[cfg(feature = "perf-trace")]
415        ATLAS_TIER_LOG.emit(caps, choice, resolved);
416        resolved
417    }
418
419    /// The budget `choice` resolves to on `caps`' adapter — [`for_caps`]
420    /// without reading the device's memory or writing the tier line, so the
421    /// table's outcome is testable for any adapter and memory size.
422    ///
423    /// [`for_caps`]: Self::for_caps
424    #[must_use]
425    pub fn for_choice(choice: TierChoice, caps: &TierCaps) -> Self {
426        let tier = Self::for_tier(choice.tier, caps);
427        match config::atlas_size() {
428            None => tier.clamped(caps),
429            Some(atlas_size) => Self {
430                atlas_size,
431                max_atlases: tier.max_atlases,
432            }
433            .within_total_bytes(tier.total_bytes())
434            .clamped(caps),
435        }
436    }
437
438    /// This budget narrowed so a fully grown array costs at most `total_bytes`.
439    ///
440    /// The extent goes first, scaled uniformly until one layer fits inside the
441    /// whole allowance; the layer count then takes whatever multiple of that
442    /// layer is left, never below one. Scaling the extent rather than only the
443    /// depth is what makes the ceiling hold at all: a single 16384-square layer
444    /// is a gigabyte on its own, so a clamp that could only reduce the layer
445    /// *count* would have nothing left to reduce.
446    ///
447    /// Both axes are floored after the uniform scale, which can only take the
448    /// product below the allowance — except where an extreme aspect ratio floors
449    /// one axis to nothing and it is raised back to a texel, so each axis is
450    /// then held against what the other leaves.
451    #[must_use]
452    pub fn within_total_bytes(self, total_bytes: u64) -> Self {
453        // A single texel is the smallest atlas that exists, so an allowance
454        // below one is treated as one rather than producing a zero extent the
455        // packer could not address.
456        let allowance = total_bytes.max(ATLAS_FORMAT_BYTES);
457        let max_texels = allowance / ATLAS_FORMAT_BYTES;
458
459        let (width, height) = self.atlas_size;
460        let texels = u64::from(width).saturating_mul(u64::from(height));
461        let atlas_size = if texels <= max_texels {
462            (width, height)
463        } else {
464            // `as` on a float saturates rather than wrapping, and both factors
465            // are finite and positive, so the narrowing is a plain floor.
466            let scale = (max_texels as f64 / texels.max(1) as f64).sqrt();
467            let scaled_w = ((f64::from(width) * scale) as u32).max(1);
468            let scaled_h = ((f64::from(height) * scale) as u32).max(1);
469            let fitted_w = fit_axis(scaled_w, max_texels / u64::from(scaled_h));
470            let fitted_h = fit_axis(scaled_h, max_texels / u64::from(fitted_w));
471            (fitted_w, fitted_h)
472        };
473
474        let layer_bytes = u64::from(atlas_size.0)
475            .saturating_mul(u64::from(atlas_size.1))
476            .saturating_mul(ATLAS_FORMAT_BYTES)
477            .max(1);
478        let layers = usize::try_from(allowance / layer_bytes).unwrap_or(MAX_ATLAS_LAYERS);
479
480        Self {
481            atlas_size,
482            max_atlases: self.max_atlases.min(layers).max(1),
483        }
484    }
485
486    /// Whether `region` lies inside an atlas array of `layers` layers at this
487    /// budget's per-layer extent.
488    ///
489    /// The pure counterpart of [`crate::gpu::atlas::AtlasArray::contains`],
490    /// answerable with no device: a consumer that records where an image lives
491    /// before the array exists checks the rectangle here, so it can never come
492    /// to name a region the array will refuse to write and then sample it as
493    /// whatever the texture happened to hold.
494    #[must_use]
495    pub fn contains(self, region: AtlasRegion, layers: u32) -> bool {
496        !region.is_empty()
497            && region.layer < layers
498            && region.offset[0].saturating_add(region.size[0]) <= self.atlas_size.0
499            && region.offset[1].saturating_add(region.size[1]) <= self.atlas_size.1
500    }
501
502    /// This budget narrowed to what `caps`' adapter can create.
503    #[must_use]
504    pub fn clamped(self, caps: &TierCaps) -> Self {
505        let edge = caps.max_texture_dimension_2d.max(1);
506        let layers = usize::try_from(caps.max_texture_array_layers).unwrap_or(MAX_ATLAS_LAYERS);
507
508        Self {
509            atlas_size: (
510                self.atlas_size.0.clamp(1, edge),
511                self.atlas_size.1.clamp(1, edge),
512            ),
513            max_atlases: self.max_atlases.clamp(1, layers.min(MAX_ATLAS_LAYERS)),
514        }
515    }
516
517    /// The `vello_common` configuration this budget describes.
518    ///
519    /// `initial_atlas_count` is zero so the first layer is created by the first
520    /// allocation that needs one: an application drawing no images pays for no
521    /// atlas at all, and [`vello_common::multi_atlas::MultiAtlasManager::new`]'s
522    /// own `expect` on eager creation can never be reached.
523    #[must_use]
524    pub fn config(self) -> AtlasConfig {
525        AtlasConfig {
526            initial_atlas_count: 0,
527            max_atlases: self.max_atlases,
528            atlas_size: self.atlas_size,
529            auto_grow: true,
530            allocation_strategy: AllocationStrategy::FirstFit,
531        }
532    }
533
534    /// The bytes one fully populated layer costs at [`ATLAS_FORMAT_BYTES`].
535    #[must_use]
536    pub fn layer_bytes(self) -> u64 {
537        u64::from(self.atlas_size.0)
538            .saturating_mul(u64::from(self.atlas_size.1))
539            .saturating_mul(ATLAS_FORMAT_BYTES)
540    }
541
542    /// The bytes every layer of this budget costs once fully grown.
543    #[must_use]
544    pub fn total_bytes(self) -> u64 {
545        self.layer_bytes()
546            .saturating_mul(self.max_atlases.try_into().unwrap_or(u64::MAX))
547    }
548}
549
550/// Bytes per texel of the atlas array (`Rgba8Unorm`).
551pub const ATLAS_FORMAT_BYTES: u64 = 4;
552
553/// `evictions` less every rectangle one of `uploads` covers exactly.
554///
555/// Exact cover only — same layer, offset and extent: an upload writes every
556/// texel of its own rectangle (its pixels are exactly the region's extent), so
557/// a clear of that same rectangle ordered ahead of it is overwritten in full.
558/// A rectangle covered only partly, or only by several uploads together, keeps
559/// its clear: whatever texels no upload reaches must still read as zero to the
560/// next tenant. Order is preserved, so the clears stay in the order the
561/// residency reported them.
562///
563/// The exact-cover match compares the reported eviction rectangle with the
564/// upload's allocated rectangle. Evictions are reported padded by
565/// [`ATLAS_PADDING`] while uploads carry the raw allocation, so the two only
566/// agree while the padding is zero; the assertion below ties the skip to that.
567#[must_use]
568pub fn uncovered_clears(evictions: &[AtlasRegion], uploads: &[ImageUpload]) -> Vec<AtlasRegion> {
569    if evictions.is_empty() || uploads.is_empty() {
570        return evictions.to_vec();
571    }
572    let written: HashSet<AtlasRegion> = uploads.iter().map(|upload| upload.region).collect();
573    evictions
574        .iter()
575        .copied()
576        .filter(|region| !written.contains(region))
577        .collect()
578}
579
580/// Whether `caps`' adapter reads as a mobile part by its capability answers
581/// alone.
582///
583/// Two signals, either of which is enough:
584///
585/// - its usable limits are clamped to the GLES-3.0/WebGL2 downlevel defaults,
586///   which is the profile every GL-backed target reports; or
587/// - it reports `transient_saves_memory`, wgpu's own answer to "is this a
588///   tile-based renderer keeping attachments on-chip" — true on the mobile
589///   TBDR parts and false on desktop immediate-mode ones.
590///
591/// This is the device-class read text hinting keys on, and the atlas tier's
592/// fallback for an adapter [`ADAPTER_FAMILIES`] does not name. It is not the
593/// atlas tier itself: some mobile drivers answer `transient_saves_memory`
594/// false (Adreno 618/620 do), which this read alone would put on the desktop
595/// budget — see [`choose_tier`].
596#[must_use]
597pub fn is_mobile_tier(caps: &TierCaps) -> bool {
598    caps.downlevel_profile != DownlevelProfile::Full || caps.transient_saves_memory
599}
600
601/// Which class of part an adapter's name places it in.
602#[derive(Debug, Clone, Copy, PartialEq, Eq)]
603pub enum AdapterFamily {
604    /// A phone/tablet GPU family: the atlas competes with the rest of a
605    /// shared-memory device's budget.
606    Mobile,
607    /// A discrete or desktop-integrated GPU family.
608    Desktop,
609}
610
611/// The adapter-name table the atlas tier is read from: a lowercase fragment
612/// and the family an adapter whose name contains it belongs to, first match
613/// winning.
614///
615/// A name table rather than capability answers alone because the capability
616/// answers disagree with the hardware: Adreno 618 and 620 report a full
617/// profile and `transient_saves_memory` false — the desktop answers — while
618/// sharing their device's memory with everything else on it. An adapter the
619/// table does not name keeps the capability read ([`is_mobile_tier`]), so a
620/// family missing here costs the old behaviour rather than a wrong one.
621///
622/// Apple is listed whole: Apple-silicon Macs already took the mobile budget
623/// through `transient_saves_memory`, so naming them changes nothing there.
624pub const ADAPTER_FAMILIES: &[(&str, AdapterFamily)] = &[
625    ("adreno", AdapterFamily::Mobile),
626    ("mali", AdapterFamily::Mobile),
627    ("immortalis", AdapterFamily::Mobile),
628    ("powervr", AdapterFamily::Mobile),
629    ("xclipse", AdapterFamily::Mobile),
630    ("maleoon", AdapterFamily::Mobile),
631    ("apple", AdapterFamily::Mobile),
632    ("tegra", AdapterFamily::Mobile),
633    ("nvidia", AdapterFamily::Desktop),
634    ("geforce", AdapterFamily::Desktop),
635    ("quadro", AdapterFamily::Desktop),
636    ("radeon", AdapterFamily::Desktop),
637    ("amd", AdapterFamily::Desktop),
638    ("intel", AdapterFamily::Desktop),
639];
640
641/// The family [`ADAPTER_FAMILIES`] places `adapter_name` in, matched
642/// case-insensitively; `None` for a name the table does not know.
643#[must_use]
644pub fn adapter_family(adapter_name: &str) -> Option<AdapterFamily> {
645    let name = adapter_name.to_ascii_lowercase();
646    ADAPTER_FAMILIES
647        .iter()
648        .find(|(fragment, _)| name.contains(fragment))
649        .map(|(_, family)| *family)
650}
651
652/// The atlas budget tiers.
653#[derive(Debug, Clone, Copy, PartialEq, Eq)]
654pub enum AtlasTier {
655    /// [`AtlasBudget::MOBILE`].
656    Mobile,
657    /// [`AtlasBudget::mobile_large`]: a mobile family part on a device with at
658    /// least [`MOBILE_LARGE_MIN_MEMORY`] of memory.
659    MobileLarge,
660    /// [`AtlasBudget::DESKTOP`].
661    Desktop,
662}
663
664impl AtlasTier {
665    /// The tier's name as the tier line spells it.
666    #[must_use]
667    pub fn label(self) -> &'static str {
668        match self {
669            Self::Mobile => "mobile",
670            Self::MobileLarge => "mobile_large",
671            Self::Desktop => "desktop",
672        }
673    }
674}
675
676/// Which rule of [`choose_tier`] decided a tier.
677#[derive(Debug, Clone, Copy, PartialEq, Eq)]
678pub enum TierRule {
679    /// The adapter's limits are clamped to the WebGL2 downlevel defaults.
680    Downlevel,
681    /// [`ADAPTER_FAMILIES`] named the adapter.
682    Table,
683    /// The table did not; [`is_mobile_tier`]'s capability read decided.
684    Heuristic,
685}
686
687impl TierRule {
688    /// The rule's name as the tier line spells it.
689    #[must_use]
690    pub fn label(self) -> &'static str {
691        match self {
692            Self::Downlevel => "downlevel",
693            Self::Table => "table",
694            Self::Heuristic => "heuristic",
695        }
696    }
697}
698
699/// The resolved tier, with what decided it.
700#[derive(Debug, Clone, Copy, PartialEq, Eq)]
701pub struct TierChoice {
702    /// The tier.
703    pub tier: AtlasTier,
704    /// The rule that chose it.
705    pub rule: TierRule,
706    /// The device memory the choice was made against, when it was known.
707    pub device_memory: Option<u64>,
708}
709
710/// The device memory from which a mobile family part takes the large budget:
711/// 4 GiB.
712///
713/// The large budget is 64 MiB of atlas, a sixteenth of that; below it the
714/// plain 16 MiB budget stays.
715pub const MOBILE_LARGE_MIN_MEMORY: u64 = 4 << 30;
716
717/// The atlas tier for `caps`' adapter on a device with `device_memory` bytes
718/// of memory (`None` when unknown).
719///
720/// In order:
721///
722/// 1. A downlevel (WebGL2/GLES) profile is [`AtlasTier::Mobile`], whatever the
723///    name: its limits are the browser ceiling, and the budget follows them.
724/// 2. An adapter [`ADAPTER_FAMILIES`] names takes its family's tier — a
725///    desktop family [`AtlasTier::Desktop`]; a mobile family
726///    [`AtlasTier::MobileLarge`] when the device reports at least
727///    [`MOBILE_LARGE_MIN_MEMORY`] and the adapter can create a large shape
728///    unclamped ([`AtlasBudget::mobile_large`]), else [`AtlasTier::Mobile`].
729///    Unknown memory is the small tier: the large one is earned, not assumed.
730/// 3. Any other adapter keeps [`is_mobile_tier`]'s capability read, mobile or
731///    desktop, exactly as before the table existed.
732#[must_use]
733pub fn choose_tier(caps: &TierCaps, device_memory: Option<u64>) -> TierChoice {
734    let (tier, rule) = if caps.downlevel_profile != DownlevelProfile::Full {
735        (AtlasTier::Mobile, TierRule::Downlevel)
736    } else {
737        match adapter_family(&caps.adapter_name) {
738            Some(AdapterFamily::Desktop) => (AtlasTier::Desktop, TierRule::Table),
739            Some(AdapterFamily::Mobile) => {
740                let large = device_memory.is_some_and(|bytes| bytes >= MOBILE_LARGE_MIN_MEMORY)
741                    && AtlasBudget::mobile_large(caps).is_some();
742                let tier = if large {
743                    AtlasTier::MobileLarge
744                } else {
745                    AtlasTier::Mobile
746                };
747                (tier, TierRule::Table)
748            }
749            None if is_mobile_tier(caps) => (AtlasTier::Mobile, TierRule::Heuristic),
750            None => (AtlasTier::Desktop, TierRule::Heuristic),
751        }
752    };
753    TierChoice {
754        tier,
755        rule,
756        device_memory,
757    }
758}
759
760/// The device's total memory in bytes, when this platform reports it.
761///
762/// Read once per process from `/proc/meminfo`'s `MemTotal` on Linux and
763/// Android — the kernel's own figure, readable without a permission on both —
764/// and `None` everywhere else, which keeps every other platform on the tier
765/// it took before the large one existed.
766#[must_use]
767pub fn device_memory_bytes() -> Option<u64> {
768    #[cfg(any(target_os = "linux", target_os = "android"))]
769    {
770        use std::sync::OnceLock;
771        static MEMORY: OnceLock<Option<u64>> = OnceLock::new();
772        *MEMORY.get_or_init(|| {
773            std::fs::read_to_string("/proc/meminfo")
774                .ok()
775                .and_then(|text| parse_mem_total(&text))
776        })
777    }
778    #[cfg(not(any(target_os = "linux", target_os = "android")))]
779    {
780        None
781    }
782}
783
784/// `MemTotal` from a `/proc/meminfo` text, in bytes; `None` when the line is
785/// missing or not the `<n> kB` the kernel writes.
786#[must_use]
787pub fn parse_mem_total(meminfo: &str) -> Option<u64> {
788    let line = meminfo
789        .lines()
790        .find_map(|line| line.strip_prefix("MemTotal:"))?;
791    let mut fields = line.split_whitespace();
792    let kib: u64 = fields.next()?.parse().ok()?;
793    match fields.next() {
794        Some("kB") => kib.checked_mul(1024),
795        _ => None,
796    }
797}
798
799/// The process's latch for the resolved-tier line, held by
800/// [`AtlasBudget::for_caps`].
801#[cfg(feature = "perf-trace")]
802static ATLAS_TIER_LOG: TierLogOnce = TierLogOnce::new();
803
804/// A one-shot latch for the resolved-tier line.
805///
806/// Once per *process* rather than once per call: a budget is resolved once per
807/// surface, and the question the line answers — which tier this adapter took —
808/// is about the device, so one line per run answers it completely while a line
809/// per surface would repeat it.
810///
811/// A latch value rather than a bare [`std::sync::Once`] because the rule it
812/// keeps is worth testing: [`emit`](Self::emit) answers whether *this* call was
813/// the one that wrote the line, so a test constructs a latch of its own and
814/// pins "the first call writes and no later one does" without installing a
815/// process-global logger.
816///
817/// Compiled only under `perf-trace`, with the line it writes: the release-lean
818/// gate asserts a shipping binary carries no `frust-perf` bytes at all, and a
819/// `#[cfg]` is what makes that the compiler's answer rather than the
820/// optimizer's.
821#[cfg(feature = "perf-trace")]
822#[derive(Debug, Default)]
823pub struct TierLogOnce {
824    emitted: AtomicBool,
825}
826
827#[cfg(feature = "perf-trace")]
828impl TierLogOnce {
829    /// A latch that has not emitted yet.
830    #[must_use]
831    pub const fn new() -> Self {
832        Self {
833            emitted: AtomicBool::new(false),
834        }
835    }
836
837    /// Log [`atlas_tier_line`] for `caps`, `choice` and `budget` if this latch
838    /// has not fired, answering whether this call was the one that did.
839    pub fn emit(&self, caps: &TierCaps, choice: TierChoice, budget: AtlasBudget) -> bool {
840        if self.emitted.swap(true, Ordering::Relaxed) {
841            return false;
842        }
843
844        log::info!("{}", atlas_tier_line(caps, choice, budget));
845        true
846    }
847}
848
849/// The one line a run records the resolved atlas tier as.
850///
851/// `perf-trace`-only, with [`TierLogOnce`] — see its own doc for why the gate
852/// is a `#[cfg]` rather than a branch.
853///
854/// `frust-perf`-prefixed deliberately: a benchmark capture keeps every line
855/// carrying that prefix and drops the rest, so this is what lets a run's own
856/// log say which budget the device was given rather than leaving it to be
857/// inferred from whether images went missing. `rule` names which of
858/// [`choose_tier`]'s rules decided, and `memory_mib` the device memory it was
859/// decided against (`unknown` where the platform does not report it). The
860/// adapter name is last because it is the one field that can contain spaces,
861/// which keeps every `key=value` field ahead of it parseable by splitting on
862/// whitespace.
863#[cfg(feature = "perf-trace")]
864#[must_use]
865pub fn atlas_tier_line(caps: &TierCaps, choice: TierChoice, budget: AtlasBudget) -> String {
866    let memory = choice
867        .device_memory
868        .map_or_else(|| "unknown".to_string(), |bytes| (bytes >> 20).to_string());
869
870    format!(
871        "frust-perf atlas tier={} rule={} budget={}x{}x{} downlevel={:?} \
872         transient_saves_memory={} memory_mib={memory} adapter={}",
873        choice.tier.label(),
874        choice.rule.label(),
875        budget.atlas_size.0,
876        budget.atlas_size.1,
877        budget.max_atlases,
878        caps.downlevel_profile,
879        caps.transient_saves_memory,
880        caps.adapter_name,
881    )
882}
883
884/// Where one image's texels live in the atlas array.
885///
886/// Hashable because the pending-clear list carries a set index beside it: the
887/// same rectangle can reach [`ImageResidency::evictions`] twice before a single
888/// clear has been serviced, and the dedup that keeps it reported once must not
889/// be a linear scan of a list that grows with the churn rate.
890#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
891pub struct AtlasRegion {
892    /// Array layer holding the region.
893    pub layer: u32,
894    /// Pixel offset of the region's top-left corner within that layer.
895    pub offset: [u32; 2],
896    /// Region extent in pixels.
897    pub size: [u32; 2],
898}
899
900impl AtlasRegion {
901    /// The bytes this region's texels occupy at [`ATLAS_FORMAT_BYTES`].
902    #[must_use]
903    pub fn byte_len(self) -> usize {
904        (self.size[0] as usize)
905            .saturating_mul(self.size[1] as usize)
906            .saturating_mul(ATLAS_FORMAT_BYTES as usize)
907    }
908
909    /// The row stride a texel copy of this region uses.
910    #[must_use]
911    pub fn bytes_per_row(self) -> u32 {
912        self.size[0].saturating_mul(ATLAS_FORMAT_BYTES as u32)
913    }
914
915    /// Whether this region has any texels at all.
916    #[must_use]
917    pub fn is_empty(self) -> bool {
918        self.size[0] == 0 || self.size[1] == 0
919    }
920}
921
922/// A resident image: the handle a paint names it by, and where its texels are.
923#[derive(Debug, Clone, Copy, PartialEq, Eq)]
924pub struct ResidentImage {
925    /// The handle an [`ImageSource::OpaqueId`] carries.
926    pub id: ImageId,
927    /// The atlas rectangle holding the image's own pixels, padding excluded.
928    ///
929    /// Equal to [`natural`](Self::natural) unless the source was minified to
930    /// fit the atlas, in which case it is the smaller rectangle actually
931    /// uploaded.
932    pub region: AtlasRegion,
933    /// The source's own declared extent, before any minification.
934    ///
935    /// Carried because it is the extent a paint transform was composed against
936    /// — `compile::paint::encode_image_command` derives its natural-to-dest
937    /// mapping from the `ImageData`'s declared size, which it reads before this
938    /// residency is consulted. A consumer that ends up sampling a *minified*
939    /// rectangle has to fold [`minify_scale`](Self::minify_scale) into that
940    /// transform, and this is the half of the ratio the atlas rectangle does
941    /// not carry.
942    pub natural: [u32; 2],
943    /// Transparent padding pixels around `region` in the atlas.
944    pub padding: u32,
945    /// Whether the image's premultiplied pixels include any non-opaque texel.
946    pub may_have_transparency: bool,
947}
948
949impl ResidentImage {
950    /// This image as the paint-side source a `vello_common` encoding carries.
951    #[must_use]
952    pub fn source(&self) -> ImageSource {
953        ImageSource::opaque_id_with_transparency_hint(self.id, self.may_have_transparency)
954    }
955
956    /// The per-axis factor mapping a natural-space texel coordinate onto the
957    /// resident rectangle, or `None` when the source was stored at full size.
958    ///
959    /// `None` rather than `Some((1.0, 1.0))` so a caller can skip the rescale
960    /// entirely on the overwhelmingly common path, and so "was this image
961    /// minified?" is one question rather than a float comparison.
962    #[must_use]
963    pub fn minify_scale(&self) -> Option<(f32, f32)> {
964        if self.region.size == self.natural {
965            return None;
966        }
967        let natural_w = self.natural[0].max(1) as f32;
968        let natural_h = self.natural[1].max(1) as f32;
969        Some((
970            self.region.size[0] as f32 / natural_w,
971            self.region.size[1] as f32 / natural_h,
972        ))
973    }
974}
975
976/// One region's texels, waiting to be written into the atlas array.
977///
978/// The pixels are the premultiplied `Rgba8Unorm` the atlas stores, produced
979/// once at allocation — an image made resident on frame 1 and drawn on frames
980/// 1..1000 is converted exactly once.
981#[derive(Debug, Clone)]
982pub struct ImageUpload {
983    /// The handle the paint naming these pixels carries.
984    ///
985    /// Carried so a consumer rebuilding its own view of residency can key this
986    /// upload directly rather than pairing the frame's uploads positionally
987    /// against its encoded paints. A plan re-emitted until it is acknowledged
988    /// breaks that pairing (an upload can outlive the frame whose draw order
989    /// produced it), and an upload that names itself needs no ordering to be
990    /// read correctly.
991    pub id: ImageId,
992    /// Where the texels go.
993    pub region: AtlasRegion,
994    /// The source's own declared extent, before any minification — see
995    /// [`ResidentImage::natural`], whose value this is.
996    ///
997    /// Carried on the upload because a consumer rebuilding its own view of
998    /// residency from a compiled frame has the upload and nothing else, and
999    /// `region` alone cannot say whether it holds a minified copy.
1000    pub natural: [u32; 2],
1001    /// The premultiplied pixels, row-major and exactly `region`'s extent.
1002    pub pixels: Arc<Pixmap>,
1003    /// Whether those pixels include any non-opaque texel — the same hint
1004    /// [`ResidentImage::may_have_transparency`] carries, so a consumer can
1005    /// rebuild the whole residency record from this upload alone.
1006    pub may_have_transparency: bool,
1007}
1008
1009/// One cache entry: the handle, its rectangle, and when it was last drawn.
1010#[derive(Debug, Clone, Copy)]
1011struct Entry {
1012    id: ImageId,
1013    region: AtlasRegion,
1014    /// The source extent this entry was allocated for, which is what a later
1015    /// frame's `ImageData` is matched against — `region` may be the minified
1016    /// rectangle and so says nothing about the source.
1017    natural: [u32; 2],
1018    may_have_transparency: bool,
1019    last_seen: u64,
1020}
1021
1022/// The images resident in the atlas array, keyed by blob identity.
1023///
1024/// One per compiler (and so one per surface). Its frame clock is advanced by
1025/// [`begin_frame`](Self::begin_frame), which is also where the age-based reap
1026/// runs — before any of the frame's own resolutions, so a rectangle freed this
1027/// frame is available to this frame's allocations and its clear is ordered
1028/// ahead of their uploads.
1029///
1030/// It also owns the atlas allocator the *glyph* policy packs into — see this
1031/// module's doc and [`allocator_mut`](Self::allocator_mut).
1032#[derive(Debug)]
1033pub struct ImageResidency {
1034    cache: ImageCache,
1035    budget: AtlasBudget,
1036    entries: HashMap<u64, Entry>,
1037    frame: u64,
1038    disabled: bool,
1039    uploads: Vec<ImageUpload>,
1040    evictions: Vec<AtlasRegion>,
1041    /// The pending clears' own set index: exactly the rectangles `evictions`
1042    /// holds, kept so [`release`](Self::release)'s "report a rectangle once"
1043    /// dedup is a hash lookup rather than a scan of a list whose length grows
1044    /// with the churn rate.
1045    eviction_set: HashSet<AtlasRegion>,
1046    skipped: u64,
1047    /// Rectangles taken back from a resident image because the packer had no
1048    /// room for a new one, over this residency's lifetime.
1049    pressure_evictions: u64,
1050    /// The same count for the current frame alone, zeroed by
1051    /// [`begin_frame`](Self::begin_frame) — what a per-frame report reads, since
1052    /// the lifetime total says nothing about whether the atlas is churning
1053    /// *now*.
1054    frame_pressure_evictions: u64,
1055    minified: u64,
1056    /// Blob keys whose minification has already been reported, so a source that
1057    /// is redrawn — or reaped and made resident again — logs once rather than
1058    /// once per residency.
1059    reported_minify: HashSet<u64>,
1060    /// Scratch for the reap's key list, kept so a frame that evicts allocates
1061    /// nothing.
1062    reaped: Vec<u64>,
1063    /// Scratch for the pressure planner's view of the layers — one rectangle
1064    /// per resident entry, grouped by layer — kept for the same reason: the
1065    /// frame that has to evict is the one least able to afford an allocation.
1066    pressure_rects: Vec<PressureRect>,
1067    /// Scratch for the candidate order over [`Self::pressure_rects`], least
1068    /// recently seen first. Built and sorted once per refused allocation, never
1069    /// once per eviction.
1070    pressure_order: Vec<usize>,
1071    /// Scratch for each layer's modelled free area, indexed by layer.
1072    pressure_free_area: Vec<u64>,
1073    /// Scratch for the distinct layers [`Self::pressure_order`] names, in the
1074    /// order their stalest candidate appears — the per-layer walk's own visit
1075    /// order, kept for the same no-allocation-while-evicting reason as the
1076    /// other pressure scratch.
1077    pressure_layers: Vec<u32>,
1078    /// Scratch for the keys an accepted plan releases.
1079    pressure_plan: Vec<u64>,
1080}
1081
1082impl ImageResidency {
1083    /// A residency over `budget`'s atlas geometry.
1084    ///
1085    /// `FRUST_ENGINE_NO_ATLAS` is consulted here, once, rather than per
1086    /// resolution: the switch is process-global and cached, so re-reading it on
1087    /// a hot path would buy nothing but a lock. A residency constructed with
1088    /// the switch set is exactly [`disabled`](Self::disabled).
1089    #[must_use]
1090    pub fn new(budget: AtlasBudget) -> Self {
1091        Self::with_enabled(budget, !config::atlas_disabled())
1092    }
1093
1094    /// A residency that refuses every image, allocating no atlas at all.
1095    ///
1096    /// What `FRUST_ENGINE_NO_ATLAS` selects, and what a caller wanting the same
1097    /// effect programmatically constructs. Every [`resolve`](Self::resolve)
1098    /// answers [`ImageSkip::AtlasDisabled`], so image draws are dropped with a
1099    /// logged reason rather than painted from an atlas that does not exist.
1100    #[must_use]
1101    pub fn disabled(budget: AtlasBudget) -> Self {
1102        Self::with_enabled(budget, false)
1103    }
1104
1105    fn with_enabled(budget: AtlasBudget, enabled: bool) -> Self {
1106        Self {
1107            cache: ImageCache::new_with_config(budget.config()),
1108            budget,
1109            entries: HashMap::new(),
1110            frame: 0,
1111            disabled: !enabled,
1112            uploads: Vec::new(),
1113            evictions: Vec::new(),
1114            eviction_set: HashSet::new(),
1115            skipped: 0,
1116            pressure_evictions: 0,
1117            frame_pressure_evictions: 0,
1118            minified: 0,
1119            reported_minify: HashSet::new(),
1120            reaped: Vec::new(),
1121            pressure_rects: Vec::new(),
1122            pressure_order: Vec::new(),
1123            pressure_free_area: Vec::new(),
1124            pressure_layers: Vec::new(),
1125            pressure_plan: Vec::new(),
1126        }
1127    }
1128
1129    /// A residency sized for `caps`' adapter.
1130    #[must_use]
1131    pub fn for_caps(caps: &TierCaps) -> Self {
1132        Self::new(AtlasBudget::for_caps(caps))
1133    }
1134
1135    /// The atlas geometry this residency allocates within.
1136    #[must_use]
1137    pub fn budget(&self) -> AtlasBudget {
1138        self.budget
1139    }
1140
1141    /// The single atlas allocator, for reading.
1142    ///
1143    /// Enough to resolve a handle either side allocated
1144    /// ([`ImageCache::get`]) or to count the layers created so far, without
1145    /// the mutable borrow allocation needs.
1146    #[must_use]
1147    pub fn allocator(&self) -> &ImageCache {
1148        &self.cache
1149    }
1150
1151    /// The single atlas allocator, for allocating through.
1152    ///
1153    /// This is the seam the glyph policy takes its slots from — see this
1154    /// module's doc for why one cache rather than two is a correctness
1155    /// requirement. `ImageId`s and rectangles handed out through here and
1156    /// through [`resolve`](Self::resolve) come from the same packer and the
1157    /// same id space, so a glyph and an image can never be given the same
1158    /// handle or overlapping texels.
1159    ///
1160    /// **The contract a borrower keeps:** allocate what you like, and
1161    /// deallocate *only handles you allocated yourself*. This residency's entry
1162    /// map records the rectangles it allocated and nothing re-checks them;
1163    /// freeing one of them from outside would leave an entry claiming a
1164    /// rectangle the packer has since handed to someone else. Nothing here
1165    /// enforces that, because the type the two sides must share is
1166    /// `vello_common`'s own and cannot carry an ownership tag — the glyph
1167    /// policy deallocates strictly inside `glifo`'s eviction, over the handles
1168    /// in its own entry map.
1169    #[must_use]
1170    pub fn allocator_mut(&mut self) -> &mut ImageCache {
1171        &mut self.cache
1172    }
1173
1174    /// Whether this residency refuses every image.
1175    #[must_use]
1176    pub fn is_disabled(&self) -> bool {
1177        self.disabled
1178    }
1179
1180    /// How many images are currently resident.
1181    #[must_use]
1182    pub fn entry_count(&self) -> usize {
1183        self.entries.len()
1184    }
1185
1186    /// The frame clock entries age against.
1187    #[must_use]
1188    pub fn frame(&self) -> u64 {
1189        self.frame
1190    }
1191
1192    /// How many atlas layers have been created so far.
1193    ///
1194    /// The array texture must be at least this deep before the frame's uploads
1195    /// are written.
1196    ///
1197    /// Read off the shared allocator rather than accumulated from this
1198    /// residency's own allocations, so a layer the *glyph* policy created
1199    /// counts too: the two classes pack into one array, and an array sized to
1200    /// the image half alone would refuse to hold the glyph half's pages. The
1201    /// count only ever grows — `MultiAtlasManager` appends layers and never
1202    /// drops one — so a depth reported on an earlier frame stays valid.
1203    #[must_use]
1204    pub fn layers(&self) -> u32 {
1205        u32::try_from(self.cache.atlas_count()).unwrap_or(u32::MAX)
1206    }
1207
1208    /// How many resolutions have been refused over this residency's lifetime.
1209    ///
1210    /// Counts only images that could not be uploaded at all — a format or
1211    /// extent the conversion refuses, a disabled atlas, or an atlas with no
1212    /// room left *and nothing evictable to make room with*. A source merely
1213    /// larger than one atlas layer is minified to fit, and one that displaced
1214    /// another image is counted by
1215    /// [`pressure_evictions`](Self::pressure_evictions) instead; neither
1216    /// reaches this counter.
1217    #[must_use]
1218    pub fn skipped(&self) -> u64 {
1219        self.skipped
1220    }
1221
1222    /// How many resident images have been evicted to make room for another
1223    /// over this residency's lifetime.
1224    ///
1225    /// The price of never refusing a draw the atlas could hold *something*
1226    /// for: each one is an image that will re-upload the next frame it is
1227    /// drawn on. A steady screen leaves this at zero; a scroll through more
1228    /// distinct images than the atlas has rectangles for grows it once per
1229    /// displaced image, which is the signal that the budget — not the
1230    /// mechanism — is what a frame is paying for.
1231    #[must_use]
1232    pub fn pressure_evictions(&self) -> u64 {
1233        self.pressure_evictions
1234    }
1235
1236    /// How many resident images the *current* frame has evicted to make room.
1237    ///
1238    /// Zeroed by [`begin_frame`](Self::begin_frame), so a per-frame report reads
1239    /// this rather than differencing
1240    /// [`pressure_evictions`](Self::pressure_evictions) itself.
1241    #[must_use]
1242    pub fn frame_pressure_evictions(&self) -> u64 {
1243        self.frame_pressure_evictions
1244    }
1245
1246    /// How many sources have been downsampled to fit the atlas over this
1247    /// residency's lifetime.
1248    ///
1249    /// Observational, and the counter that makes "an image too large for the
1250    /// atlas is minified, not dropped" measurable rather than asserted.
1251    #[must_use]
1252    pub fn minified(&self) -> u64 {
1253        self.minified
1254    }
1255
1256    /// The regions whose texels must be cleared before the pending uploads.
1257    ///
1258    /// Pending rather than per-frame: an entry freed on a frame nobody
1259    /// serviced is still waiting to be cleared on the next one. Both kinds of
1260    /// eviction land here — the age-based reap at the head of a frame and a
1261    /// rectangle taken back under pressure mid-frame — so the consumer's
1262    /// clear-then-upload contract covers the second without knowing it exists.
1263    #[must_use]
1264    pub fn evictions(&self) -> &[AtlasRegion] {
1265        &self.evictions
1266    }
1267
1268    /// The regions whose texels must be written after the pending evictions.
1269    ///
1270    /// Pending rather than per-frame: an image made resident on a frame that was
1271    /// refused is still waiting to be written on the next one.
1272    #[must_use]
1273    pub fn uploads(&self) -> &[ImageUpload] {
1274        &self.uploads
1275    }
1276
1277    /// A copy of the pending clear-then-write plan, evictions first.
1278    ///
1279    /// A *copy*, not a drain: the plan stays pending until
1280    /// [`acknowledge_plan`](Self::acknowledge_plan) says it was serviced against
1281    /// a live atlas, so a frame refused after compiling re-emits the same plan
1282    /// on the next frame rather than losing it (see the module doc). The upload
1283    /// pixels are behind an `Arc`, so a re-emitted plan copies handles rather
1284    /// than texels, and the steady-state plan is empty on both counts.
1285    ///
1286    /// The clears are [`evictions`](Self::evictions) less every rectangle a
1287    /// pending upload covers exactly (see [`uncovered_clears`]): that upload
1288    /// writes every texel the clear would have zeroed, after it, in the same
1289    /// plan — so the clear is pure cost, a rectangle's worth of zeroes per
1290    /// churned image. Dropped from the plan rather than from `evictions`, so
1291    /// the pending set and its acknowledgement are unchanged; a consumer
1292    /// forgetting the evicted tenant by rectangle (the renderer's registry)
1293    /// still needs the full eviction set: [`evictions`](Self::evictions) names
1294    /// every rectangle to forget, whether or not its clear was skipped.
1295    #[must_use]
1296    pub fn plan(&self) -> (Vec<AtlasRegion>, Vec<ImageUpload>) {
1297        (
1298            uncovered_clears(&self.evictions, &self.uploads),
1299            self.uploads.clone(),
1300        )
1301    }
1302
1303    /// Whether any part of the pending plan is still unserviced.
1304    #[must_use]
1305    pub fn has_pending_plan(&self) -> bool {
1306        !self.evictions.is_empty() || !self.uploads.is_empty()
1307    }
1308
1309    /// Record that the pending plan reached the atlas array, clearing it.
1310    ///
1311    /// Call this only once every region it names has actually been written or
1312    /// cleared. Calling it on a frame that was refused, or whose writes the
1313    /// array declined, is exactly the defect the pending plan exists to prevent:
1314    /// the entry map would go on claiming an image is resident whose texels are
1315    /// whatever the atlas texture happened to hold.
1316    pub fn acknowledge_plan(&mut self) {
1317        self.evictions.clear();
1318        self.eviction_set.clear();
1319        self.uploads.clear();
1320    }
1321
1322    /// Advance the frame clock and reclaim everything unseen for
1323    /// [`MAX_UNSEEN_FRAMES`].
1324    ///
1325    /// Call once at the head of a frame, before any [`resolve`](Self::resolve).
1326    /// An unacknowledged plan deliberately survives this: the frame that was to
1327    /// service it may have been refused, and dropping it here is what would turn
1328    /// a refused frame into a permanently unwritten atlas region.
1329    pub fn begin_frame(&mut self) {
1330        self.frame = self.frame.saturating_add(1);
1331        self.frame_pressure_evictions = 0;
1332        self.reap();
1333    }
1334
1335    /// The atlas rectangle for `data`'s pixels, allocating and scheduling an
1336    /// upload on the first frame that asks for it.
1337    ///
1338    /// # Errors
1339    ///
1340    /// Returns the [`ImageSkip`] naming why the image was refused. Every
1341    /// refusal happens before any `vello_common` call that could assert on the
1342    /// same condition, so an image the renderer cannot hold is a skipped draw
1343    /// rather than a panicked frame (E17).
1344    pub fn resolve(&mut self, data: &ImageData) -> Result<ResidentImage, ImageSkip> {
1345        let result = self.resolve_inner(data);
1346        if result.is_err() {
1347            self.skipped = self.skipped.saturating_add(1);
1348        }
1349        result
1350    }
1351
1352    fn resolve_inner(&mut self, data: &ImageData) -> Result<ResidentImage, ImageSkip> {
1353        if self.disabled {
1354            return Err(ImageSkip::AtlasDisabled);
1355        }
1356
1357        check_supported(data)?;
1358
1359        let key = data.data.id();
1360        if let Some(entry) = self.entries.get_mut(&key)
1361            && entry.natural == [data.width, data.height]
1362        {
1363            entry.last_seen = self.frame;
1364            return Ok(ResidentImage {
1365                id: entry.id,
1366                region: entry.region,
1367                natural: entry.natural,
1368                padding: u32::from(ATLAS_PADDING),
1369                may_have_transparency: entry.may_have_transparency,
1370            });
1371        }
1372
1373        // A blob id recorded at a different extent is a caller that rebuilt the
1374        // metadata around the same buffer. Its old rectangle describes the
1375        // wrong pixels, so it is released before the new one is taken rather
1376        // than left to age out holding both — unless *this* frame has already
1377        // resolved it at the other extent. The plan clears before it uploads,
1378        // so freeing that rectangle would schedule a clear over texels a draw
1379        // this same frame already recorded, which is exactly what the pressure
1380        // path refuses to do. The second extent is skipped instead, on the same
1381        // terms as a frame whose own working set outgrows the atlas — but this
1382        // is a same-frame ordering conflict on one blob key, not an atlas
1383        // capacity shortage, so it is reported as its own reason rather than
1384        // folded into [`ImageSkip::NoAtlasSpace`].
1385        if self
1386            .entries
1387            .get(&key)
1388            .is_some_and(|entry| entry.last_seen == self.frame)
1389        {
1390            let (width, height) = fit_extent([data.width, data.height], self.budget.atlas_size);
1391            return Err(ImageSkip::SameFrameExtentConflict { width, height });
1392        }
1393        self.release(key);
1394
1395        // Checked above, so this cannot assert; the conversion is also the one
1396        // place the straight-alpha premultiply happens, and it happens once per
1397        // image rather than once per frame.
1398        let ImageSource::Pixmap(pixels) = ImageSource::from_peniko_image_data(data) else {
1399            // `from_peniko_image_data` only ever yields a pixmap; an id-backed
1400            // source would mean a `vello_common` change, which is a skip rather
1401            // than an unreachable.
1402            return Err(ImageSkip::UnsupportedFormat(data.format));
1403        };
1404
1405        let natural = [data.width, data.height];
1406        // Minified before the allocation rather than after it: the packer is
1407        // asked for the rectangle that is actually going to be uploaded, so an
1408        // over-ceiling source never takes (and then has to give back) space at
1409        // its declared extent.
1410        let (width, height) = fit_extent(natural, self.budget.atlas_size);
1411        let pixels = if [width, height] == natural {
1412            pixels
1413        } else {
1414            self.note_minify(key, natural, [width, height]);
1415            Arc::new(minify(&pixels, width, height))
1416        };
1417
1418        let no_space = ImageSkip::NoAtlasSpace {
1419            width,
1420            height,
1421            atlas_width: self.budget.atlas_size.0,
1422            atlas_height: self.budget.atlas_size.1,
1423        };
1424
1425        let id = match self.cache.allocate(width, height, ATLAS_PADDING) {
1426            Ok(id) => id,
1427            // The atlas is full. Take a rectangle back from an image no longer
1428            // on screen rather than dropping this draw — see
1429            // [`allocate_under_pressure`](Self::allocate_under_pressure), and
1430            // the module doc for why a hole is worse than a re-upload.
1431            Err(_) => self
1432                .allocate_under_pressure(width, height)
1433                .ok_or(no_space)?,
1434        };
1435
1436        let Some(resource) = self.cache.get(id) else {
1437            // A successful allocation always populates its slot; refusing here
1438            // keeps the frame path free of the `expect` the reference renderer
1439            // takes at this same point.
1440            return Err(no_space);
1441        };
1442
1443        let region = AtlasRegion {
1444            layer: resource.atlas_id.as_u32(),
1445            offset: resource.offsets(),
1446            size: resource.size(),
1447        };
1448        let may_have_transparency = pixels.may_have_transparency();
1449
1450        self.entries.insert(
1451            key,
1452            Entry {
1453                id,
1454                region,
1455                natural,
1456                may_have_transparency,
1457                last_seen: self.frame,
1458            },
1459        );
1460        self.uploads.push(ImageUpload {
1461            id,
1462            region,
1463            natural,
1464            pixels,
1465            may_have_transparency,
1466        });
1467
1468        Ok(ResidentImage {
1469            id,
1470            region,
1471            natural,
1472            padding: u32::from(ATLAS_PADDING),
1473            may_have_transparency,
1474        })
1475    }
1476
1477    /// Allocate `width` x `height` by taking rectangles back from resident
1478    /// images, least recently seen first, once the packer has refused.
1479    ///
1480    /// Answers the handle that finally fit, or `None` when no bounded set of
1481    /// candidates would have made room — which is what
1482    /// [`ImageSkip::NoAtlasSpace`] then reports, with the residency exactly as
1483    /// it was found.
1484    ///
1485    /// **Nothing is released until a plan that fits has been found.** An
1486    /// eviction that does not produce the allocation buys nothing and costs a
1487    /// clear plus a re-upload per rectangle, every frame the condition recurs
1488    /// (see the module doc). So the search runs over a *model* of the layers
1489    /// and commits only when the model says the request would then be placed:
1490    ///
1491    /// 1. **Candidates.** Only an entry last seen on an earlier frame. The
1492    ///    frame path services [`evictions`](Self::evictions) *before*
1493    ///    [`uploads`](Self::uploads), so freeing a rectangle this frame has
1494    ///    already resolved would schedule a clear over texels the same frame is
1495    ///    about to sample — painting the very hole this eviction exists to
1496    ///    prevent. A frame whose own working set outgrows the atlas therefore
1497    ///    still skips, because everything in the atlas belongs to it.
1498    /// 2. **Order.** `(last_seen, slot, key)`, least recently seen first, built
1499    ///    and sorted *once* per refused allocation rather than once per
1500    ///    eviction. The slot index breaks a tie between two entries last drawn
1501    ///    on the same frame, so which image is displaced is a property of the
1502    ///    residency rather than of the entry map's iteration order. Because the
1503    ///    order is by age, an entry drawn on the *immediately preceding* frame
1504    ///    is reached only after every staler one has already been planned —
1505    ///    which is what keeps a large image redrawn every frame from thrashing
1506    ///    a working set that is merely idle. The candidates are then walked one
1507    ///    *layer* at a time, layers visited in that same staleness order (the
1508    ///    layer whose stalest candidate sorts earliest goes first).
1509    /// 3. **Fit.** After each candidate joins the plan, the layer it sat on is
1510    ///    re-tested: the request must fit inside the freed area that layer now
1511    ///    models, and must clear every rectangle still occupying it
1512    ///    ([`layer_admits`]). The first prefix that passes ends the search.
1513    /// 4. **Budget, per layer.** Each layer's own walk stops after
1514    ///    [`MAX_PRESSURE_EVICTION_CANDIDATES`] of *that layer's* candidates, or
1515    ///    once *that layer's* plan has freed [`PRESSURE_EVICTION_AREA_BUDGET`]
1516    ///    times the requested area — a fresh allowance per layer rather than one
1517    ///    pool the whole residency shares, so candidates on a layer too
1518    ///    fragmented to ever admit the request cannot spend the allowance a
1519    ///    later layer needed to prove it could. A layer that ends on its own
1520    ///    budget contributes nothing and the walk moves to the next layer.
1521    /// 5. **Commit.** Only the planned rectangles *on the layer the fit was
1522    ///    found in* are released — a candidate the walk passed over on another
1523    ///    layer contributed nothing and is left resident.
1524    ///
1525    /// The model is optimistic about occupants it cannot see: the glyph policy
1526    /// packs through the same allocator (see the module doc) and its pages are
1527    /// not in this entry map, so [`layer_admits`] tests a subset of the real
1528    /// obstacles. That direction is the safe one — a placement clearing every
1529    /// real occupant clears the subset too, so a request eviction could satisfy
1530    /// on a *given layer* is never refused for that reason. The converse is
1531    /// possible: the packer may refuse a plan the model approved, which costs
1532    /// the plan's evictions and no allocation. It is bounded by the same
1533    /// per-layer budget as the plan itself, and it is the one residual the
1534    /// planner cannot rule out without knowing where every glyph page sits.
1535    fn allocate_under_pressure(&mut self, width: u32, height: u32) -> Option<ImageId> {
1536        // The overwhelmingly common refusal — a frame that filled the atlas
1537        // with its own draws — costs one pass over the entry map and no model
1538        // at all.
1539        if !self
1540            .entries
1541            .values()
1542            .any(|entry| entry.last_seen < self.frame)
1543        {
1544            return None;
1545        }
1546
1547        // Taken from the residency's own scratch, so the frame least able to
1548        // afford an allocation does not make one to evict with.
1549        let mut rects = std::mem::take(&mut self.pressure_rects);
1550        let mut order = std::mem::take(&mut self.pressure_order);
1551        let mut free_area = std::mem::take(&mut self.pressure_free_area);
1552        let mut layers = std::mem::take(&mut self.pressure_layers);
1553        let mut plan = std::mem::take(&mut self.pressure_plan);
1554
1555        self.model_layers(&mut rects, &mut order, &mut free_area);
1556        let fit = plan_pressure_eviction(
1557            width,
1558            height,
1559            self.budget.atlas_size,
1560            &mut rects,
1561            &order,
1562            &mut free_area,
1563            &mut layers,
1564        );
1565
1566        plan.clear();
1567        if let Some(layer) = fit {
1568            plan.extend(
1569                rects
1570                    .iter()
1571                    .filter(|rect| rect.freed && rect.layer == layer)
1572                    .map(|rect| rect.key),
1573            );
1574        }
1575
1576        let allocated = if plan.is_empty() {
1577            None
1578        } else {
1579            for key in &plan {
1580                self.release(*key);
1581                self.pressure_evictions = self.pressure_evictions.saturating_add(1);
1582                self.frame_pressure_evictions = self.frame_pressure_evictions.saturating_add(1);
1583            }
1584            self.cache.allocate(width, height, ATLAS_PADDING).ok()
1585        };
1586
1587        rects.clear();
1588        order.clear();
1589        free_area.clear();
1590        layers.clear();
1591        plan.clear();
1592        self.pressure_rects = rects;
1593        self.pressure_order = order;
1594        self.pressure_free_area = free_area;
1595        self.pressure_layers = layers;
1596        self.pressure_plan = plan;
1597        allocated
1598    }
1599
1600    /// Fill the planner's scratch with this residency's own view of the atlas.
1601    ///
1602    /// `rects` takes one entry per resident image, grouped by layer so a fit
1603    /// test reads one layer's slice; `order` takes the candidate indices, least
1604    /// recently seen first across the *whole* residency — [`plan_pressure_eviction`]
1605    /// is what regroups it per layer for its own bounded walk; `free_area` takes
1606    /// each layer's free area *as the packer reports it*, which is the one
1607    /// figure that accounts for the glyph pages this map cannot see.
1608    fn model_layers(
1609        &self,
1610        rects: &mut Vec<PressureRect>,
1611        order: &mut Vec<usize>,
1612        free_area: &mut Vec<u64>,
1613    ) {
1614        rects.clear();
1615        rects.extend(self.entries.iter().map(|(key, entry)| {
1616            let region = padded(entry.region, ATLAS_PADDING);
1617            PressureRect {
1618                key: *key,
1619                layer: region.layer,
1620                x0: region.offset[0],
1621                y0: region.offset[1],
1622                x1: region.offset[0].saturating_add(region.size[0]),
1623                y1: region.offset[1].saturating_add(region.size[1]),
1624                area: u64::from(region.size[0]).saturating_mul(u64::from(region.size[1])),
1625                last_seen: entry.last_seen,
1626                slot: entry.id.as_u32(),
1627                freed: false,
1628            }
1629        }));
1630        rects.sort_unstable_by_key(|rect| rect.layer);
1631
1632        order.clear();
1633        order.extend(
1634            rects
1635                .iter()
1636                .enumerate()
1637                .filter_map(|(index, rect)| (rect.last_seen < self.frame).then_some(index)),
1638        );
1639        order.sort_unstable_by_key(|index| {
1640            let rect = &rects[*index];
1641            (rect.last_seen, rect.slot, rect.key)
1642        });
1643
1644        free_area.clear();
1645        free_area.resize(self.cache.atlas_count(), 0);
1646        for (id, stats) in self.cache.atlas_manager().atlas_stats() {
1647            if let Some(free) = free_area.get_mut(id.as_u32() as usize) {
1648                *free = u64::from(stats.total_area.saturating_sub(stats.allocated_area));
1649            }
1650        }
1651    }
1652
1653    /// Count and report one source downsampled to fit the atlas.
1654    ///
1655    /// Once per blob id, at warning level: a minified image is a silent quality
1656    /// decision the application may not have intended (a full-resolution photo
1657    /// where a thumbnail was wanted), so it says so — but a scene redrawing it
1658    /// every frame must not repeat the message.
1659    fn note_minify(&mut self, key: u64, natural: [u32; 2], fitted: [u32; 2]) {
1660        self.minified = self.minified.saturating_add(1);
1661        if !self.reported_minify.insert(key) {
1662            return;
1663        }
1664        log::warn!(
1665            "image {}x{} is larger than the {}x{} atlas budget; uploading a {}x{} box-filtered \
1666             copy instead (reported once per image)",
1667            natural[0],
1668            natural[1],
1669            self.budget.atlas_size.0,
1670            self.budget.atlas_size.1,
1671            fitted[0],
1672            fitted[1],
1673        );
1674    }
1675
1676    /// Deallocate every entry unseen for [`MAX_UNSEEN_FRAMES`], recording each
1677    /// one's rectangle for clearing.
1678    ///
1679    /// The age half of the two bounds; the population half is
1680    /// [`allocate_under_pressure`](Self::allocate_under_pressure), which runs
1681    /// only when an allocation has already failed. Keeping them separate is
1682    /// what makes an idle screen give its texels back promptly whether or not
1683    /// anything is asking for them.
1684    fn reap(&mut self) {
1685        let mut reaped = std::mem::take(&mut self.reaped);
1686        reaped.clear();
1687        reaped.extend(self.entries.iter().filter_map(|(key, entry)| {
1688            (self.frame.saturating_sub(entry.last_seen) > MAX_UNSEEN_FRAMES).then_some(*key)
1689        }));
1690
1691        for key in reaped.drain(..) {
1692            self.release(key);
1693        }
1694        self.reaped = reaped;
1695    }
1696
1697    /// Drop `key`'s entry, freeing its atlas rectangle and recording it for
1698    /// clearing.
1699    ///
1700    /// The padded rectangle is what is cleared: an allocation reserves its
1701    /// padding too, so leaving the border behind would leave stale texels a
1702    /// later allocation could sample through.
1703    ///
1704    /// An upload still pending for that rectangle goes with the entry. It
1705    /// describes pixels no live entry claims any more, and writing them after
1706    /// the clear would put an evicted image back into a rectangle the packer has
1707    /// already handed to something else.
1708    fn release(&mut self, key: u64) {
1709        let Some(entry) = self.entries.remove(&key) else {
1710            return;
1711        };
1712        // The handle came from this cache's own `allocate`, so the atlas it
1713        // names exists and the deallocation cannot fail.
1714        self.cache.deallocate(entry.id);
1715        self.uploads.retain(|upload| upload.id != entry.id);
1716
1717        // The plan survives an unacknowledged frame, so the same rectangle can
1718        // reach here twice before a single clear has been issued for it; one
1719        // clear is what the second one would write anyway.
1720        let region = padded(entry.region, ATLAS_PADDING);
1721        if self.eviction_set.insert(region) {
1722            self.evictions.push(region);
1723        }
1724    }
1725}
1726
1727/// The most rectangles one refused allocation may plan to take back from a
1728/// single atlas layer.
1729///
1730/// A bound on the *search*, not only on the damage: a layer whose walk
1731/// reaches it without finding a fit contributes nothing to the plan, so the
1732/// cost of one hopeless layer stays bounded rather than eating into the
1733/// allowance a later, satisfiable layer needs to prove itself. Sixty-four is
1734/// far past what any satisfiable request has needed — a fit is normally found
1735/// in one, since the rectangle handed back is usually the size of the one
1736/// being asked for — while still being a small fraction of what a
1737/// mobile-tier layer can hold. Applied once per layer a refused allocation
1738/// considers, never once across every layer together — see
1739/// [`plan_pressure_eviction`].
1740const MAX_PRESSURE_EVICTION_CANDIDATES: usize = 64;
1741
1742/// How much area one refused allocation may plan to free from a single atlas
1743/// layer, as a multiple of the area it asked for.
1744///
1745/// The other half of the bound, and the one that scales with the request: a
1746/// layer that has already given back four times the texels the request wants
1747/// and still has nowhere to put them is fighting fragmentation, not scarcity,
1748/// and freeing more of that same layer will not fix it. Applied per layer,
1749/// the same way [`MAX_PRESSURE_EVICTION_CANDIDATES`] is.
1750const PRESSURE_EVICTION_AREA_BUDGET: u64 = 4;
1751
1752/// One resident rectangle as the pressure planner sees it.
1753///
1754/// A flat copy rather than a borrow of the entry map, because the plan is drawn
1755/// up while the map is still intact and the release that follows needs it
1756/// mutably. `x1`/`y1` are exclusive, and every field is the *padded* allocation
1757/// — the rectangle the packer actually holds — rather than the image's own.
1758#[derive(Debug, Clone, Copy)]
1759struct PressureRect {
1760    /// Blob key of the entry holding this rectangle.
1761    key: u64,
1762    /// Array layer the rectangle sits in.
1763    layer: u32,
1764    /// Left edge, inclusive.
1765    x0: u32,
1766    /// Top edge, inclusive.
1767    y0: u32,
1768    /// Right edge, exclusive.
1769    x1: u32,
1770    /// Bottom edge, exclusive.
1771    y1: u32,
1772    /// Texels the rectangle covers.
1773    area: u64,
1774    /// The entry's frame stamp, which is what makes it a candidate or not.
1775    last_seen: u64,
1776    /// The entry's slot index, which breaks a tie between two entries last
1777    /// drawn on the same frame.
1778    slot: u32,
1779    /// Whether the plan under construction has given this rectangle back.
1780    freed: bool,
1781}
1782
1783/// The layer a bounded prefix of `order` would make room in, or `None`.
1784///
1785/// Frees nothing: it marks `rects` and grows `free_area` in the model only, so
1786/// a request no prefix satisfies leaves the residency untouched. `order` is
1787/// least-recently-seen-first across the *whole* residency; this regroups it
1788/// one layer at a time — `layers` scratch, cleared and refilled here, in the
1789/// order each layer's stalest candidate appears in `order` — and gives each
1790/// layer its own [`MAX_PRESSURE_EVICTION_CANDIDATES`]/
1791/// [`PRESSURE_EVICTION_AREA_BUDGET`] allowance rather than pooling one across
1792/// every layer. Without that split, a layer with many candidates that could
1793/// never admit the request (too fragmented, or blocked by an occupant no
1794/// eviction touches) spends the whole budget before a later, satisfiable
1795/// layer is ever tried — the request is then refused even though a bounded
1796/// eviction on that later layer would have placed it. See
1797/// [`ImageResidency::allocate_under_pressure`] for the whole shape and for why
1798/// the answer is a layer rather than a bool — only the planned rectangles on
1799/// the fitting layer are worth releasing.
1800fn plan_pressure_eviction(
1801    width: u32,
1802    height: u32,
1803    atlas_size: (u32, u32),
1804    rects: &mut [PressureRect],
1805    order: &[usize],
1806    free_area: &mut [u64],
1807    layers: &mut Vec<u32>,
1808) -> Option<u32> {
1809    let padding = u32::from(ATLAS_PADDING).saturating_mul(2);
1810    let request_w = width.saturating_add(padding);
1811    let request_h = height.saturating_add(padding);
1812    let request_area = u64::from(request_w).saturating_mul(u64::from(request_h));
1813    let area_budget = request_area.saturating_mul(PRESSURE_EVICTION_AREA_BUDGET);
1814
1815    layers.clear();
1816    for &index in order {
1817        let Some(rect) = rects.get(index) else {
1818            continue;
1819        };
1820        if !layers.contains(&rect.layer) {
1821            layers.push(rect.layer);
1822        }
1823    }
1824
1825    for &layer in layers.iter() {
1826        let mut layer_freed_area = 0_u64;
1827        let mut step = 0_usize;
1828
1829        for &index in order {
1830            let on_this_layer = rects.get(index).is_some_and(|rect| rect.layer == layer);
1831            if !on_this_layer {
1832                continue;
1833            }
1834            if step >= MAX_PRESSURE_EVICTION_CANDIDATES || layer_freed_area > area_budget {
1835                break;
1836            }
1837            step += 1;
1838
1839            let area = {
1840                let Some(rect) = rects.get_mut(index) else {
1841                    continue;
1842                };
1843                rect.freed = true;
1844                rect.area
1845            };
1846            layer_freed_area = layer_freed_area.saturating_add(area);
1847
1848            let Some(free) = free_area.get_mut(layer as usize) else {
1849                continue;
1850            };
1851            *free = free.saturating_add(area);
1852            // Area first because it is exact — it comes off the packer's own
1853            // accounting, so it counts the glyph pages `layer_admits` cannot see —
1854            // and because it is the cheaper of the two questions.
1855            if *free >= request_area && layer_admits(rects, layer, request_w, request_h, atlas_size)
1856            {
1857                return Some(layer);
1858            }
1859        }
1860    }
1861
1862    None
1863}
1864
1865/// Whether a `width` x `height` rectangle could still be placed in `layer` once
1866/// every rectangle the plan has marked freed is out of the way.
1867///
1868/// A *necessary* condition rather than an exact packing answer. Any placement
1869/// has to avoid each occupant whole, so it must lie entirely to one side of it:
1870/// if some occupant leaves less than the requested extent on all four sides,
1871/// there is nowhere for the rectangle to go and the plan cannot succeed. A plan
1872/// this refuses is therefore certainly hopeless; one it admits may still be
1873/// refused by the packer, which is the direction that costs a bounded number of
1874/// evictions rather than a wrong picture (see
1875/// [`ImageResidency::allocate_under_pressure`]).
1876///
1877/// `rects` is grouped by layer, so this reads one layer's slice rather than the
1878/// whole residency.
1879fn layer_admits(
1880    rects: &[PressureRect],
1881    layer: u32,
1882    width: u32,
1883    height: u32,
1884    atlas_size: (u32, u32),
1885) -> bool {
1886    if width > atlas_size.0 || height > atlas_size.1 {
1887        return false;
1888    }
1889
1890    let start = rects.partition_point(|rect| rect.layer < layer);
1891    let end = rects.partition_point(|rect| rect.layer <= layer);
1892    rects[start..end]
1893        .iter()
1894        .filter(|rect| !rect.freed)
1895        .all(|rect| {
1896            width <= rect.x0
1897                || width <= atlas_size.0.saturating_sub(rect.x1)
1898                || height <= rect.y0
1899                || height <= atlas_size.1.saturating_sub(rect.y1)
1900        })
1901}
1902
1903/// The extent `natural` is stored at inside an `atlas`-sized layer: itself when
1904/// it already fits, and otherwise the largest rectangle of the same aspect ratio
1905/// that does.
1906///
1907/// The scale is taken as the smaller of the two axis ratios and applied to both,
1908/// so the two axes of the paint transform stay in step — a per-axis fit would
1909/// stretch the image. Each axis is floored (never rounding *up* past the layer)
1910/// and then held at one texel: a source that scales below a whole texel still
1911/// has to have somewhere to be, and a zero-extent allocation is not a rectangle
1912/// the packer or the shader can address.
1913#[must_use]
1914pub fn fit_extent(natural: [u32; 2], atlas: (u32, u32)) -> (u32, u32) {
1915    let (width, height) = (natural[0], natural[1]);
1916    if width <= atlas.0 && height <= atlas.1 {
1917        return (width, height);
1918    }
1919
1920    let scale = (f64::from(atlas.0) / f64::from(width.max(1)))
1921        .min(f64::from(atlas.1) / f64::from(height.max(1)));
1922    let axis = |value: u32, ceiling: u32| {
1923        let scaled = (f64::from(value) * scale).floor();
1924        // `as` on a non-finite or out-of-range float saturates in Rust, so the
1925        // clamps below are the whole check rather than a second line of defence.
1926        (scaled as u32).clamp(1, ceiling.max(1))
1927    };
1928
1929    (axis(width, atlas.0), axis(height, atlas.1))
1930}
1931
1932/// `source` box-filtered down to `width` x `height`.
1933///
1934/// Every output texel is the unweighted mean of the half-open source rectangle
1935/// it covers, computed on the premultiplied bytes the atlas stores — which is
1936/// the space the samples are composited in, so averaging there is what keeps a
1937/// partially transparent source from bleeding its colour outward.
1938///
1939/// Minification only: a request at or above the source's own extent copies the
1940/// texels straight across rather than inventing any, since a box filter has no
1941/// magnification arm and the caller never asks for one ([`fit_extent`] only ever
1942/// shrinks).
1943#[must_use]
1944pub fn minify(source: &Pixmap, width: u32, height: u32) -> Pixmap {
1945    let source_w = u32::from(source.width());
1946    let source_h = u32::from(source.height());
1947    let width = width.clamp(1, source_w.max(1));
1948    let height = height.clamp(1, source_h.max(1));
1949
1950    let texels = source.data();
1951    let mut out = Vec::with_capacity((width as usize).saturating_mul(height as usize));
1952    let mut may_have_transparency = false;
1953
1954    for y in 0..height {
1955        // Half-open source rows, derived from the output row rather than
1956        // accumulated, so rounding never leaves a gap or an overlap between
1957        // consecutive rows.
1958        let y0 = (u64::from(y) * u64::from(source_h) / u64::from(height)) as u32;
1959        let y1 = ((u64::from(y) + 1) * u64::from(source_h) / u64::from(height)) as u32;
1960        let y1 = y1.max(y0.saturating_add(1)).min(source_h);
1961
1962        for x in 0..width {
1963            let x0 = (u64::from(x) * u64::from(source_w) / u64::from(width)) as u32;
1964            let x1 = ((u64::from(x) + 1) * u64::from(source_w) / u64::from(width)) as u32;
1965            let x1 = x1.max(x0.saturating_add(1)).min(source_w);
1966
1967            let mut sum = [0_u64; 4];
1968            let mut count = 0_u64;
1969            for row in y0..y1 {
1970                let base = (row as usize).saturating_mul(source_w as usize);
1971                for column in x0..x1 {
1972                    let Some(texel) = texels.get(base.saturating_add(column as usize)) else {
1973                        continue;
1974                    };
1975                    sum[0] += u64::from(texel.r);
1976                    sum[1] += u64::from(texel.g);
1977                    sum[2] += u64::from(texel.b);
1978                    sum[3] += u64::from(texel.a);
1979                    count += 1;
1980                }
1981            }
1982
1983            // A count of zero is unreachable — both ranges are non-empty by
1984            // construction — and flooring it at one rather than branching keeps
1985            // the divide-by-zero panic off the frame path (E17) while answering
1986            // a transparent texel if it ever were reachable, since the sums
1987            // would then be zero too.
1988            let divisor = count.max(1);
1989            // Round to nearest rather than truncating: a uniform source has to
1990            // come back bit-identical, which truncating an exact integer mean
1991            // already gives, but a near-uniform one must not drift a level
1992            // darker.
1993            let mean = |channel: usize| {
1994                ((sum[channel] + divisor / 2) / divisor).min(u64::from(u8::MAX)) as u8
1995            };
1996
1997            let texel = PremulRgba8 {
1998                r: mean(0),
1999                g: mean(1),
2000                b: mean(2),
2001                a: mean(3),
2002            };
2003            may_have_transparency |= texel.a != u8::MAX;
2004            out.push(texel);
2005        }
2006    }
2007
2008    // Both extents were clamped into `1..=u16::MAX` above (a source pixmap's own
2009    // extents are `u16`), so neither conversion can lose information.
2010    Pixmap::from_parts_with_opacity(
2011        out,
2012        width.min(u32::from(u16::MAX)) as u16,
2013        height.min(u32::from(u16::MAX)) as u16,
2014        may_have_transparency,
2015    )
2016}
2017
2018/// `axis` held at or below `ceiling`, and never below the one texel an
2019/// addressable rectangle needs.
2020fn fit_axis(axis: u32, ceiling: u64) -> u32 {
2021    axis.min(u32::try_from(ceiling).unwrap_or(u32::MAX)).max(1)
2022}
2023
2024/// `region` grown by `padding` on every side, clamped at the layer origin.
2025fn padded(region: AtlasRegion, padding: u16) -> AtlasRegion {
2026    let padding = u32::from(padding);
2027    let pad_x = padding.min(region.offset[0]);
2028    let pad_y = padding.min(region.offset[1]);
2029
2030    AtlasRegion {
2031        layer: region.layer,
2032        offset: [region.offset[0] - pad_x, region.offset[1] - pad_y],
2033        size: [
2034            region.size[0].saturating_add(pad_x + padding),
2035            region.size[1].saturating_add(pad_y + padding),
2036        ],
2037    }
2038}
2039
2040/// Refuse an image `vello_common`'s own conversion would assert on.
2041///
2042/// The three conditions are exactly the three assertions downstream: the
2043/// format `unimplemented!()`, the `u16::MAX` dimension check, and
2044/// `Pixmap::from_parts_with_opacity`'s length equality. Checking them here is
2045/// what keeps the panic off the frame path.
2046fn check_supported(data: &ImageData) -> Result<(), ImageSkip> {
2047    match data.format {
2048        ImageFormat::Rgba8 | ImageFormat::Bgra8 => {}
2049        format => return Err(ImageSkip::UnsupportedFormat(format)),
2050    }
2051
2052    if data.width == 0 || data.height == 0 {
2053        return Err(ImageSkip::DegenerateExtent {
2054            width: data.width,
2055            height: data.height,
2056        });
2057    }
2058
2059    if data.width > MAX_IMAGE_DIMENSION || data.height > MAX_IMAGE_DIMENSION {
2060        return Err(ImageSkip::TooLarge {
2061            width: data.width,
2062            height: data.height,
2063            max: MAX_IMAGE_DIMENSION,
2064        });
2065    }
2066
2067    let actual = data.data.data().len();
2068    let expected = data
2069        .format
2070        .size_in_bytes(data.width, data.height)
2071        .unwrap_or(usize::MAX);
2072    if actual != expected {
2073        return Err(ImageSkip::MalformedPixels {
2074            width: data.width,
2075            height: data.height,
2076            expected,
2077            actual,
2078        });
2079    }
2080
2081    Ok(())
2082}
2083
2084#[cfg(test)]
2085mod tests {
2086    use super::*;
2087    use peniko::{Blob, ImageAlphaType};
2088
2089    fn image(width: u32, height: u32) -> ImageData {
2090        let len = (width as usize) * (height as usize) * 4;
2091        ImageData {
2092            data: Blob::new(Arc::new(vec![255_u8; len])),
2093            format: ImageFormat::Rgba8,
2094            alpha_type: ImageAlphaType::Alpha,
2095            width,
2096            height,
2097        }
2098    }
2099
2100    fn residency() -> ImageResidency {
2101        ImageResidency::new(AtlasBudget {
2102            atlas_size: (64, 64),
2103            max_atlases: 2,
2104        })
2105    }
2106
2107    #[test]
2108    fn a_mobile_budget_is_never_the_vello_default() {
2109        let default = AtlasConfig::default();
2110        for budget in [AtlasBudget::MOBILE, AtlasBudget::DESKTOP] {
2111            assert_ne!(budget.atlas_size, default.atlas_size);
2112            assert!(budget.total_bytes() < 4096 * 4096 * ATLAS_FORMAT_BYTES * 8);
2113        }
2114        assert_eq!(AtlasBudget::MOBILE.total_bytes(), 16 << 20);
2115        assert_eq!(AtlasBudget::DESKTOP.total_bytes(), 128 << 20);
2116    }
2117
2118    #[test]
2119    fn a_downlevel_adapter_takes_the_mobile_budget() {
2120        let webgl2 = TierCaps::fake(DownlevelProfile::WebGl2);
2121        assert!(is_mobile_tier(&webgl2));
2122
2123        let mut tiler = TierCaps::fake(DownlevelProfile::Full);
2124        assert!(!is_mobile_tier(&tiler));
2125        tiler.transient_saves_memory = true;
2126        assert!(is_mobile_tier(&tiler));
2127    }
2128
2129    fn named(name: &str) -> TierCaps {
2130        let mut caps = TierCaps::fake(DownlevelProfile::Full);
2131        caps.adapter_name = name.to_string();
2132        caps
2133    }
2134
2135    #[test]
2136    fn the_adapter_table_names_the_mobile_and_desktop_families() {
2137        for name in [
2138            "Adreno (TM) 618",
2139            "Adreno (TM) 660",
2140            "Mali-G78 MP14",
2141            "Immortalis-G715",
2142            "PowerVR B-Series BXM-8-256",
2143            "Samsung Xclipse 920",
2144            "Maleoon 910",
2145            "Apple A15 GPU",
2146            "NVIDIA Tegra X1",
2147        ] {
2148            assert_eq!(adapter_family(name), Some(AdapterFamily::Mobile), "{name}");
2149        }
2150        for name in [
2151            "NVIDIA T400",
2152            "NVIDIA GeForce RTX 4090",
2153            "Quadro P1000",
2154            "AMD Radeon RX 7900 XTX",
2155            "Intel(R) UHD Graphics 770",
2156        ] {
2157            assert_eq!(adapter_family(name), Some(AdapterFamily::Desktop), "{name}");
2158        }
2159        for name in ["llvmpipe (LLVM 19.1.7, 256 bits)", "fake-full", ""] {
2160            assert_eq!(adapter_family(name), None, "{name}");
2161        }
2162    }
2163
2164    #[test]
2165    fn the_tier_table_overrides_a_mobile_driver_that_answers_like_a_desktop() {
2166        const GIB: u64 = 1 << 30;
2167        // Adreno 618/620 answer full profile and no transient savings — the
2168        // desktop answers — and used to take the desktop budget on them.
2169        let adreno = named("Adreno (TM) 618");
2170        assert!(!is_mobile_tier(&adreno));
2171
2172        let small = choose_tier(&adreno, Some(3 * GIB));
2173        assert_eq!(
2174            (small.tier, small.rule),
2175            (AtlasTier::Mobile, TierRule::Table)
2176        );
2177        assert_eq!(AtlasBudget::for_choice(small, &adreno), AtlasBudget::MOBILE);
2178
2179        let unknown = choose_tier(&adreno, None);
2180        assert_eq!(
2181            unknown.tier,
2182            AtlasTier::Mobile,
2183            "unknown memory earns nothing"
2184        );
2185
2186        let large = choose_tier(&adreno, Some(6 * GIB));
2187        assert_eq!(
2188            (large.tier, large.rule),
2189            (AtlasTier::MobileLarge, TierRule::Table)
2190        );
2191        assert_eq!(
2192            AtlasBudget::for_choice(large, &adreno),
2193            AtlasBudget::MOBILE_LARGE_WIDE
2194        );
2195        assert_eq!(
2196            AtlasBudget::for_choice(choose_tier(&adreno, Some(MOBILE_LARGE_MIN_MEMORY)), &adreno),
2197            AtlasBudget::MOBILE_LARGE_WIDE,
2198            "the threshold is inclusive"
2199        );
2200        assert_eq!(AtlasBudget::MOBILE_LARGE_WIDE.total_bytes(), 64 << 20);
2201        assert_eq!(AtlasBudget::MOBILE_LARGE_DEEP.total_bytes(), 64 << 20);
2202
2203        // A desktop family stays on the desktop budget whatever it answers.
2204        let mut nvidia = named("NVIDIA T400");
2205        nvidia.transient_saves_memory = true;
2206        let desktop = choose_tier(&nvidia, Some(GIB));
2207        assert_eq!(
2208            (desktop.tier, desktop.rule),
2209            (AtlasTier::Desktop, TierRule::Table)
2210        );
2211    }
2212
2213    #[test]
2214    fn the_large_mobile_shape_is_the_one_the_adapter_can_create() {
2215        let mut caps = named("Mali-G78 MP14");
2216        assert_eq!(
2217            AtlasBudget::mobile_large(&caps),
2218            Some(AtlasBudget::MOBILE_LARGE_WIDE)
2219        );
2220
2221        // An edge short of 2048 takes the deep shape instead.
2222        caps.max_texture_dimension_2d = 1024;
2223        assert_eq!(
2224            AtlasBudget::mobile_large(&caps),
2225            Some(AtlasBudget::MOBILE_LARGE_DEEP)
2226        );
2227
2228        // Neither shape fits: the tier is the plain mobile one, never a large
2229        // budget clamped below it.
2230        caps.max_texture_array_layers = 8;
2231        assert_eq!(AtlasBudget::mobile_large(&caps), None);
2232        let choice = choose_tier(&caps, Some(16 << 30));
2233        assert_eq!(choice.tier, AtlasTier::Mobile);
2234        assert_eq!(AtlasBudget::for_choice(choice, &caps), AtlasBudget::MOBILE);
2235    }
2236
2237    #[test]
2238    fn an_unnamed_adapter_keeps_the_capability_read_and_downlevel_wins_over_the_table() {
2239        let unnamed = TierCaps::fake(DownlevelProfile::Full);
2240        let choice = choose_tier(&unnamed, Some(64 << 30));
2241        assert_eq!(
2242            (choice.tier, choice.rule),
2243            (AtlasTier::Desktop, TierRule::Heuristic)
2244        );
2245
2246        let mut tiler = unnamed.clone();
2247        tiler.transient_saves_memory = true;
2248        let choice = choose_tier(&tiler, Some(64 << 30));
2249        assert_eq!(
2250            (choice.tier, choice.rule),
2251            (AtlasTier::Mobile, TierRule::Heuristic),
2252            "the heuristic never reaches the large tier"
2253        );
2254
2255        let mut gl_nvidia = TierCaps::fake(DownlevelProfile::WebGl2);
2256        gl_nvidia.adapter_name = "ANGLE (NVIDIA, NVIDIA T400)".to_string();
2257        let choice = choose_tier(&gl_nvidia, Some(64 << 30));
2258        assert_eq!(
2259            (choice.tier, choice.rule),
2260            (AtlasTier::Mobile, TierRule::Downlevel)
2261        );
2262    }
2263
2264    #[test]
2265    fn mem_total_is_read_in_bytes_from_meminfo() {
2266        let meminfo = "MemTotal:        5768520 kB\nMemFree:          123456 kB\n";
2267        assert_eq!(parse_mem_total(meminfo), Some(5_768_520 * 1024));
2268        assert_eq!(parse_mem_total("MemFree: 1 kB\n"), None);
2269        assert_eq!(parse_mem_total("MemTotal: lots\n"), None);
2270        assert_eq!(parse_mem_total("MemTotal: 1024 MB\n"), None);
2271    }
2272
2273    #[test]
2274    fn a_budget_is_clamped_to_the_adapters_own_ceilings() {
2275        let mut caps = TierCaps::fake(DownlevelProfile::Full);
2276        caps.max_texture_dimension_2d = 1024;
2277        caps.max_texture_array_layers = 2;
2278
2279        let budget = AtlasBudget::DESKTOP.clamped(&caps);
2280        assert_eq!(budget.atlas_size, (1024, 1024));
2281        assert_eq!(budget.max_atlases, 2);
2282
2283        // The eight-bit `atlas_index` field is a ceiling of its own.
2284        caps.max_texture_array_layers = 4096;
2285        let budget = AtlasBudget {
2286            atlas_size: (256, 256),
2287            max_atlases: 4096,
2288        }
2289        .clamped(&caps);
2290        assert_eq!(budget.max_atlases, MAX_ATLAS_LAYERS);
2291    }
2292
2293    #[test]
2294    fn the_first_layer_is_created_lazily() {
2295        let residency = residency();
2296        assert_eq!(residency.layers(), 0);
2297        assert_eq!(residency.entry_count(), 0);
2298    }
2299
2300    #[test]
2301    fn one_image_over_many_frames_uploads_exactly_once() {
2302        let mut residency = residency();
2303        let data = image(8, 8);
2304        let mut uploads = 0;
2305
2306        for _ in 0..60 {
2307            residency.begin_frame();
2308            residency.resolve(&data).expect("8x8 fits a 64x64 atlas");
2309            uploads += residency.uploads().len();
2310            // Every frame here is a frame that reached the atlas.
2311            residency.acknowledge_plan();
2312        }
2313
2314        assert_eq!(uploads, 1, "residency survives every frame that draws it");
2315        assert_eq!(residency.entry_count(), 1);
2316        assert_eq!(residency.layers(), 1);
2317    }
2318
2319    #[test]
2320    fn an_unseen_image_is_reaped_and_its_padded_region_cleared() {
2321        let mut residency = residency();
2322        let data = image(8, 8);
2323
2324        residency.begin_frame();
2325        let resident = residency.resolve(&data).expect("fits");
2326        residency.acknowledge_plan();
2327
2328        for _ in 0..MAX_UNSEEN_FRAMES {
2329            residency.begin_frame();
2330            assert!(residency.evictions().is_empty(), "still inside the window");
2331        }
2332
2333        residency.begin_frame();
2334        assert_eq!(residency.entry_count(), 0);
2335        assert_eq!(
2336            residency.evictions(),
2337            &[padded(resident.region, ATLAS_PADDING)]
2338        );
2339
2340        // The freed rectangle is available again, and the reallocation
2341        // schedules a fresh upload.
2342        residency.resolve(&data).expect("fits");
2343        assert_eq!(residency.uploads().len(), 1);
2344    }
2345
2346    #[test]
2347    fn an_image_larger_than_the_atlas_is_minified_to_fit_rather_than_skipped() {
2348        let mut residency = residency();
2349        residency.begin_frame();
2350
2351        // Twice the layer's width at a 16:1 aspect ratio: the width sets the
2352        // scale, and the height follows it rather than being fitted on its own.
2353        let resident = residency.resolve(&image(128, 8)).expect("minified to fit");
2354
2355        assert_eq!(resident.region.size, [64, 4]);
2356        assert_eq!(resident.natural, [128, 8]);
2357        assert_eq!(resident.minify_scale(), Some((0.5, 0.5)));
2358        assert_eq!(residency.minified(), 1);
2359        assert_eq!(residency.skipped(), 0, "a big image is not a skipped one");
2360        assert_eq!(residency.uploads().len(), 1);
2361        let upload = &residency.uploads()[0];
2362        assert_eq!(
2363            (upload.pixels.width(), upload.pixels.height()),
2364            (64, 4),
2365            "the upload carries exactly the region's texels"
2366        );
2367    }
2368
2369    #[test]
2370    fn an_atlas_with_no_room_left_is_still_a_skip() {
2371        // Minification fits an image to a LAYER, not to the space left in one:
2372        // a budget already full refuses the next allocation as it always did.
2373        let mut residency = ImageResidency::new(AtlasBudget {
2374            atlas_size: (64, 64),
2375            max_atlases: 1,
2376        });
2377        residency.begin_frame();
2378        residency
2379            .resolve(&image(64, 64))
2380            .expect("fills the one layer");
2381
2382        let skip = residency
2383            .resolve(&image(32, 32))
2384            .expect_err("nothing is left to allocate from");
2385        assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
2386        assert_eq!(residency.skipped(), 1);
2387        assert_eq!(residency.minified(), 0);
2388    }
2389
2390    #[test]
2391    fn a_minified_image_stays_resident_across_frames_at_its_declared_extent() {
2392        // Residency is keyed on the SOURCE extent, not the resident one: an
2393        // entry matched against its own minified rectangle would miss every
2394        // frame and re-upload the image on each of them.
2395        let mut residency = residency();
2396        let data = image(128, 8);
2397
2398        for _ in 0..8 {
2399            residency.begin_frame();
2400            let resident = residency.resolve(&data).expect("minified to fit");
2401            assert_eq!(resident.natural, [128, 8]);
2402            residency.acknowledge_plan();
2403        }
2404
2405        assert_eq!(residency.entry_count(), 1);
2406        assert_eq!(
2407            residency.minified(),
2408            1,
2409            "downsampled once, not once a frame"
2410        );
2411        assert!(
2412            residency.uploads().is_empty(),
2413            "no re-upload after the first"
2414        );
2415    }
2416
2417    #[test]
2418    fn a_fit_extent_preserves_the_aspect_ratio_and_never_grows() {
2419        let atlas = (64, 64);
2420        // Already inside the layer: unchanged, both axes.
2421        assert_eq!(fit_extent([64, 64], atlas), (64, 64));
2422        assert_eq!(fit_extent([8, 4], atlas), (8, 4));
2423
2424        // The `adv-huge-image` shape, at this budget's scale.
2425        assert_eq!(fit_extent([5000, 5000], (2048, 2048)), (2048, 2048));
2426
2427        // The tighter axis sets the scale for both.
2428        assert_eq!(fit_extent([128, 8], atlas), (64, 4));
2429        assert_eq!(fit_extent([8, 128], atlas), (4, 64));
2430        assert_eq!(fit_extent([1000, 10], atlas), (64, 1));
2431
2432        // A non-square layer is fitted per axis and still uniformly scaled.
2433        assert_eq!(fit_extent([400, 400], (100, 50)), (50, 50));
2434    }
2435
2436    #[test]
2437    fn an_extreme_aspect_ratio_still_fits_at_one_texel_rather_than_none() {
2438        // 10000:1 into a 64-square layer scales the short axis to 0.0064 texels.
2439        // A zero extent is not a rectangle the packer or the shader can
2440        // address, so it is held at one.
2441        let fitted = fit_extent([10_000, 1], (64, 64));
2442        assert_eq!(fitted, (64, 1));
2443
2444        let mut residency = residency();
2445        residency.begin_frame();
2446        let resident = residency
2447            .resolve(&image(10_000, 1))
2448            .expect("a sliver still becomes a rectangle");
2449        assert_eq!(resident.region.size, [64, 1]);
2450    }
2451
2452    #[test]
2453    fn a_uniform_source_minifies_to_exactly_its_own_colour() {
2454        // What `adv-huge-image` rests on: a uniform source comes back bit
2455        // identical whatever the scale, so the case's `Exact` probe is a
2456        // property of the filter rather than a tolerance.
2457        let texel = PremulRgba8 {
2458            r: 255,
2459            g: 0,
2460            b: 255,
2461            a: 255,
2462        };
2463        let source = Pixmap::from_parts_with_opacity(vec![texel; 100 * 100], 100, 100, false);
2464
2465        let small = minify(&source, 7, 3);
2466        assert_eq!((small.width(), small.height()), (7, 3));
2467        assert!(small.data().iter().all(|out| *out == texel));
2468        assert!(
2469            !small.may_have_transparency(),
2470            "an opaque source stays opaque"
2471        );
2472    }
2473
2474    #[test]
2475    fn a_box_filter_averages_the_source_texels_each_output_covers() {
2476        // A 2x1 source of black and white halves down to one texel, which must
2477        // be their mean rather than either end.
2478        let black = PremulRgba8 {
2479            r: 0,
2480            g: 0,
2481            b: 0,
2482            a: 255,
2483        };
2484        let white = PremulRgba8 {
2485            r: 255,
2486            g: 255,
2487            b: 255,
2488            a: 255,
2489        };
2490        let source = Pixmap::from_parts_with_opacity(vec![black, white], 2, 1, false);
2491
2492        let small = minify(&source, 1, 1);
2493        assert_eq!((small.width(), small.height()), (1, 1));
2494        // 255 / 2 rounded to nearest.
2495        assert_eq!(small.data()[0].r, 128);
2496        assert_eq!(small.data()[0].a, 255);
2497
2498        // Transparency is recomputed from the result, not inherited: averaging
2499        // an opaque and a fully transparent texel produces a partial one.
2500        let clear = PremulRgba8 {
2501            r: 0,
2502            g: 0,
2503            b: 0,
2504            a: 0,
2505        };
2506        let mixed = Pixmap::from_parts_with_opacity(vec![white, clear], 2, 1, true);
2507        let small = minify(&mixed, 1, 1);
2508        assert_eq!(small.data()[0].a, 128);
2509        assert!(small.may_have_transparency());
2510    }
2511
2512    #[test]
2513    fn minify_never_magnifies() {
2514        let texel = PremulRgba8 {
2515            r: 1,
2516            g: 2,
2517            b: 3,
2518            a: 255,
2519        };
2520        let source = Pixmap::from_parts_with_opacity(vec![texel; 4], 2, 2, false);
2521
2522        // A request above the source's own extent is clamped to it rather than
2523        // inventing texels the box filter has no arm for.
2524        let same = minify(&source, 8, 8);
2525        assert_eq!((same.width(), same.height()), (2, 2));
2526        assert!(same.data().iter().all(|out| *out == texel));
2527    }
2528
2529    #[test]
2530    fn a_malformed_or_degenerate_image_is_refused_before_the_conversion() {
2531        let mut short = image(4, 4);
2532        short.data = Blob::new(Arc::new(vec![0_u8; 8]));
2533        assert!(matches!(
2534            check_supported(&short),
2535            Err(ImageSkip::MalformedPixels { .. })
2536        ));
2537
2538        assert!(matches!(
2539            check_supported(&image(0, 4)),
2540            Err(ImageSkip::DegenerateExtent { .. })
2541        ));
2542
2543        let mut huge = image(1, 1);
2544        huge.width = MAX_IMAGE_DIMENSION + 1;
2545        assert!(matches!(
2546            check_supported(&huge),
2547            Err(ImageSkip::TooLarge { .. })
2548        ));
2549    }
2550
2551    #[test]
2552    fn two_images_share_a_layer_and_hold_distinct_rectangles() {
2553        let mut residency = residency();
2554        residency.begin_frame();
2555
2556        let first = residency.resolve(&image(8, 8)).expect("fits");
2557        let second = residency.resolve(&image(16, 16)).expect("fits");
2558
2559        assert_ne!(first.id, second.id);
2560        assert_eq!(first.region.layer, second.region.layer);
2561        assert_ne!(first.region.offset, second.region.offset);
2562        assert_eq!(residency.uploads().len(), 2);
2563        assert_eq!(residency.layers(), 1);
2564    }
2565
2566    #[test]
2567    fn a_second_layer_is_created_when_the_first_is_full() {
2568        let mut residency = residency();
2569        residency.begin_frame();
2570
2571        // Two 64x48 images cannot share one 64x64 layer.
2572        let first = residency.resolve(&image(64, 48)).expect("fits a layer");
2573        let second = residency.resolve(&image(64, 48)).expect("fits the next");
2574
2575        assert_eq!(first.region.layer, 0);
2576        assert_eq!(second.region.layer, 1);
2577        assert_eq!(residency.layers(), 2);
2578    }
2579
2580    #[test]
2581    fn a_slot_taken_through_the_shared_allocator_is_a_distinct_id_on_the_same_budget() {
2582        // The allocator half of the glyph/image seam, stated without the glyph
2583        // policy: whatever else packs into this cache draws from the same id
2584        // space and the same layer allowance, and its pages are visible in the
2585        // depth the array is built to.
2586        let mut residency = residency();
2587        residency.begin_frame();
2588        let resident = residency.resolve(&image(64, 48)).expect("fills a layer");
2589
2590        // Big enough that it cannot share the layer the image took, so it must
2591        // create the second one rather than report the first.
2592        let borrowed = residency
2593            .allocator_mut()
2594            .allocate(64, 48, ATLAS_PADDING)
2595            .expect("the second layer is free");
2596
2597        assert_ne!(
2598            resident.id, borrowed,
2599            "one cache never hands the same slot index to two live occupants"
2600        );
2601        assert_eq!(
2602            residency.layers(),
2603            2,
2604            "a layer created by the other class still has to be in the array"
2605        );
2606        assert_eq!(
2607            residency
2608                .allocator()
2609                .get(resident.id)
2610                .map(|resource| resource.size()),
2611            Some(resident.region.size),
2612            "and neither allocation disturbed the other's rectangle"
2613        );
2614    }
2615
2616    #[test]
2617    fn a_disabled_residency_refuses_every_image_and_touches_no_atlas() {
2618        let mut residency = ImageResidency::disabled(AtlasBudget::MOBILE);
2619        assert!(residency.is_disabled());
2620
2621        residency.begin_frame();
2622        let skip = residency
2623            .resolve(&image(8, 8))
2624            .expect_err("the kill switch refuses every image");
2625
2626        assert_eq!(skip, ImageSkip::AtlasDisabled);
2627        assert_eq!(residency.entry_count(), 0);
2628        assert_eq!(residency.layers(), 0, "no atlas layer is ever created");
2629        assert!(residency.uploads().is_empty());
2630        assert_eq!(residency.skipped(), 1);
2631    }
2632
2633    #[test]
2634    fn an_enabled_residency_is_what_an_unset_kill_switch_produces() {
2635        // The default test environment leaves `FRUST_ENGINE_NO_ATLAS` unset, so
2636        // this pins that `new` consults the switch rather than hardcoding
2637        // either answer; the disabled half is pinned above through the
2638        // constructor the switch selects.
2639        assert!(!ImageResidency::new(AtlasBudget::MOBILE).is_disabled());
2640    }
2641
2642    #[test]
2643    fn reading_the_plan_leaves_it_pending_and_acknowledging_it_clears_it() {
2644        let mut residency = residency();
2645        residency.begin_frame();
2646        residency.resolve(&image(8, 8)).expect("fits");
2647
2648        let (evictions, uploads) = residency.plan();
2649        assert!(evictions.is_empty());
2650        assert_eq!(uploads.len(), 1);
2651        assert!(
2652            residency.has_pending_plan(),
2653            "reading the plan is not servicing it"
2654        );
2655
2656        residency.acknowledge_plan();
2657        assert!(!residency.has_pending_plan());
2658        assert!(residency.uploads().is_empty());
2659        assert!(residency.evictions().is_empty());
2660    }
2661
2662    #[test]
2663    fn a_clear_is_dropped_only_for_a_rectangle_an_upload_covers_exactly() {
2664        let region = |layer, x, y, w, h| AtlasRegion {
2665            layer,
2666            offset: [x, y],
2667            size: [w, h],
2668        };
2669        let upload = |region| ImageUpload {
2670            id: ImageId::new(0),
2671            region,
2672            natural: region.size,
2673            pixels: Arc::new(Pixmap::new(
2674                u16::try_from(region.size[0]).expect("small"),
2675                u16::try_from(region.size[1]).expect("small"),
2676            )),
2677            may_have_transparency: false,
2678        };
2679
2680        let exact = region(0, 0, 0, 16, 16);
2681        let other_layer = region(1, 0, 0, 16, 16);
2682        let partial = region(0, 16, 0, 16, 16);
2683        let evictions = [exact, other_layer, partial];
2684        let uploads = [upload(exact), upload(region(0, 16, 0, 16, 8))];
2685
2686        assert_eq!(
2687            uncovered_clears(&evictions, &uploads),
2688            vec![other_layer, partial],
2689            "only the exactly covered clear goes, the rest keep their order"
2690        );
2691        assert_eq!(uncovered_clears(&evictions, &[]), evictions.to_vec());
2692        assert!(uncovered_clears(&[], &uploads).is_empty());
2693    }
2694
2695    #[test]
2696    fn the_clear_skip_is_tied_to_zero_padding() {
2697        // Evictions are reported as `padded(region, ATLAS_PADDING)`, uploads
2698        // carry the raw allocation; the exact-cover match needs them equal.
2699        let raw = AtlasRegion {
2700            layer: 0,
2701            offset: [4, 8],
2702            size: [16, 16],
2703        };
2704        assert_eq!(padded(raw, ATLAS_PADDING), raw);
2705    }
2706
2707    #[test]
2708    fn a_pressure_eviction_into_the_same_rectangle_plans_no_clear() {
2709        // One slot: the second image can only take the first one's rectangle.
2710        let mut residency = ImageResidency::new(AtlasBudget {
2711            atlas_size: (16, 16),
2712            max_atlases: 1,
2713        });
2714        residency.begin_frame();
2715        let first = residency.resolve(&image(16, 16)).expect("fits");
2716        residency.acknowledge_plan();
2717
2718        residency.begin_frame();
2719        let second = residency
2720            .resolve(&image(16, 16))
2721            .expect("displaces the first");
2722        assert_eq!(second.region, first.region);
2723        assert_eq!(
2724            residency.evictions(),
2725            &[first.region],
2726            "the eviction itself is still pending"
2727        );
2728
2729        let (clears, uploads) = residency.plan();
2730        assert!(clears.is_empty(), "but the plan issues no clear for it");
2731        assert_eq!(uploads.len(), 1);
2732
2733        residency.acknowledge_plan();
2734        assert!(!residency.has_pending_plan());
2735    }
2736
2737    #[test]
2738    fn an_unacknowledged_upload_is_re_emitted_until_it_is_serviced() {
2739        // The invariant the whole pending plan exists for: a frame refused
2740        // after compiling never wrote these texels, so the entry map claiming
2741        // the image is resident has to stay answerable by a later frame's plan.
2742        let mut residency = residency();
2743        let data = image(8, 8);
2744
2745        residency.begin_frame();
2746        let first = residency.resolve(&data).expect("fits");
2747        let region = residency.uploads()[0].region;
2748
2749        for _ in 0..4 {
2750            residency.begin_frame();
2751            let again = residency.resolve(&data).expect("still resident");
2752            assert_eq!(again.region, first.region, "residency does not move");
2753            assert_eq!(
2754                residency.uploads().len(),
2755                1,
2756                "the same upload is re-offered, never duplicated"
2757            );
2758            assert_eq!(residency.uploads()[0].region, region);
2759        }
2760
2761        residency.acknowledge_plan();
2762        residency.begin_frame();
2763        residency.resolve(&data).expect("still resident");
2764        assert!(
2765            residency.uploads().is_empty(),
2766            "a serviced upload is never offered again"
2767        );
2768    }
2769
2770    #[test]
2771    fn reaping_an_unacknowledged_entry_withdraws_its_upload_with_it() {
2772        // Otherwise the plan would clear the rectangle and then write the
2773        // evicted image straight back into it, over whatever the packer handed
2774        // that space to next.
2775        let mut residency = residency();
2776        let data = image(8, 8);
2777
2778        residency.begin_frame();
2779        let resident = residency.resolve(&data).expect("fits");
2780        assert_eq!(residency.uploads().len(), 1);
2781
2782        for _ in 0..=MAX_UNSEEN_FRAMES {
2783            residency.begin_frame();
2784        }
2785
2786        assert_eq!(residency.entry_count(), 0);
2787        assert!(
2788            residency.uploads().is_empty(),
2789            "the reaped entry's unserviced upload goes with it"
2790        );
2791        assert_eq!(
2792            residency.evictions(),
2793            &[padded(resident.region, ATLAS_PADDING)]
2794        );
2795    }
2796
2797    #[test]
2798    fn every_pending_upload_lies_inside_the_atlas_its_layer_count_asks_for() {
2799        // What keeps a consumer from recording where an image lives and then
2800        // sampling a rectangle the array refused to write: the plan can only
2801        // ever name regions the budget and the reported depth already admit.
2802        let mut residency = residency();
2803        residency.begin_frame();
2804        residency.resolve(&image(64, 48)).expect("fills a layer");
2805        residency.resolve(&image(64, 48)).expect("takes the next");
2806
2807        let budget = residency.budget();
2808        let layers = residency.layers();
2809        assert_eq!(layers, 2);
2810        for upload in residency.uploads() {
2811            assert!(
2812                budget.contains(upload.region, layers),
2813                "{:?} is outside a {layers}-layer {budget:?}",
2814                upload.region
2815            );
2816        }
2817    }
2818
2819    /// A budget holding exactly two 64-square images: one per layer, with no
2820    /// room to pack a third anywhere, so "the atlas is full" is reachable in
2821    /// two allocations and needs no fragmentation reasoning.
2822    fn two_slot_residency() -> ImageResidency {
2823        ImageResidency::new(AtlasBudget {
2824            atlas_size: (64, 64),
2825            max_atlases: 2,
2826        })
2827    }
2828
2829    #[test]
2830    fn a_working_set_larger_than_the_atlas_re_uploads_rather_than_skipping() {
2831        // The scrolling-list shape: more distinct images pass the viewport
2832        // inside one age window than the atlas has rectangles for. Every draw
2833        // must still resolve — the price is a re-upload, never a hole.
2834        let mut residency = two_slot_residency();
2835        let images: Vec<ImageData> = (0..12).map(|_| image(64, 64)).collect();
2836
2837        for data in &images {
2838            residency.begin_frame();
2839            residency
2840                .resolve(data)
2841                .expect("a full atlas gives a rectangle back rather than refusing");
2842            residency.acknowledge_plan();
2843        }
2844
2845        assert_eq!(residency.skipped(), 0, "no draw was ever dropped");
2846        assert_eq!(
2847            residency.pressure_evictions(),
2848            10,
2849            "the ten images past the two slots each displaced one"
2850        );
2851        assert_eq!(residency.entry_count(), 2, "and residency stays bounded");
2852    }
2853
2854    #[test]
2855    fn an_image_resolved_this_frame_is_never_the_one_evicted() {
2856        // The ordering contract's own boundary: the plan clears before it
2857        // uploads, so a rectangle this frame already resolved cannot be handed
2858        // back inside it. A frame whose own working set outgrows the atlas is
2859        // still a skip, and nothing is scheduled for clearing.
2860        let mut residency = two_slot_residency();
2861        residency.begin_frame();
2862        residency.resolve(&image(64, 64)).expect("takes layer 0");
2863        residency.resolve(&image(64, 64)).expect("takes layer 1");
2864
2865        let skip = residency
2866            .resolve(&image(64, 64))
2867            .expect_err("nothing this frame drew is evictable");
2868
2869        assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
2870        assert_eq!(residency.skipped(), 1);
2871        assert_eq!(residency.pressure_evictions(), 0);
2872        assert_eq!(residency.entry_count(), 2, "both occupants stayed put");
2873        assert!(
2874            residency.evictions().is_empty(),
2875            "no region a frame is about to sample was scheduled for clearing"
2876        );
2877    }
2878
2879    #[test]
2880    fn the_least_recently_seen_image_is_the_one_displaced() {
2881        let mut residency = two_slot_residency();
2882        let old = image(64, 64);
2883        let refreshed = image(64, 64);
2884
2885        residency.begin_frame();
2886        let refreshed_region = residency.resolve(&refreshed).expect("takes a layer").region;
2887        residency.begin_frame();
2888        let old_region = residency.resolve(&old).expect("takes the other").region;
2889        residency.acknowledge_plan();
2890
2891        // The older entry is drawn again, which makes the *other* one the least
2892        // recently seen despite having been made resident second.
2893        residency.begin_frame();
2894        residency.resolve(&refreshed).expect("still resident");
2895        let arrival = residency.resolve(&image(64, 64)).expect("displaces one");
2896
2897        assert_eq!(residency.pressure_evictions(), 1);
2898        assert_eq!(
2899            arrival.region, old_region,
2900            "the rectangle taken is the least recently seen one"
2901        );
2902        assert_eq!(
2903            residency.resolve(&refreshed).map(|again| again.region),
2904            Ok(refreshed_region),
2905            "the image drawn this frame kept its own"
2906        );
2907    }
2908
2909    #[test]
2910    fn a_pressure_eviction_reports_its_rectangle_once_and_withdraws_its_upload() {
2911        let mut residency = two_slot_residency();
2912        let displaced = image(64, 64);
2913
2914        residency.begin_frame();
2915        let displaced_region = residency.resolve(&displaced).expect("takes a layer").region;
2916        residency.resolve(&image(64, 64)).expect("takes the other");
2917        // Deliberately NOT acknowledged: the displaced image still has an
2918        // upload pending, which must go with its entry rather than be written
2919        // into a rectangle its new occupant now owns.
2920        assert_eq!(residency.uploads().len(), 2);
2921
2922        residency.begin_frame();
2923        let arrival = residency
2924            .resolve(&image(64, 64))
2925            .expect("displaces the oldest");
2926
2927        assert_eq!(
2928            residency.evictions(),
2929            &[padded(displaced_region, ATLAS_PADDING)],
2930            "exactly one rectangle, reported exactly once"
2931        );
2932        assert_eq!(
2933            arrival.region, displaced_region,
2934            "and the freed rectangle is what the new occupant took"
2935        );
2936        assert_eq!(
2937            residency.uploads().len(),
2938            2,
2939            "the displaced image's unserviced upload went with its entry — \
2940             writing it would put an evicted image back over its successor"
2941        );
2942    }
2943
2944    #[test]
2945    fn an_age_reap_and_a_pressure_eviction_are_counted_separately() {
2946        // The two bounds answer different questions: one says a screen moved
2947        // on, the other says the atlas is too small for what is on it.
2948        let mut residency = two_slot_residency();
2949        let data = image(64, 64);
2950
2951        residency.begin_frame();
2952        residency.resolve(&data).expect("fits");
2953        for _ in 0..=MAX_UNSEEN_FRAMES {
2954            residency.begin_frame();
2955        }
2956
2957        assert_eq!(residency.entry_count(), 0, "the age reap ran");
2958        assert_eq!(
2959            residency.pressure_evictions(),
2960            0,
2961            "an age reap is not a pressure eviction"
2962        );
2963        assert_eq!(residency.frame_pressure_evictions(), 0);
2964    }
2965
2966    #[test]
2967    fn the_per_frame_eviction_count_is_this_frames_alone() {
2968        let mut residency = two_slot_residency();
2969        residency.begin_frame();
2970        residency.resolve(&image(64, 64)).expect("takes a layer");
2971        residency.resolve(&image(64, 64)).expect("takes the other");
2972
2973        residency.begin_frame();
2974        residency.resolve(&image(64, 64)).expect("displaces one");
2975        assert_eq!(residency.frame_pressure_evictions(), 1);
2976        assert_eq!(residency.pressure_evictions(), 1);
2977
2978        residency.begin_frame();
2979        assert_eq!(
2980            residency.frame_pressure_evictions(),
2981            0,
2982            "a new frame starts from nothing"
2983        );
2984        assert_eq!(
2985            residency.pressure_evictions(),
2986            1,
2987            "while the lifetime total keeps counting"
2988        );
2989    }
2990
2991    /// A budget of exactly four 32-square cells in one layer, so "the atlas is
2992    /// full" needs no reasoning about how a packer splits free space and a
2993    /// request that fits *no* arrangement of the holes is one rectangle.
2994    fn four_slot_residency() -> ImageResidency {
2995        ImageResidency::new(AtlasBudget {
2996            atlas_size: (64, 64),
2997            max_atlases: 1,
2998        })
2999    }
3000
3001    #[test]
3002    fn a_request_no_candidate_set_can_place_is_refused_before_anything_is_freed() {
3003        // The storm, at the cache seam. One 32-square is this frame's and three
3004        // are last frame's; a 48-square fits none of the holes those three
3005        // would leave while the fourth is resident. Freeing them buys nothing
3006        // — the draw is skipped either way — and costs three clears plus three
3007        // re-uploads, every frame the scene repeats. So the refusal has to come
3008        // before the first rectangle is handed back.
3009        let mut residency = four_slot_residency();
3010        let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3011        residency.begin_frame();
3012        for data in &resident {
3013            residency
3014                .resolve(data)
3015                .expect("the four cells take four images");
3016        }
3017        residency.acknowledge_plan();
3018
3019        residency.begin_frame();
3020        residency.resolve(&resident[0]).expect("still resident");
3021
3022        let skip = residency
3023            .resolve(&image(48, 48))
3024            .expect_err("no arrangement of the three candidates admits a 48-square");
3025
3026        assert!(matches!(skip, ImageSkip::NoAtlasSpace { .. }));
3027        assert_eq!(
3028            residency.pressure_evictions(),
3029            0,
3030            "a request that could not have been placed displaced nothing"
3031        );
3032        assert_eq!(residency.entry_count(), 4, "the residency is untouched");
3033        assert!(
3034            residency.evictions().is_empty(),
3035            "and nothing was scheduled for clearing on the way to failing"
3036        );
3037        assert!(residency.uploads().is_empty());
3038    }
3039
3040    #[test]
3041    fn a_request_larger_than_every_candidate_together_is_refused_on_area_alone() {
3042        // The cheaper half of the same precheck: freeing all three candidates
3043        // leaves three quarters of one layer, and the request wants all of it.
3044        // The area comes off the packer's own accounting, so it counts what the
3045        // glyph half of the allocator holds as well as this residency's own.
3046        let mut residency = four_slot_residency();
3047        let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3048        residency.begin_frame();
3049        for data in &resident {
3050            residency
3051                .resolve(data)
3052                .expect("the four cells take four images");
3053        }
3054        residency.acknowledge_plan();
3055
3056        residency.begin_frame();
3057        residency.resolve(&resident[0]).expect("still resident");
3058        residency
3059            .resolve(&image(64, 64))
3060            .expect_err("a whole layer is more than the three candidates hold");
3061
3062        assert_eq!(residency.pressure_evictions(), 0);
3063        assert_eq!(residency.entry_count(), 4);
3064        assert!(residency.evictions().is_empty());
3065    }
3066
3067    #[test]
3068    fn a_satisfiable_request_under_pressure_still_evicts_the_minimum() {
3069        // The other side of the precheck: it must not turn a working eviction
3070        // into a skip. One 32-square arriving against four resident ones is
3071        // satisfiable by freeing exactly one, and that is what it costs.
3072        let mut residency = four_slot_residency();
3073        let resident: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3074        residency.begin_frame();
3075        for data in &resident {
3076            residency
3077                .resolve(data)
3078                .expect("the four cells take four images");
3079        }
3080        residency.acknowledge_plan();
3081
3082        residency.begin_frame();
3083        residency.resolve(&image(32, 32)).expect("displaces one");
3084
3085        assert_eq!(
3086            residency.pressure_evictions(),
3087            1,
3088            "the minimum, not the lot"
3089        );
3090        assert_eq!(residency.entry_count(), 4);
3091        assert_eq!(residency.evictions().len(), 1);
3092    }
3093
3094    #[test]
3095    fn a_blob_redrawn_at_a_new_extent_never_frees_what_this_frame_resolved() {
3096        // The extent-mismatch release under the pressure path's own rule. The
3097        // plan clears before it uploads, so giving the rectangle back on a
3098        // frame whose first draw already resolved it would zero texels that
3099        // draw is about to sample.
3100        let square = image(16, 8);
3101        let transposed = ImageData {
3102            data: square.data.clone(),
3103            format: square.format,
3104            alpha_type: square.alpha_type,
3105            width: 8,
3106            height: 16,
3107        };
3108
3109        let mut residency = residency();
3110        residency.begin_frame();
3111        let first = residency.resolve(&square).expect("takes a rectangle");
3112        let skip = residency
3113            .resolve(&transposed)
3114            .expect_err("the second extent cannot take the first one's rectangle");
3115
3116        assert!(
3117            matches!(skip, ImageSkip::SameFrameExtentConflict { .. }),
3118            "a same-frame ordering conflict, not an atlas-capacity refusal: {skip:?}"
3119        );
3120        assert!(
3121            residency.evictions().is_empty(),
3122            "no clear over a rectangle this frame already sampled"
3123        );
3124        assert_eq!(residency.entry_count(), 1);
3125        assert_eq!(residency.uploads().len(), 1);
3126
3127        // On a later frame nothing has sampled it yet, so the release is
3128        // ordinary again and the new extent takes its place.
3129        residency.acknowledge_plan();
3130        residency.begin_frame();
3131        let second = residency.resolve(&transposed).expect("the new extent fits");
3132        assert_eq!(second.natural, [8, 16]);
3133        assert_eq!(
3134            residency.evictions(),
3135            &[padded(first.region, ATLAS_PADDING)],
3136            "the rectangle the old extent held is reported for clearing"
3137        );
3138    }
3139
3140    #[test]
3141    fn a_layer_admits_only_what_every_occupant_leaves_room_for() {
3142        // The planner's own necessary condition, in isolation. A placement has
3143        // to lie entirely to one side of each occupant, so a 32-square in one
3144        // corner of a 64-square layer leaves nothing a 48-square can occupy —
3145        // while a 32-square still has three cells to choose from.
3146        let corner = PressureRect {
3147            key: 1,
3148            layer: 0,
3149            x0: 0,
3150            y0: 0,
3151            x1: 32,
3152            y1: 32,
3153            area: 1024,
3154            last_seen: 0,
3155            slot: 0,
3156            freed: false,
3157        };
3158        let rects = [corner];
3159
3160        assert!(!layer_admits(&rects, 0, 48, 48, (64, 64)));
3161        assert!(layer_admits(&rects, 0, 32, 32, (64, 64)));
3162        assert!(layer_admits(&rects, 0, 32, 64, (64, 64)));
3163        // A freed rectangle is not an obstacle, and a layer nothing occupies
3164        // admits anything the layer itself can hold.
3165        assert!(layer_admits(
3166            &[PressureRect {
3167                freed: true,
3168                ..corner
3169            }],
3170            0,
3171            64,
3172            64,
3173            (64, 64)
3174        ));
3175        assert!(!layer_admits(&[], 0, 65, 64, (64, 64)));
3176        // An occupant on another layer says nothing about this one.
3177        assert!(layer_admits(
3178            &[PressureRect { layer: 1, ..corner }],
3179            0,
3180            64,
3181            64,
3182            (64, 64)
3183        ));
3184    }
3185
3186    #[test]
3187    fn a_plan_that_would_outrun_its_budget_frees_nothing() {
3188        // The bound is on the *search*, not only on the damage: a walk that
3189        // reaches the budget without finding a fit answers `None`, and the
3190        // caller has released nothing by then.
3191        let mut rects: Vec<PressureRect> = (0..MAX_PRESSURE_EVICTION_CANDIDATES + 8)
3192            .map(|index| {
3193                let index = u32::try_from(index).expect("fits");
3194                PressureRect {
3195                    key: u64::from(index),
3196                    layer: 0,
3197                    x0: index,
3198                    y0: 0,
3199                    x1: index + 1,
3200                    y1: 1,
3201                    area: 1,
3202                    last_seen: 0,
3203                    slot: index,
3204                    freed: false,
3205                }
3206            })
3207            .collect();
3208        let order: Vec<usize> = (0..rects.len()).collect();
3209        let mut free_area = vec![0_u64];
3210        let mut layers = Vec::new();
3211
3212        assert_eq!(
3213            plan_pressure_eviction(
3214                64,
3215                64,
3216                (64, 64),
3217                &mut rects,
3218                &order,
3219                &mut free_area,
3220                &mut layers
3221            ),
3222            None,
3223            "a walk that cannot fit the request inside its budget plans nothing"
3224        );
3225        assert!(
3226            rects.iter().filter(|rect| rect.freed).count() <= MAX_PRESSURE_EVICTION_CANDIDATES + 1,
3227            "and it stopped walking rather than marking the whole residency"
3228        );
3229    }
3230
3231    #[test]
3232    fn a_satisfiable_layer_is_not_starved_by_a_hopeless_ones_candidates() {
3233        // The multi-layer shape the per-layer budget exists for. Layer 0 is
3234        // "this frame's" 64x40 anchor plus more than
3235        // `MAX_PRESSURE_EVICTION_CANDIDATES` one-pixel stale fillers packed
3236        // into the 64x24 strip left over — every one of them stale (older
3237        // than the incoming request's frame) and none of them, individually
3238        // or together, ever able to free a 32-tall rectangle, since the strip
3239        // they sit in is only 24 pixels tall. Layer 1 is four 32-square
3240        // quadrants exactly tiling the layer, three "this frame's" and one
3241        // stale. Layers 2 and 3 are each one "this frame's" 64-square image,
3242        // filling them outright so the incoming request cannot simply grow
3243        // into an unused layer.
3244        //
3245        // A single pooled budget spends its whole allowance walking layer 0's
3246        // fillers — none of which can ever satisfy the request — and never
3247        // reaches layer 1's one satisfiable candidate at all. A per-layer
3248        // budget exhausts layer 0's own allowance the same way, then gives
3249        // layer 1 a fresh one and finds the fit immediately.
3250        let mut residency = ImageResidency::new(AtlasBudget {
3251            atlas_size: (64, 64),
3252            max_atlases: 4,
3253        });
3254
3255        let anchor = image(64, 40);
3256        let fillers: Vec<ImageData> = (0..MAX_PRESSURE_EVICTION_CANDIDATES + 6)
3257            .map(|_| image(1, 1))
3258            .collect();
3259        let quadrants: Vec<ImageData> = (0..4).map(|_| image(32, 32)).collect();
3260        let layer2_filler = image(64, 64);
3261        let layer3_filler = image(64, 64);
3262
3263        residency.begin_frame();
3264        residency
3265            .resolve(&anchor)
3266            .expect("the anchor takes layer 0's top");
3267        for filler in &fillers {
3268            residency
3269                .resolve(filler)
3270                .expect("a one-pixel filler always finds room in the leftover strip");
3271        }
3272        for quadrant in &quadrants {
3273            residency
3274                .resolve(quadrant)
3275                .expect("a 32-square quadrant packs layer 1's empty corners");
3276        }
3277        residency
3278            .resolve(&layer2_filler)
3279            .expect("a whole-layer image grows into a fresh layer 2");
3280        residency
3281            .resolve(&layer3_filler)
3282            .expect("a whole-layer image grows into a fresh layer 3");
3283        residency.acknowledge_plan();
3284        assert_eq!(
3285            residency.layers(),
3286            4,
3287            "fixture precondition: all four layers exist"
3288        );
3289
3290        // Refresh everything except the fillers and the first quadrant, so
3291        // only they are stale candidates when the request arrives — the
3292        // fillers sort ahead of the stale quadrant because they were
3293        // resolved first and share the same `last_seen`.
3294        residency.begin_frame();
3295        residency.resolve(&anchor).expect("still resident");
3296        residency.resolve(&quadrants[1]).expect("still resident");
3297        residency.resolve(&quadrants[2]).expect("still resident");
3298        residency.resolve(&quadrants[3]).expect("still resident");
3299        residency.resolve(&layer2_filler).expect("still resident");
3300        residency.resolve(&layer3_filler).expect("still resident");
3301
3302        let arrival = residency
3303            .resolve(&image(32, 32))
3304            .expect("a per-layer budget reaches layer 1's satisfiable candidate");
3305
3306        assert_eq!(
3307            residency.pressure_evictions(),
3308            1,
3309            "exactly the one quadrant needed, not layer 0's hopeless fillers"
3310        );
3311        assert_eq!(residency.evictions().len(), 1);
3312        assert_eq!(
3313            residency.evictions()[0].layer,
3314            arrival.region.layer,
3315            "the eviction and the arrival land on the same, single layer"
3316        );
3317        assert_ne!(
3318            arrival.region.layer, 0,
3319            "the winning layer is not the one full of hopeless fillers"
3320        );
3321    }
3322
3323    /// The tier line's own text, which only a `perf-trace` build has.
3324    #[cfg(feature = "perf-trace")]
3325    #[test]
3326    fn the_resolved_tier_line_names_the_tier_and_is_written_once() {
3327        let mobile = TierCaps::fake(DownlevelProfile::WebGl2);
3328        let choice = choose_tier(&mobile, None);
3329        assert_eq!(
3330            atlas_tier_line(&mobile, choice, AtlasBudget::MOBILE),
3331            "frust-perf atlas tier=mobile rule=downlevel budget=1024x1024x4 downlevel=WebGl2 \
3332             transient_saves_memory=false memory_mib=unknown adapter=fake-webgl2"
3333        );
3334
3335        let mut adreno = TierCaps::fake(DownlevelProfile::Full);
3336        adreno.adapter_name = "Adreno (TM) 660".to_string();
3337        let choice = choose_tier(&adreno, Some(8 << 30));
3338        assert_eq!(
3339            atlas_tier_line(&adreno, choice, AtlasBudget::MOBILE_LARGE_WIDE),
3340            "frust-perf atlas tier=mobile_large rule=table budget=2048x2048x4 downlevel=Full \
3341             transient_saves_memory=false memory_mib=8192 adapter=Adreno (TM) 660"
3342        );
3343
3344        // An adapter the table does not name says which capability answer
3345        // decided it.
3346        let mut tiler = TierCaps::fake(DownlevelProfile::Full);
3347        tiler.transient_saves_memory = true;
3348        let line = atlas_tier_line(&tiler, choose_tier(&tiler, None), AtlasBudget::MOBILE);
3349        assert!(line.contains("tier=mobile rule=heuristic"));
3350        assert!(line.contains("transient_saves_memory=true"));
3351
3352        let latch = TierLogOnce::new();
3353        let choice = choose_tier(&mobile, None);
3354        assert!(
3355            latch.emit(&mobile, choice, AtlasBudget::MOBILE),
3356            "the first call writes the line"
3357        );
3358        assert!(
3359            !latch.emit(&mobile, choice, AtlasBudget::MOBILE),
3360            "and no later one repeats it"
3361        );
3362    }
3363
3364    #[test]
3365    fn an_override_redistributes_a_tiers_memory_but_never_exceeds_it() {
3366        let tier = AtlasBudget::DESKTOP;
3367
3368        // The pathological override the adapter alone would admit: a
3369        // 16384-square layer is a gibibyte on its own, so clamping the layer
3370        // count could not have brought it inside the tier's 128 MiB.
3371        let huge = AtlasBudget {
3372            atlas_size: (16_384, 16_384),
3373            max_atlases: tier.max_atlases,
3374        }
3375        .within_total_bytes(tier.total_bytes());
3376        assert!(huge.total_bytes() <= tier.total_bytes());
3377        assert_eq!(huge.atlas_size.0, huge.atlas_size.1, "aspect preserved");
3378        assert!(huge.max_atlases >= 1);
3379
3380        // A modest override is left exactly as asked for.
3381        let modest = AtlasBudget {
3382            atlas_size: (1024, 1024),
3383            max_atlases: tier.max_atlases,
3384        }
3385        .within_total_bytes(tier.total_bytes());
3386        assert_eq!(modest.atlas_size, (1024, 1024));
3387        assert_eq!(modest.max_atlases, tier.max_atlases);
3388
3389        // An extreme aspect ratio floors one axis to nothing; it is raised back
3390        // to a texel and the other axis gives the room up, so the pair still
3391        // fits and neither axis is zero.
3392        let sliver = AtlasBudget {
3393            atlas_size: (65_535, 4),
3394            max_atlases: 1,
3395        }
3396        .within_total_bytes(64);
3397        assert!(sliver.atlas_size.0 >= 1 && sliver.atlas_size.1 >= 1);
3398        assert!(sliver.total_bytes() <= 64);
3399    }
3400}