Skip to main content

frust_engine/
renderer.rs

1//! [`EngineRenderer`]: the public seam a host drives one surface's 2D frames
2//! through.
3//!
4//! # The encode contract
5//!
6//! [`EngineRenderer::encode`] **records into a `wgpu::CommandEncoder` the
7//! caller owns and never submits it.** That is the whole point of the seam: a
8//! host compositing 2D over its own 3D content records its passes before and
9//! after the engine's into one encoder and submits once, and a submit hidden
10//! inside the engine would split that into two command buffers with a pipeline
11//! flush between them. Two obligations follow, and both are contract rather
12//! than preference:
13//!
14//! - every pass the engine begins is ended before `encode` returns, so the
15//!   caller's next `begin_render_pass` on the same encoder is legal; and
16//! - the engine issues no `queue.submit` for scene work. Its per-frame
17//!   uploads — coverage, encoded paints, instances, the viewport uniform and
18//!   each page round's — are written into a mapped staging slot and moved into
19//!   place by copy commands recorded into the caller's encoder, ahead of the
20//!   passes that read them (see [`frust_gpu::staging`]). The slot's re-mapping
21//!   rides the same encoder (`map_buffer_on_submit`), so the caller's one
22//!   submit is also what returns the slot to the engine; an encoder dropped
23//!   unsubmitted just costs the engine that slot. Its on-change uploads
24//!   (colour ramps, atlas regions) — and every upload under
25//!   `FRUST_ENGINE_NO_STAGING_RING`, or on a frame the ring has no free slot
26//!   for — are `queue.write_texture`/`write_buffer` calls, which are ordered
27//!   ahead of the command buffers submitted after them and so land before the
28//!   passes that read them.
29//!
30//! One thing the engine *does* submit, and it is worth naming precisely because
31//! the rule above is otherwise absolute: growing the image atlas array submits a
32//! command buffer of its own, holding one texture copy and nothing else, on the
33//! rare frame that grows it. That is **maintenance, not scene work** — it never
34//! touches the caller's encoder, records no pass and no draw, and exists because
35//! the queued writes it has to precede would otherwise be flushed ahead of it
36//! (see [`crate::gpu::atlas`]'s *Why growth submits a command buffer of its
37//! own*). The caller's encoder is still never submitted by the engine, and the
38//! frame's own passes still reach the queue only when the caller submits it.
39//!
40//! # Sharing the depth attachment
41//!
42//! A caller recording its own depth-writing passes into that encoder hands the
43//! same attachment in as `EngineTarget::depth`, and the two renderers then
44//! occlude each other correctly in either order. Three rules make that work,
45//! and [`crate::gpu::depth`] is where they are stated in full: the shared
46//! comparison and which end of the range is near
47//! ([`DEPTH_COMPARE`](crate::gpu::depth::DEPTH_COMPARE) over a buffer whose far
48//! plane is [`DEPTH_CLEAR`](crate::gpu::depth::DEPTH_CLEAR)); the depth
49//! attachment's extent matching the colour target's; and who owns the clear —
50//! whichever pass runs first in the encoder, which the caller states through
51//! [`EngineRenderer::set_depth_pre_cleared`].
52//!
53//! What that contract does *not* extend to is colour. The frame's opening
54//! below clears the frame's colour target unconditionally, so content painted
55//! into that target before `encode` keeps its depth and loses its pixels: a
56//! host compositing over its own content records that content after the frame,
57//! or into a target of its own the frame composites.
58//!
59//! # The frame's passes
60//!
61//! A frame records into that encoder, in this order.
62//!
63//! 1. **The opening** ([`FramePass::Opening`]), the first pass over the
64//!    surface, which does three things in one render pass:
65//!    - **clears** the colour target to the frame's base colour, and the depth
66//!      attachment to the far plane unless the caller stated it is already
67//!      populated (see [`crate::gpu::depth`]) — the pass's own load
68//!      operations, so a frame with no draws at all still resolves to a clean
69//!      surface;
70//!    - draws the **opaque strips**, depth-tested and depth-writing,
71//!      unblended: only the fully-covered interior spans of opaque draws
72//!      *targeting the surface* — an anti-aliased edge is by definition not
73//!      opaque, and a page carries no depth attachment for the split to be
74//!      sound against. The depth they establish is what lets every blended
75//!      strip after them reject fragments an opaque draw in front of it
76//!      already covered;
77//!    - then draws the **first surface round's own strips** (pass 2 below),
78//!      in order, the pipeline switching between the two inside the pass.
79//!
80//!    It ends exactly where that round's own pass would have ended — at a
81//!    punch cut, at a page or filter round, or at the end of the frame. Page
82//!    and filter rounds scheduled ahead of the first surface round run ahead
83//!    of the opening: they write pooled pages and read neither the surface nor
84//!    its depth, so the opening waits past them. A punch issued before any
85//!    surface round draws takes the opening as a pass of its own, ahead of the
86//!    erase, and so does the end of a frame whose surface rounds drew nothing
87//!    blended.
88//!
89//!    The opaque strips are drawn once, not once per surface round, and that
90//!    is a correctness requirement: the surface can take several rounds (a
91//!    [cut](crate::schedule::cut_at) round is how a wide sibling fan is
92//!    served), and re-drawing them ahead of each would re-draw opaque coverage
93//!    at equal stored depth over composites the round before it had already
94//!    blended.
95//!
96//!    One pass rather than three is what a tile-based GPU is paid for: every
97//!    pass over the surface is a whole-framebuffer load into tile memory and a
98//!    store back out, and the clear and the opaque strips were two such passes
99//!    that drew little or nothing of their own. The pixels are identical —
100//!    the same draws in the same order against the same attachments, only the
101//!    pass boundaries gone. `FRUST_ENGINE_NO_PASS_MERGE`
102//!    ([`EngineRenderer::set_pass_merge`]) records them apart again: a
103//!    [`FramePass::Clear`] pass drawing nothing and a [`FramePass::Opaque`]
104//!    pass loading it, ahead of every round, and then every surface round in
105//!    a pass of its own.
106//! 2. **One pass per [round](crate::schedule)**, the first surface round that
107//!    draws excepted — it is drawn inside the opening. A page round
108//!    ([`FramePass::Page`]) renders one isolated layer into a pooled
109//!    intermediate [page](crate::schedule::pages) it clears to transparent, at
110//!    the page's own origin — so every instance is shifted by the page's
111//!    tile-aligned bounds, and clipped to them, which is what makes a
112//!    [banded](crate::schedule::pages::page_bands) layer's column pages tile
113//!    their layer instead of each holding a clamped copy of it (see
114//!    [`PageWindow`]) — and through the page's own viewport uniform; a round
115//!    continuing a page an earlier round of the same layer opened loads it
116//!    instead. A surface round ([`FramePass::Surface`]) draws **alpha
117//!    strips**, premultiplied-blended, in painter order, loading the surface:
118//!    depth-tested but not depth-writing when a depth attachment is in play, a
119//!    plain painter's-algorithm pass when it is not. A surface round with
120//!    nothing blended to draw records no pass. A round's ops run in the order
121//!    [`Schedule::build`] listed them, so a finished child page composites into
122//!    its parent exactly where the recording entered it.
123//!
124//!    A **filter round** is the one round that draws no strip at all: it runs
125//!    one pass of a [filter](crate::filters)'s sequence, one instanced quad
126//!    through [`EnginePipeline::Filter`], reading the layer's other pooled page
127//!    through the engine's only sampler and clearing the page it writes (see
128//!    [`FilterResources`]). It is an ordinary round of this walk in every other
129//!    respect — recorded into the caller's own encoder, in the order the
130//!    scheduler listed it, with its pages handed back the moment its pass ends.
131//!    It is *not* an own-encoder exception; the atlas replay remains the only
132//!    one of those.
133//!
134//!    A **region-copy round** ([`FramePass::Copy`]) is a backdrop blur's
135//!    first: it reads a region of the frame's own target back into a pooled
136//!    page ([`RegionCopyMode`]) for the blur's filter rounds to run over. It is
137//!    the one page round that *follows* the opening rather than waiting past
138//!    it — it reads the surface, so the clear, the opaque strips and every
139//!    surface round ahead of it in painter order must have landed (the
140//!    scheduler cuts the surface round there; a punch recorded ahead of it is
141//!    erased first) — and, for the same reason, no draw at or past the first
142//!    backdrop's depth is routed into the opaque strips. The backdrop's own
143//!    rectangle is then drawn in the next surface round as strip instances
144//!    reading the blurred page through the composite's layer input, so a clip
145//!    open around it masks it like any fill.
146//! 3. **The hole punch** ([`FramePass::Punch`]), destination-out, when the
147//!    frame recorded a `ClearRect` — see [`crate::compile::clear`] for the
148//!    whole contract this pass implements. It is issued at the punch's own painter-order position,
149//!    not at the end of the frame: a surface round is *cut* where the punch
150//!    was recorded, the punch pass goes into that cut, and the round's
151//!    remaining ops resume in a pass of their own after it. Everything drawn
152//!    over the slot is therefore recorded after the erase and survives it,
153//!    whether or not it wrote depth. A punch past every op of the frame — the
154//!    ordinary case, a `ClearRect` recorded last — cuts nothing and lands after
155//!    the last round exactly as it always did.
156//!
157//! With depth unavailable — no attachment, or `FRUST_ENGINE_NO_DEPTH` set —
158//! the opening draws no opaque strips (and, unmerged, the opaque pass
159//! disappears) and every instance travels through the surface rounds, blended
160//! in painter order. That is a correctness requirement rather than a
161//! fallback detail: routing the opaque spans into a separate, earlier pass is
162//! only sound because the depth buffer re-establishes their ordering against the
163//! blended ones.
164//!
165//! # Compositing a layer
166//!
167//! A finished page reaches its parent as ONE instanced quad through the strip
168//! program every draw goes through (its [`PaintClass::LayerComposite`]
169//! specialisation — see *Paint-class runs*), flagged as a whole rectangle and
170//! naming the layer colour source: the fragment stage then reads the page
171//! bound as `layer_input_texture` at the quad's own texel and scales it by the
172//! opacity packed into the instance's low byte. The page is bound through a
173//! bind group of its own rather than the frame's shared one, because group 0
174//! carries both the pass's viewport uniform and that layer input, and a page
175//! round's viewport is its own.
176//!
177//! A retained page ([`crate::cache::retained`]) is composited by exactly the
178//! same quad, bound the same way; only the texture differs — the store's own,
179//! named by the retained bracket rather than by a page group. A round
180//! rendering a body into one is a page round in every respect above (cleared on
181//! its first round, loaded on a continued one, through its own viewport
182//! uniform, never taking the surface opening) and is listed as a
183//! [`FramePass::Page`]. A `DrawRetained`'s composite carries the painter-order
184//! index the command itself was recorded at.
185//!
186//! A composite carries the deepest painter's-order index of everything inside
187//! the layer it composites, nested layers included. That is what keeps a
188//! translucent layer correctly ordered against the root round's own draws
189//! without giving the scheduler a depth model: every draw recorded before the
190//! layer sits behind that index and every draw recorded after it sits in front.
191//!
192//! # Paint-class runs
193//!
194//! Every strip pass is drawn in **runs**, each through the pipeline of one
195//! [`PaintClass`] — the strip program specialised to the paints the run holds
196//! (see [`crate::gpu::pipelines`]'s *The paint-class axis*). The runs are cut
197//! when a segment is planned (`Scratch::push_run`, by
198//! [`merge_class_runs`]): contiguous, in painter order, covering the segment
199//! exactly, so a pass draws the same instances in the same order it always did
200//! and only the program shading each run changes. A run shorter than the
201//! renderer's threshold ([`EngineRenderer::run_split_min`],
202//! [`crate::gpu::strips::RUN_SPLIT_MIN`] by default) is merged and drawn with
203//! [`PaintClass::Uber`] rather than paying a pipeline switch of its own; a
204//! composite always takes [`PaintClass::LayerComposite`], the opaque pass its
205//! own runs, and a punch [`PaintClass::SolidCoverage`]. A pass sets a pipeline
206//! only where the class changes between runs.
207//!
208//! `FRUST_ENGINE_UBER_PROGRAM` ([`EngineRenderer::set_uber_program`]) turns all
209//! of it off: no segment is split, every pass draws through its state's one
210//! Uber pipeline, and no class program is warmed or built.
211//!
212//! `FRUST_ENGINE_IMAGE_SAMPLER` ([`EngineRenderer::with_image_sampler`]) picks
213//! the program *family* the classes are compiled from, not a class: in the
214//! sampler family — the default — every program that reads atlas images (the
215//! image classes and Uber) filters a bilinear image through one hardware
216//! sampler, which every strip bind group then carries beside the atlas array;
217//! its kill switch (`0`) compiles the manual four-tap family instead. The
218//! runs, and the pipelines a frame resolves, are the same in both families.
219//!
220//! # Paint resolution
221//!
222//! A solid colour travels inside the strip instance itself. Anything else the
223//! compiler encoded — a gradient, an image, a blurred rounded rectangle — is
224//! resolved once per frame before a single instance is built, in three steps
225//! that have to happen in this order:
226//!
227//! 1. residency is settled for the whole frame: the frame's LUT requests are
228//!    serviced through the [`GradientCache`] so every gradient's colour ramp
229//!    has an offset into the packed LUT buffer, and every image paint's atlas
230//!    rectangle is looked up (or learned, the first time it is drawn) in the
231//!    renderer's own image registry — see [`FrameResources::resolve_paints`];
232//! 2. each encoded paint is lowered into the [`GpuEncodedPaint`] record the
233//!    fragment shader samples, carrying that residency; and
234//! 3. the records are serialized back to back, which fixes the texel each one
235//!    starts at — the index a strip instance names its paint by.
236//!
237//! Ramp offsets are only valid within the frame that took them: the cache
238//! compacts and rewrites them in [`EngineRenderer::end_frame`], which is why
239//! residency is decided here rather than at compile time. An image's atlas
240//! rectangle, by contrast, is stable for as long as the image stays resident
241//! (the compiler's [`crate::cache::images::ImageResidency`] does not move a
242//! live image), so the renderer's own registry only ever forgets an entry
243//! when the frame that compiled it reports the entry's region evicted.
244//!
245//! A paint that still cannot be resolved — an image the atlas has no room
246//! for, a gradient whose ramp could not be baked, an external texture (nothing
247//! binds one yet) — leaves its draw skipped rather than stamped in a wrong
248//! colour: the same "a frame draws less, never wrong" rule the compiler
249//! follows for the commands it does not lower.
250//!
251//! An image the atlas holds a *minified* copy of is the one paint whose lowered
252//! record needs a correction here. The compiler composed its natural-to-device
253//! transform against the source's declared extent, before residency was
254//! consulted and so before the fit was known; the shader samples the resident
255//! rectangle. [`ResidentImage::minify_scale`] is the ratio between the two, and
256//! folding it into the lowered record's transform is what keeps a downsampled
257//! image landing on the destination rectangle the display list asked for.
258
259use core::ops::Range;
260use std::collections::{HashMap, HashSet};
261use std::sync::{Arc, Once};
262use std::time::{Duration, Instant};
263
264use frust_gpu::staging::{
265    BUFFER_COPY_ALIGNMENT, StagingFrame, StagingRing, TEXTURE_COPY_ALIGNMENT,
266};
267use frust_gpu::{PipelineCache, PooledTexture, SceneTextureId, ShaderId, ShaderLibrary, TierCaps};
268use frust_scene::{Scene, SceneBuilder};
269use glifo::{AtlasCommand, AtlasCommandRecorder, AtlasPaint};
270use kurbo::Affine;
271use peniko::{Brush, Color};
272use vello_common::encode::{EncodedImage, EncodedPaint};
273use vello_common::fearless_simd::Level;
274use vello_common::paint::{ImageId, ImageSource, Paint};
275use vello_common::strip::Strip;
276
277use crate::cache::images::{ATLAS_PADDING, AtlasRegion};
278use crate::cache::retained::{
279    RetainedBudget, RetainedDecision, RetainedEntry, RetainedPages, RetainedPlan,
280};
281use crate::cache::{
282    AtlasBudget, BYTES_PER_TEXEL, CachedRamp, GradientCache, GradientTextureLayout, ResidentImage,
283};
284#[cfg(any(test, feature = "perf-trace"))]
285use crate::compile::CompileSpans;
286use crate::compile::paint::resolve_lut_request;
287use crate::compile::{
288    ClearPunch, CompiledFrame, GlyphQuad, PhaseClock, RetainedKind, SceneCompiler,
289};
290use crate::config;
291use crate::diag::{EngineSpan, FrameTimestamps};
292use crate::error::EngineError;
293use crate::filters::blur::{FilterInstanceData, GpuFilterData, GpuGaussianBlur};
294use crate::filters::drop_shadow::GpuDropShadow;
295use crate::filters::{Backdrop, FilterStep, ServedFilter, served_filter};
296use crate::gpu::atlas::{
297    AtlasPageBuffers, AtlasRenderReport, AtlasRenderer, lower_encoded_image, push_solid_strips,
298};
299use crate::gpu::bindings::{
300    ATLAS_SAMPLER_BINDING, ExternalRuns, ExternalTextures, GroupKind, GroupSource,
301    SourceGroupCache, SourceGroupKey, lower_encoded_external,
302};
303use crate::gpu::config::resource_widths_agree;
304use crate::gpu::depth::DepthAttachment;
305use crate::gpu::paint_texture::lower_encoded_paint;
306use crate::gpu::pipelines::{
307    EnginePipeline, EngineShaders, INTERMEDIATE_FORMAT, atlas_strip_desc, warm_up_plan,
308};
309use crate::gpu::strips::{
310    ClassRun, ClassSet, PaintClass, PaintType, merge_class_runs, pack_paint_descriptor,
311};
312use crate::gpu::targets::{
313    IntermediateTargets, IntermediateTexture, filter_data_texture_descriptor,
314    filter_data_texture_height, filter_sampler,
315};
316use crate::gpu::{self, AtlasArray, GpuConfig, GpuEncodedPaint, GpuStrip, StripDraw};
317use crate::schedule::pages::{PageConfig, PageSize};
318use crate::schedule::{
319    Composite, MAX_LIVE_PAGES, PageParity, PageTarget, RetainedLayers, RetainedRole,
320    RetainedTarget, Round, RoundOp, Schedule,
321};
322use crate::{EngineTarget, OutputAlpha};
323use vello_common::geometry::RectU16;
324use vello_common::geometry::SizeU16;
325use vello_common::record::RecordedLayerKind;
326
327/// The packed paint descriptor of an inline premultiplied solid colour.
328///
329/// A solid paint indexes no encoded-paint record, so its descriptor carries
330/// only the colour source and paint type — both of which are zero, which is
331/// why this is named rather than written as a bare `0` at its call site: the
332/// zero is a coincidence of the layout, not an absence of information.
333const SOLID_PAINT: u32 = pack_paint_descriptor(PaintType::Solid, 0);
334
335/// The smallest resource-texture dimension the engine can address.
336///
337/// A resource texture's row stride is its width times its texel size, and
338/// `wgpu::COPY_BYTES_PER_ROW_ALIGNMENT` is 256; the narrowest of the three
339/// resources is the gradient LUT at 4 bytes per texel, so 64 texels is the
340/// point below which an upload's `bytes_per_row` stops being legal.
341const MIN_RESOURCE_TEXTURE_DIM: u32 = 64;
342
343/// The smallest strip instance buffer the engine allocates, in instances.
344///
345/// A first frame with a handful of strips should not force a second allocation
346/// on the second frame.
347const MIN_INSTANCE_CAPACITY: u64 = 4096;
348
349/// The colour source a composite instance names its finished page by:
350/// `COLOR_SOURCE_LAYER` in bits 29-30 of the packed paint descriptor.
351///
352/// Deliberately not built through
353/// [`pack_paint_descriptor`](crate::gpu::strips::pack_paint_descriptor): that
354/// helper packs a paint type and a paint-record index into the low bits, and a
355/// composite spends the same bits on a constant opacity instead. Two readings
356/// of one word, so each is written where its own reading is obvious.
357const LAYER_PAINT_SOURCE: u32 = 1 << 29;
358
359/// The premultiplied source colour a hole-punch instance erases with.
360///
361/// Opaque white. Destination-out weights the erase by the source's own *alpha*
362/// and multiplies its colour by zero, so the colour channels never reach the
363/// target and only full alpha matters — it is what makes a fully covered pixel
364/// read exactly `(0, 0, 0, 0)`.
365const PUNCH_SOURCE: u32 = u32::MAX;
366
367/// The label every intermediate page is acquired from the pool under.
368const PAGE_LABEL: &str = "frust-engine layer page";
369
370/// The label every filter round's pass is recorded under.
371const FILTER_LABEL: &str = "frust-engine filter pass";
372
373/// Vertices one filter pass's quad is built from, the same four-vertex
374/// triangle strip every engine program expands an instance into.
375const FILTER_QUAD_VERTICES: u32 = 4;
376
377/// The smallest filter instance buffer the engine allocates, in instances.
378///
379/// A σ-32 blur is ten passes, so this is roughly "one deep blur costs no second
380/// allocation"; the buffer is 32 bytes an instance and grows from here.
381const MIN_FILTER_INSTANCE_CAPACITY: u64 = 16;
382
383static INDEXED_PAINT_WARNING: Once = Once::new();
384
385/// Raised the first time a frame's backdrop blurs are drawn as nothing because
386/// its target can be neither copied nor sampled (see [`RegionCopyMode`]), on
387/// the same once-per-process terms as the compiler's skip warnings.
388static BACKDROP_UNREADABLE_WARNING: Once = Once::new();
389
390/// Vertices the region-copy program's one quad draws (`copy.wgsl`'s
391/// `quad_corner`, a triangle strip).
392const COPY_QUAD_VERTICES: u32 = 4;
393
394/// The bytes of one region-copy instance: four `u16` pairs
395/// ([`crate::gpu::pipelines::copy_vertex_layout`]).
396const COPY_INSTANCE_BYTES: u64 = 16;
397
398/// Raised the first time an atlas region is declined, so a renderer whose
399/// budget and array have gone out of agreement says so once rather than every
400/// frame.
401static ATLAS_REFUSAL_WARNING: Once = Once::new();
402
403/// The CPU phases one [`EngineRenderer::encode_traced`] call splits into.
404///
405/// The GPU half of a frame is [`EngineSpan`]'s, measured by the timestamp ring
406/// at pass boundaries; this is the CPU half, measured by lapping a
407/// [`PhaseClock`] between the encode's own steps. The two answer different
408/// questions and neither substitutes for the other — a text-heavy scene can
409/// cost milliseconds here while its passes cost a fraction of one there.
410///
411/// The phases partition the call in the order they run, so [`Self::total`] is
412/// the whole encode. [`counts`](Self::counts) trails them and is not one of
413/// them.
414///
415/// - [`compile`](Self::compile) — [`SceneCompiler::compile`] end to end,
416///   itself split six ways by [`CompileSpans`].
417/// - [`schedule`](Self::schedule) — building the frame's pass plan and
418///   settling its depth attachment.
419/// - [`paints`](Self::paints) — resolving every draw's paint to a texel of the
420///   encoded-paint texture.
421/// - [`instances`](Self::instances) — building the per-round GPU instance
422///   arrays from the recording.
423/// - [`resize`](Self::resize) — the fallible capacity checks, the image
424///   atlas's own growth, the glyph-rectangle clears an earlier frame's
425///   eviction owes, and the three resource-texture resizes.
426/// - [`upload`](Self::upload) — the frame's queue writes: alpha coverage,
427///   encoded paints, gradient ramps, image texels and the instance arrays.
428/// - [`replay`](Self::replay) — the render-to-atlas pass that draws newly
429///   cached glyphs into their array layers (zero on a steady frame, which
430///   caches none).
431/// - [`pipelines`](Self::pipelines) — taking (or building) this frame's
432///   pipelines and bind groups, and preparing the filter blocks.
433/// - [`record`](Self::record) — recording the frame's passes into the caller's
434///   encoder.
435///
436/// Compiled only under `perf-trace`, with everything that reads it: the whole
437/// `frust-perf enc` route is a `#[cfg]` island rather than a runtime branch the
438/// optimizer is trusted to fold, so a release-lean binary carries none of its
439/// field names, format strings or window (see [`EncodeTrace`]).
440#[cfg(feature = "perf-trace")]
441#[derive(Debug, Clone, Copy, Default)]
442struct EncodeSpans {
443    /// Scene compilation, split further by its own six phases.
444    compile: CompileSpans,
445    /// Whole-call cost of [`SceneCompiler::compile`], call overhead included.
446    ///
447    /// Deliberately measured from outside rather than summed from
448    /// [`Self::compile`]: the difference between the two is the part of the
449    /// call the six inner phases do not cover, and a breakdown that could only
450    /// report its own sum could never show that gap.
451    compile_total: Duration,
452    /// Pass planning and depth settlement.
453    schedule: Duration,
454    /// Paint resolution.
455    paints: Duration,
456    /// Instance-array building.
457    instances: Duration,
458    /// Capacity checks, atlas growth, glyph-rectangle clears, resource resizes.
459    resize: Duration,
460    /// The frame's queue writes.
461    upload: Duration,
462    /// The render-to-atlas pass for newly cached glyphs.
463    replay: Duration,
464    /// Pipeline/bind-group acquisition and filter preparation.
465    pipelines: Duration,
466    /// Recording the frame's passes into the caller's encoder.
467    record: Duration,
468    /// This frame's own draw, strip, alpha and glyph counts.
469    ///
470    /// Last because it is not a phase: it is what the phases above were spent
471    /// on. Carried here because a phase's cost is only readable against the
472    /// work it did — "the walk cost two milliseconds" says nothing on its own,
473    /// while "two milliseconds over three thousand strips and three hundred
474    /// glyphs" says where a lever would have to bite.
475    counts: EncodeCounts,
476}
477
478/// The columns one `frust-perf enc` line reports, in row order.
479///
480/// The first seven are [`CompileSpans`]' own — six phases plus the `glyphs`
481/// subset of the walk — and `compile` after them is the whole compile they sit
482/// inside (so the six sum to at most it, never past it). The next eight are
483/// [`EncodeSpans`]' remaining phases, and `total` closes the line. Pinned as
484/// one list because the line's field order is what a capture is graded by —
485/// the same contract the `frust-perf img` line keeps.
486#[cfg(feature = "perf-trace")]
487const ENCODE_TRACE_COLUMNS: [&str; 17] = [
488    "validate",
489    "prepare",
490    "classify",
491    "admit",
492    "walk",
493    "glyphs",
494    "finish",
495    "compile",
496    "schedule",
497    "paints",
498    "instances",
499    "resize",
500    "upload",
501    "replay",
502    "pipelines",
503    "record",
504    "total",
505];
506
507/// The unitless per-frame counts the line reports after its phases, in row
508/// order — what the phases above were spent on.
509///
510/// The field order is a capture contract: append only. The first five are
511/// per-frame medians over the window. The next five are **sums over the
512/// window** (frames' worth of events, not a typical frame):
513///
514/// - `replays` — frames in which the render-to-atlas replay pass ran.
515/// - `dirty_pages` — atlas pages those replays rendered.
516/// - `atlas_refused` — pages, clears and uploads the replays refused.
517/// - `glyph_misses` — glyphs that took the cache-miss path.
518/// - `glyph_evicted` — atlas rectangles the eviction pass freed.
519///
520/// `run_keys` is a per-frame median again: the distinct run identities (font,
521/// size, hinting) the policy classified in a frame. The replay figures read
522/// zero on a frame whose replay did not run.
523///
524/// The last five cover the staging ring and the bind-group cache. `staged_bytes`
525/// is the frame's staging-slot bytes, a median (zero on the queue-write route).
526/// The other four are **sums over the window** of this frame's change in the
527/// ring's counters and in the bind-group count: `staging_busy` and
528/// `staging_too_large` are frames the ring refused, `staging_buffers` is slot
529/// buffers it created, and `bind_groups` is bind groups the frame path created.
530/// A ring that is off reports zero for all three staging counts.
531#[cfg(feature = "perf-trace")]
532const ENCODE_TRACE_COUNTS: [&str; 16] = [
533    "draws",
534    "strips",
535    "alphas",
536    "glyph_draws",
537    "atlas_glyphs",
538    "replays",
539    "dirty_pages",
540    "atlas_refused",
541    "glyph_misses",
542    "glyph_evicted",
543    "run_keys",
544    "staged_bytes",
545    "staging_busy",
546    "staging_too_large",
547    "staging_buffers",
548    "bind_groups",
549];
550
551/// Index into [`ENCODE_TRACE_COUNTS`] of the first count reported as a sum over
552/// the window rather than its median, up to and excluding
553/// [`ENCODE_TRACE_MEDIAN_AGAIN`]: `replays`, `dirty_pages`, `atlas_refused`,
554/// `glyph_misses` and `glyph_evicted` are events, mostly zero on a frame, so a
555/// median would hide exactly the frames the line exists to count.
556#[cfg(feature = "perf-trace")]
557const ENCODE_TRACE_FIRST_SUMMED: usize = 5;
558
559/// Index into [`ENCODE_TRACE_COUNTS`] of the first count after the summed run
560/// (`run_keys`, a per-frame level, reported as its median).
561#[cfg(feature = "perf-trace")]
562const ENCODE_TRACE_MEDIAN_AGAIN: usize = 10;
563
564/// Index into [`ENCODE_TRACE_COUNTS`] of the first count summed over the window
565/// again, through the end of the line: the staging refusals, buffer creations
566/// and bind-group creations. `staged_bytes`, just before it, is a median.
567#[cfg(feature = "perf-trace")]
568const ENCODE_TRACE_SUMMED_AGAIN: usize = 12;
569
570/// Width of one window row: every phase column followed by every count.
571#[cfg(feature = "perf-trace")]
572const ENCODE_TRACE_ROW: usize = ENCODE_TRACE_COLUMNS.len() + ENCODE_TRACE_COUNTS.len();
573
574/// The staging ring's and the bind-group cache's counters, read at one point so
575/// a frame's row can carry only the change since the previous read.
576#[cfg(feature = "perf-trace")]
577#[derive(Debug, Clone, Copy, Default)]
578struct TraceCounters {
579    busy: u64,
580    too_large: u64,
581    buffers_created: u64,
582    bind_groups: u64,
583}
584
585/// `now - before` as one frame's count, saturating at both ends.
586#[cfg(feature = "perf-trace")]
587fn counter_delta(now: u64, before: u64) -> u32 {
588    u32::try_from(now.saturating_sub(before)).unwrap_or(u32::MAX)
589}
590
591/// What one frame drew, carried beside its phases.
592///
593/// Counts rather than durations, and so reported without a unit suffix.
594/// `perf-trace`-only, with [`EncodeSpans`], which is all that carries it.
595#[cfg(feature = "perf-trace")]
596#[derive(Debug, Clone, Copy, Default)]
597struct EncodeCounts {
598    /// Recorded draws.
599    draws: u32,
600    /// Strips generated.
601    strips: u32,
602    /// Bytes of alpha coverage those strips index — the frame's largest
603    /// single queue write.
604    alphas: u32,
605    /// Draws that painted one glyph outline.
606    glyph_draws: u32,
607    /// How many of those sampled the glyph atlas rather than rasterizing.
608    atlas_glyphs: u32,
609    /// 1 when the render-to-atlas replay ran this frame, else 0.
610    replays: u32,
611    /// Atlas pages the replay rendered this frame.
612    dirty_pages: u32,
613    /// Atlas pages, clears and uploads the replay refused this frame.
614    atlas_refused: u32,
615    /// Glyphs that took the cache-miss path while compiling this frame.
616    glyph_misses: u32,
617    /// Atlas rectangles this frame's eviction freed.
618    glyph_evicted: u32,
619    /// Distinct run identities classified this frame.
620    run_keys: u32,
621    /// Bytes the frame placed in its staging slot, alignment padding included;
622    /// zero on the queue-write route.
623    staged_bytes: u32,
624    /// Frames the staging ring refused this frame because every slot was in
625    /// flight. Filled by [`EncodeTrace::record`], which holds the previous
626    /// reading.
627    staging_busy: u32,
628    /// Frames the staging ring refused this frame because they needed more than
629    /// a slot may hold. Filled by [`EncodeTrace::record`].
630    staging_too_large: u32,
631    /// Slot buffers the staging ring created this frame. Filled by
632    /// [`EncodeTrace::record`].
633    staging_buffers: u32,
634    /// Bind groups the frame path created this frame. Filled by
635    /// [`EncodeTrace::record`].
636    bind_groups: u32,
637}
638
639#[cfg(feature = "perf-trace")]
640impl EncodeCounts {
641    /// What `frame` drew, with the atlas figures the frame's replay and policy
642    /// reported and the bytes it staged. The staging-counter and bind-group
643    /// deltas are left at zero for [`EncodeTrace::record`] to fill.
644    fn of(
645        frame: &CompiledFrame,
646        replayed: bool,
647        report: &AtlasRenderReport,
648        policy: &crate::text::atlas_policy::AtlasTrace,
649        staged: u64,
650    ) -> Self {
651        fn count(len: usize) -> u32 {
652            u32::try_from(len).unwrap_or(u32::MAX)
653        }
654        Self {
655            draws: count(frame.draws().len()),
656            strips: count(frame.strip_buf().len()),
657            alphas: count(frame.alphas().len()),
658            glyph_draws: frame.glyph_draws,
659            atlas_glyphs: frame.atlas_glyph_draws,
660            replays: u32::from(replayed),
661            dirty_pages: report.pages,
662            atlas_refused: report.refused,
663            glyph_misses: policy.glyph_misses,
664            glyph_evicted: policy.glyph_evicted,
665            run_keys: policy.run_keys(),
666            staged_bytes: u32::try_from(staged).unwrap_or(u32::MAX),
667            staging_busy: 0,
668            staging_too_large: 0,
669            staging_buffers: 0,
670            bind_groups: 0,
671        }
672    }
673}
674
675/// Frames one `frust-perf enc` line summarises — one line a second at 60 Hz.
676///
677/// A window rather than a line per frame: the encode is the very thing being
678/// measured, so formatting and logging inside it once per frame would charge
679/// the measurement to its own subject. One line per sixty frames keeps that
680/// charge under a microsecond a frame while still resolving a scenario's
681/// phases (a thirty-second run reports thirty times), and it matches the
682/// cadence `frust-shell-common`'s own rate-limited frame summary already
683/// emits at.
684#[cfg(feature = "perf-trace")]
685const ENCODE_TRACE_WINDOW: usize = 60;
686
687#[cfg(feature = "perf-trace")]
688impl EncodeSpans {
689    /// The whole encode.
690    ///
691    /// Saturating throughout: a diagnostic sum must not take the frame with it
692    /// on overflow (E17).
693    fn total(&self) -> Duration {
694        // `compile.glyphs` is deliberately absent: it is a subset of the walk
695        // inside `compile_total`, and adding it would count text twice.
696        [
697            self.schedule,
698            self.paints,
699            self.instances,
700            self.resize,
701            self.upload,
702            self.replay,
703            self.pipelines,
704            self.record,
705        ]
706        .iter()
707        .fold(self.compile_total, |acc, span| acc.saturating_add(*span))
708    }
709
710    /// This frame's row, in [`ENCODE_TRACE_COLUMNS`] order, in nanoseconds.
711    ///
712    /// Nanoseconds in a `u32` rather than a `Duration` per column: a window of
713    /// sixty rows is then three kilobytes of plain integers to sort, and no
714    /// single phase of one frame reaches the four-second ceiling that would
715    /// saturate one.
716    fn row(&self) -> [u32; ENCODE_TRACE_ROW] {
717        fn ns(span: Duration) -> u32 {
718            u32::try_from(span.as_nanos()).unwrap_or(u32::MAX)
719        }
720        [
721            ns(self.compile.validate),
722            ns(self.compile.prepare),
723            ns(self.compile.classify),
724            ns(self.compile.admit),
725            ns(self.compile.walk),
726            ns(self.compile.glyphs),
727            ns(self.compile.finish),
728            ns(self.compile_total),
729            ns(self.schedule),
730            ns(self.paints),
731            ns(self.instances),
732            ns(self.resize),
733            ns(self.upload),
734            ns(self.replay),
735            ns(self.pipelines),
736            ns(self.record),
737            ns(self.total()),
738            self.counts.draws,
739            self.counts.strips,
740            self.counts.alphas,
741            self.counts.glyph_draws,
742            self.counts.atlas_glyphs,
743            self.counts.replays,
744            self.counts.dirty_pages,
745            self.counts.atlas_refused,
746            self.counts.glyph_misses,
747            self.counts.glyph_evicted,
748            self.counts.run_keys,
749            self.counts.staged_bytes,
750            self.counts.staging_busy,
751            self.counts.staging_too_large,
752            self.counts.staging_buffers,
753            self.counts.bind_groups,
754        ]
755    }
756}
757
758/// The rolling window of per-frame [`EncodeSpans`] one `frust-perf enc` line
759/// is computed over.
760///
761/// Absent altogether from a build without `perf-trace`, rather than inert in
762/// one. The type, its window, its column names and the `frust-perf enc` literal
763/// are all inside a `#[cfg]` island — including the renderer's own field — so
764/// "no frame pays a push, a sort or a format" is what the compiler emitted and
765/// not what the optimizer was expected to prove about a constant branch. That
766/// distinction is the one the release-lean gate measures: it greps a shipping
767/// binary for `frust-perf` and expects to find nothing.
768#[cfg(feature = "perf-trace")]
769#[derive(Debug, Default)]
770struct EncodeTrace {
771    /// One row per frame in the window, in [`ENCODE_TRACE_COLUMNS`] order.
772    window: Vec<[u32; ENCODE_TRACE_ROW]>,
773    /// The column being ranked, kept across windows so a steady stream of
774    /// them allocates once.
775    ranked: Vec<u32>,
776    /// Frames recorded since this renderer was built — the line's own `n`,
777    /// which is the engine's count and deliberately not the host's raw-line
778    /// `n` (a different emitter counting different frames).
779    frames: u64,
780    /// The counters as the previous frame read them, so each row reports only
781    /// its own frame's change.
782    previous: TraceCounters,
783}
784
785#[cfg(feature = "perf-trace")]
786impl EncodeTrace {
787    /// Adds one frame's phases to the window, emitting the window's line when
788    /// it fills.
789    ///
790    /// Called at the very end of the encode, so the formatting a full window
791    /// costs falls outside every phase the line reports — the numbers describe
792    /// the encode, not the encode plus its own accounting.
793    ///
794    /// `now` is the counters read at the same point in the encode; the row
795    /// carries the change since the previous frame's reading.
796    fn record(&mut self, spans: &EncodeSpans, now: TraceCounters) {
797        // Saturating like every other counter on this path: a renderer that
798        // outlived `u64::MAX` frames would report a wrapped `n`, and a wrong
799        // number in a capture is worse than a stuck one.
800        self.frames = self.frames.saturating_add(1);
801        let mut spans = *spans;
802        spans.counts.staging_busy = counter_delta(now.busy, self.previous.busy);
803        spans.counts.staging_too_large = counter_delta(now.too_large, self.previous.too_large);
804        spans.counts.staging_buffers =
805            counter_delta(now.buffers_created, self.previous.buffers_created);
806        spans.counts.bind_groups = counter_delta(now.bind_groups, self.previous.bind_groups);
807        self.previous = now;
808        self.window.push(spans.row());
809        if self.window.len() < ENCODE_TRACE_WINDOW {
810            return;
811        }
812        let line = self.line();
813        self.window.clear();
814        log::info!("{line}");
815    }
816
817    /// The `frust-perf enc` line the current window reports.
818    ///
819    /// The counts `replays` through `glyph_evicted` and `staging_busy` through
820    /// `bind_groups` are window sums; every other column (including `run_keys`
821    /// and `staged_bytes`) is its own median over the window.
822    /// The trailing field is the whole encode's 95th percentile, so a column
823    /// answers "what does this phase usually cost" while the tail answers "how
824    /// bad is a bad frame". Medians are taken per column and so do not sum to
825    /// the `total_us`
826    /// median exactly — they are close on a steady scene, and the gap is
827    /// itself the signal that the phases are not moving together.
828    fn line(&mut self) -> String {
829        // Sized for the longest line the loops below write: a
830        // `<name>_us=<7 digits>.<1>` field per phase, a `<name>=<10 digits>`
831        // field per count, the two `n`/`w` fields and the p95 tail, with room
832        // to spare rather than a byte-exact fit.
833        let mut line = String::with_capacity(896);
834        line.push_str("frust-perf enc n=");
835        line.push_str(&self.frames.to_string());
836        line.push_str(" w=");
837        line.push_str(&self.window.len().to_string());
838        for (column, name) in ENCODE_TRACE_COLUMNS.iter().enumerate() {
839            let p50 = self.percentile(column, 50);
840            line.push(' ');
841            line.push_str(name);
842            line.push_str("_us=");
843            line.push_str(&format_us(p50));
844        }
845        let tail = self.percentile(ENCODE_TRACE_COLUMNS.len() - 1, 95);
846        line.push_str(" total_p95_us=");
847        line.push_str(&format_us(tail));
848        for (offset, name) in ENCODE_TRACE_COUNTS.iter().enumerate() {
849            let column = ENCODE_TRACE_COLUMNS.len() + offset;
850            let summed = (ENCODE_TRACE_FIRST_SUMMED..ENCODE_TRACE_MEDIAN_AGAIN).contains(&offset)
851                || offset >= ENCODE_TRACE_SUMMED_AGAIN;
852            let value = if summed {
853                self.window
854                    .iter()
855                    .filter_map(|row| row.get(column))
856                    .fold(0_u64, |acc, v| acc.saturating_add(u64::from(*v)))
857            } else {
858                u64::from(self.percentile(column, 50))
859            };
860            line.push(' ');
861            line.push_str(name);
862            line.push('=');
863            line.push_str(&value.to_string());
864        }
865        line
866    }
867
868    /// The `pct`-th percentile of `column` over the window, in nanoseconds.
869    ///
870    /// Nearest-rank on the sorted column, which needs no interpolation and so
871    /// reports a value the window really contains — the same choice
872    /// `frust-shell-common`'s frame summary makes, and the one that keeps a
873    /// sixty-sample window honest.
874    fn percentile(&mut self, column: usize, pct: usize) -> u32 {
875        self.ranked.clear();
876        self.ranked
877            .extend(self.window.iter().filter_map(|row| row.get(column)));
878        self.ranked.sort_unstable();
879        let len = self.ranked.len();
880        if len == 0 {
881            return 0;
882        }
883        let rank = (len * pct).div_ceil(100).clamp(1, len);
884        self.ranked.get(rank - 1).copied().unwrap_or(0)
885    }
886}
887
888/// Nanoseconds as microseconds with one decimal — the unit every
889/// `frust-perf enc` field is reported in.
890///
891/// Fixed-point rather than a float format: a phase can be tens of nanoseconds
892/// (an empty punch pass) or milliseconds (a ten-thousand-row table's walk), and
893/// one decimal microsecond reads the same either way without a float's
894/// locale-dependent formatting reaching a capture the harness greps.
895#[cfg(feature = "perf-trace")]
896fn format_us(nanos: u32) -> String {
897    let tenths = u64::from(nanos).div_ceil(100);
898    format!("{}.{}", tenths / 10, tenths % 10)
899}
900
901/// Where one [`EngineRenderer`] construction spent its wall time, measured with
902/// `Instant` reads inside [`EngineRenderer::with_image_sampler`]. Evidence for
903/// the startup sub-step stamps only; no behaviour depends on it.
904#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
905pub struct ConstructionTimings {
906    /// `ShaderLibrary::new` plus `EngineShaders::register_family`: registering
907    /// the shader sources. No module is created here any more — each is created
908    /// on first use, the first frame's on the shader warm-up pool — so this
909    /// reads the registration cost alone.
910    pub shader_modules: Duration,
911    /// `PipelineCache::new`, `warm_up_shaders` (the first-frame module pool's
912    /// spawn) and `warm_up` (enqueue and background-worker spawn).
913    pub warm_up_start: Duration,
914    /// Everything else in construction (routing, gradients, atlas, resources).
915    pub rest: Duration,
916}
917
918/// One surface's 2D render engine: a scene in, recorded passes out.
919///
920/// Create one per surface and keep it across frames — the retained scene
921/// compiler, gradient cache, pipeline cache, resource textures and intermediate
922/// pool are the reason a steady-state frame allocates nothing.
923#[derive(Debug)]
924pub struct EngineRenderer {
925    /// Where [`Self::with_image_sampler`] spent its time (startup evidence only).
926    construction: ConstructionTimings,
927    caps: TierCaps,
928    format: wgpu::TextureFormat,
929    shaders: EngineShaders,
930    pipelines: PipelineCache,
931    compiler: SceneCompiler,
932    gradients: GradientCache,
933    depth: DepthAttachment,
934    targets: IntermediateTargets,
935    /// The bounds an intermediate page is sized between — the policy half of
936    /// page sizing, kept beside the pool the extents are requested from.
937    pages: PageConfig,
938    /// The caller-owned textures a `Command::SceneTexture` resolves against
939    /// (see [`crate::gpu::bindings`]). Its extent half lives on the compiler,
940    /// written by the same two calls that write this.
941    textures: ExternalTextures,
942    resources: FrameResources,
943    scratch: Scratch,
944    /// How each pass's instances are split into paint-class runs — the kill
945    /// switch and the run threshold, settled at construction from
946    /// [`crate::config`] and overridable per renderer.
947    routing: ClassRouting,
948    /// Whether the surface opening is one pass — settled at construction from
949    /// `FRUST_ENGINE_NO_PASS_MERGE` and overridable per renderer (see
950    /// [`Self::set_pass_merge`]).
951    pass_merge: bool,
952    /// The passes the last encoded frame recorded into the caller's encoder,
953    /// in order; refilled every frame, so its allocation is reused.
954    frame_passes: Vec<FramePass>,
955    /// The strip pipelines resolved so far, by state and class, so a frame
956    /// takes each one it draws with by a two-enum lookup rather than by
957    /// rebuilding and re-keying its description through the cache.
958    resolved: ResolvedPipelines,
959    /// The render-to-atlas pass, created by the first frame that caches a
960    /// glyph.
961    ///
962    /// Lazy because it owns a coverage texture, an instance buffer and five
963    /// stand-in bindings of its own, and a renderer that never draws text —
964    /// or one running with `FRUST_ENGINE_NO_ATLAS` — should pay for none of
965    /// them.
966    atlas_glyphs: Option<AtlasRenderer>,
967    /// The compiler the atlas replay lowers a page's recorded commands
968    /// through, sized to the atlas page rather than to the surface.
969    ///
970    /// A second compiler rather than this renderer's own: the frame's compiler
971    /// is mid-frame (its glyph entry map is exactly what the replay is
972    /// draining) and its viewport is the surface's, while a page's commands are
973    /// in page space. Created on the first replay and kept, so a steady stream
974    /// of first-seen glyphs allocates a strip generator once.
975    atlas_lowering: Option<SceneCompiler>,
976    /// What the last frame's replay serviced, for
977    /// [`Self::atlas_render_report`].
978    atlas_report: AtlasRenderReport,
979    /// The filter-data texture, sampler and instance buffer a filter round is
980    /// executed with.
981    ///
982    /// Lazy for the same reason [`Self::atlas_glyphs`] is: a renderer that
983    /// never blurs should own neither a sampler nor a resource texture it will
984    /// not read. Created by the first frame that schedules a filter round.
985    filters: Option<FilterResources>,
986    /// The instance buffer a frame's fragment region copies are drawn from
987    /// ([`RegionCopyMode::Fragment`]). Lazy for [`Self::filters`]' reason: a
988    /// renderer whose frames never copy a region owns none.
989    region_copies: Option<RegionCopyResources>,
990    /// The composite and backdrop groups of the round being recorded, kept
991    /// here so the buffer is reused; emptied at the end of every round.
992    composite_groups: Vec<wgpu::BindGroup>,
993    /// Bind groups the region copy passes built outside the cache.
994    region_copy_groups: u64,
995    /// The retained pages `PushRetained` bodies render into and
996    /// `DrawRetained`s composite from, kept across frames (see
997    /// [`crate::cache::retained`]).
998    retained: RetainedPages,
999    /// The roles the store gave this frame's retained layers, handed to the
1000    /// scheduler — retained across frames for its allocation alone.
1001    retained_layers: RetainedLayers,
1002    /// The scene [`Self::record_range`] projects a range into, kept for its
1003    /// allocation alone; empty until the first range is recorded.
1004    range_scene: Scene,
1005    /// The mapped staging slots a frame's per-frame uploads are written into
1006    /// and copied out of inside the caller's encoder (see
1007    /// [`frust_gpu::staging`]); `None` under `FRUST_ENGINE_NO_STAGING_RING`,
1008    /// which sends every one of them through queue writes instead.
1009    staging: Option<StagingRing>,
1010    /// Bytes the last encoded frame placed in its staging slot — zero when it
1011    /// took the queue-write route.
1012    last_staged: u64,
1013    /// The rolling CPU-phase window `frust-perf enc` lines are emitted from.
1014    ///
1015    /// Absent entirely from a build without `perf-trace`, along with the type
1016    /// itself — see [`EncodeTrace`].
1017    #[cfg(feature = "perf-trace")]
1018    encode_trace: EncodeTrace,
1019}
1020
1021/// Every pipeline a renderer warms up, in the order its background worker
1022/// builds them.
1023///
1024/// [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for)'s list, reordered and extended for the frames that
1025/// follow an install: the worker opens with what an ordinary first frame draws
1026/// (the atlas page pass, the depth-tested surface pass's glyph and solid
1027/// programs, the opaque pass's solid one, then the surface pass's Uber
1028/// program) because that frame waits behind whatever the worker is building,
1029/// and it ends with the surface's layer composite, which only a later
1030/// transition needs.
1031///
1032/// A variant listed twice is queued once, so the reordering needs no
1033/// de-duplication. With the Uber program forced the list is the Uber states
1034/// alone, exactly as [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for) gives them.
1035fn warm_up_list(
1036    shaders: &EngineShaders,
1037    format: wgpu::TextureFormat,
1038    uber_only: bool,
1039) -> Vec<frust_gpu::RenderPipelineDesc> {
1040    ordered_warm_up(
1041        shaders.image_sampler,
1042        uber_only,
1043        |state, class| state.class_desc(shaders, format, class),
1044        || atlas_strip_desc(shaders),
1045    )
1046}
1047
1048/// [`warm_up_list`]'s order and membership over any description type, so it
1049/// can be checked without a device (see [`warm_up_plan`]).
1050fn ordered_warm_up<D>(
1051    image_sampler: bool,
1052    uber_only: bool,
1053    describe: impl Fn(EnginePipeline, PaintClass) -> D,
1054    atlas: impl Fn() -> D,
1055) -> Vec<D> {
1056    let full = warm_up_plan(image_sampler, uber_only, &describe, &atlas);
1057    if uber_only {
1058        return full;
1059    }
1060    let mut descs = Vec::with_capacity(full.len() + 6);
1061    descs.push(atlas());
1062    descs.push(describe(EnginePipeline::StripDepthAlpha, PaintClass::Glyph));
1063    descs.push(describe(
1064        EnginePipeline::StripDepthAlpha,
1065        PaintClass::SolidCoverage,
1066    ));
1067    descs.push(describe(
1068        EnginePipeline::StripOpaque,
1069        PaintClass::SolidCoverage,
1070    ));
1071    descs.push(describe(EnginePipeline::StripDepthAlpha, PaintClass::Uber));
1072    descs.extend(full);
1073    descs.push(describe(
1074        EnginePipeline::StripDepthAlpha,
1075        PaintClass::LayerComposite,
1076    ));
1077    descs
1078}
1079
1080/// The shader modules the first frame after an install draws with, in the
1081/// order it first needs them: the distinct programs of [`warm_up_list`]'s
1082/// opening — the manual family's Uber program the atlas page pass draws with,
1083/// the glyph and solid class programs, and the family's own Uber program —
1084/// or, with the Uber program forced, the Uber module(s) alone.
1085///
1086/// [`EngineRenderer::with_image_sampler`] hands them to
1087/// `PipelineCache::warm_up_shaders`, so they are created side by side at
1088/// install instead of one after another on the worker the frame waits behind.
1089fn first_frame_shaders(shaders: &EngineShaders, uber_only: bool) -> Vec<ShaderId> {
1090    first_frame_programs(
1091        uber_only,
1092        |class| shaders.strip_class(class),
1093        || shaders.atlas_strip,
1094    )
1095}
1096
1097/// [`first_frame_shaders`]' order and membership over any program type, so it
1098/// can be checked against [`ordered_warm_up`]'s opening without a device.
1099/// `program(class)` is the family's program for `class`, `atlas()` the atlas
1100/// page pass's.
1101fn first_frame_programs<P: PartialEq>(
1102    uber_only: bool,
1103    program: impl Fn(PaintClass) -> P,
1104    atlas: impl Fn() -> P,
1105) -> Vec<P> {
1106    let classes: &[PaintClass] = if uber_only {
1107        &[PaintClass::Uber]
1108    } else {
1109        &[
1110            PaintClass::Glyph,
1111            PaintClass::SolidCoverage,
1112            PaintClass::Uber,
1113        ]
1114    };
1115    let mut programs = vec![atlas()];
1116    for class in classes {
1117        let next = program(*class);
1118        if !programs.contains(&next) {
1119            programs.push(next);
1120        }
1121    }
1122    programs
1123}
1124
1125impl EngineRenderer {
1126    /// Builds a renderer for `format` targets on `caps`' adapter, warming
1127    /// every engine pipeline in the background.
1128    ///
1129    /// `pipeline_cache` is the host's persisted driver cache when it has one —
1130    /// `None` on every backend but Vulkan.
1131    ///
1132    /// # Errors
1133    ///
1134    /// [`EngineError::AtlasError`] when the adapter's `resource_texture_dim`
1135    /// is not a power of two of at least [`MIN_RESOURCE_TEXTURE_DIM`]. The
1136    /// strip shader reconstructs a resource texture's width as `1 << bits` and
1137    /// an upload's row stride must satisfy wgpu's copy alignment, so neither
1138    /// the shader's addressing nor the uploads would be valid otherwise.
1139    pub fn new(
1140        device: &wgpu::Device,
1141        caps: &TierCaps,
1142        format: wgpu::TextureFormat,
1143        pipeline_cache: Option<&wgpu::PipelineCache>,
1144    ) -> Result<Self, EngineError> {
1145        Self::with_image_sampler(
1146            device,
1147            caps,
1148            format,
1149            pipeline_cache,
1150            config::image_sampler_enabled(),
1151        )
1152    }
1153
1154    /// [`Self::new`] with the strip program family stated rather than read
1155    /// from `FRUST_ENGINE_IMAGE_SAMPLER` — the programmatic counterpart of
1156    /// that switch, which only sets the family [`Self::new`] picks.
1157    ///
1158    /// With `image_sampler` set, every strip program that reads atlas images
1159    /// filters a bilinear one through a hardware sampler instead of four
1160    /// `textureLoad` taps (see [`crate::gpu::shader_src`]'s *The two program
1161    /// families*), and every strip bind group carries the one linear,
1162    /// clamp-to-edge sampler this renderer creates for it. A family is a
1163    /// construction-time choice rather than a per-frame setter because it
1164    /// decides which programs are compiled and warmed: a caller comparing the
1165    /// two builds one renderer of each.
1166    ///
1167    /// # Errors
1168    ///
1169    /// As [`Self::new`].
1170    pub fn with_image_sampler(
1171        device: &wgpu::Device,
1172        caps: &TierCaps,
1173        format: wgpu::TextureFormat,
1174        pipeline_cache: Option<&wgpu::PipelineCache>,
1175        image_sampler: bool,
1176    ) -> Result<Self, EngineError> {
1177        let dim = caps.resource_texture_dim;
1178        if !dim.is_power_of_two() || dim < MIN_RESOURCE_TEXTURE_DIM {
1179            return Err(EngineError::AtlasError);
1180        }
1181
1182        let started = Instant::now();
1183        let routing = ClassRouting::from_config();
1184        let shaders_started = Instant::now();
1185        let mut library = ShaderLibrary::new();
1186        let shaders = EngineShaders::register_family(&mut library, device, image_sampler);
1187        let shader_modules = shaders_started.elapsed();
1188        let warm_started = Instant::now();
1189        let mut pipelines = PipelineCache::new(Arc::new(library), pipeline_cache.cloned());
1190        // The first frame's modules are created side by side on a short-lived
1191        // pool, started ahead of the worker: the worker's first jobs and the
1192        // frame's own requests then find them built or in flight. Every other
1193        // module is created by whichever build first needs it.
1194        pipelines.warm_up_shaders(device, &first_frame_shaders(&shaders, routing.uber));
1195        pipelines.warm_up(device, &warm_up_list(&shaders, format, routing.uber));
1196        let warm_up_start = warm_started.elapsed();
1197
1198        let level = Level::try_detect().unwrap_or(Level::baseline());
1199        let gradients = GradientCache::for_texture(GradientTextureLayout::square(dim), level);
1200
1201        let mut renderer = Self {
1202            construction: ConstructionTimings::default(),
1203            caps: caps.clone(),
1204            format,
1205            shaders,
1206            pipelines,
1207            // The viewport is re-asserted on every compile, so the extent is
1208            // only an initial allocation hint. `caps` is not: it is what fixes
1209            // the image atlas budget for this renderer's whole life (mobile or
1210            // desktop tier, the adapter's own ceilings, and any
1211            // `FRUST_ENGINE_ATLAS_SIZE` override), and the adapter is known
1212            // exactly here — a compiler built without it would silently keep
1213            // the mobile budget on every device.
1214            compiler: {
1215                let mut compiler = SceneCompiler::for_caps(1, 1, caps);
1216                // A renderer has a store behind its compiler, so retained
1217                // brackets are recorded for it — unless the engine-side kill
1218                // switch says otherwise.
1219                compiler.set_retained_layers(!config::layers_disabled());
1220                compiler
1221            },
1222            gradients,
1223            depth: DepthAttachment::new(),
1224            targets: IntermediateTargets::new(caps),
1225            pages: PageConfig::default(),
1226            textures: ExternalTextures::new(),
1227            resources: FrameResources::new(device, dim, image_sampler),
1228            scratch: Scratch::default(),
1229            routing,
1230            pass_merge: !config::pass_merge_disabled(),
1231            frame_passes: Vec::new(),
1232            resolved: ResolvedPipelines::default(),
1233            atlas_glyphs: None,
1234            atlas_lowering: None,
1235            atlas_report: AtlasRenderReport::default(),
1236            filters: None,
1237            region_copies: None,
1238            composite_groups: Vec::new(),
1239            region_copy_groups: 0,
1240            retained: RetainedPages::for_caps(caps),
1241            retained_layers: RetainedLayers::new(),
1242            range_scene: Scene::new(),
1243            staging: (!config::staging_ring_disabled()).then(StagingRing::new),
1244            last_staged: 0,
1245            #[cfg(feature = "perf-trace")]
1246            encode_trace: EncodeTrace::default(),
1247        };
1248        let rest = started
1249            .elapsed()
1250            .saturating_sub(shader_modules)
1251            .saturating_sub(warm_up_start);
1252        renderer.construction = ConstructionTimings {
1253            shader_modules,
1254            warm_up_start,
1255            rest,
1256        };
1257        Ok(renderer)
1258    }
1259
1260    /// Where this renderer's construction spent its time.
1261    #[must_use]
1262    pub fn construction_timings(&self) -> ConstructionTimings {
1263        self.construction
1264    }
1265
1266    /// The target format this renderer warmed its pipelines for.
1267    #[must_use]
1268    pub fn format(&self) -> wgpu::TextureFormat {
1269        self.format
1270    }
1271
1272    /// Whether this renderer's strip programs filter bilinear atlas images
1273    /// through the hardware sampler (see [`Self::with_image_sampler`]).
1274    #[must_use]
1275    pub fn image_sampler(&self) -> bool {
1276        self.shaders.image_sampler
1277    }
1278
1279    /// The adapter capabilities every sizing decision is made against.
1280    #[must_use]
1281    pub fn caps(&self) -> &TierCaps {
1282        &self.caps
1283    }
1284
1285    /// The pool the engine's off-screen intermediates come from.
1286    #[must_use]
1287    pub fn targets(&self) -> &IntermediateTargets {
1288        &self.targets
1289    }
1290
1291    /// The bounds this renderer sizes intermediate layer pages between.
1292    #[must_use]
1293    pub fn page_config(&self) -> PageConfig {
1294        self.pages
1295    }
1296
1297    /// The atlas geometry image residency allocates within.
1298    ///
1299    /// Derived from the adapter in [`Self::new`], so this is the tier's budget
1300    /// narrowed to what the adapter can create — not a constant.
1301    #[must_use]
1302    pub fn atlas_budget(&self) -> AtlasBudget {
1303        self.compiler.images().budget()
1304    }
1305
1306    /// Re-budget image residency, dropping every image currently resident and
1307    /// the atlas array holding them.
1308    ///
1309    /// An atlas rectangle only means anything against the geometry it was
1310    /// allocated in, so a new budget invalidates every one already handed out —
1311    /// which is why the array, the registry of where each image lives and the
1312    /// bind groups naming that array all go in the same step, and each image
1313    /// re-uploads on the next frame that draws it. A start-up or adapter-change
1314    /// operation, never a per-frame one.
1315    pub fn set_atlas_budget(&mut self, budget: AtlasBudget) {
1316        self.compiler.set_atlas_budget(budget);
1317        self.resources.reset_atlas();
1318    }
1319
1320    /// Replace image residency wholesale, on the same invalidation terms as
1321    /// [`Self::set_atlas_budget`].
1322    ///
1323    /// The programmatic counterpart to `FRUST_ENGINE_NO_ATLAS`: an
1324    /// [`ImageResidency::disabled`] residency takes both atlas classes out of
1325    /// the frame — images are skipped and every glyph is drawn as outline
1326    /// strips — without a process-global environment variable, which is what a
1327    /// caller comparing the two paths on one device needs.
1328    pub fn set_image_residency(&mut self, images: crate::cache::images::ImageResidency) {
1329        self.compiler.set_image_residency(images);
1330        self.resources.reset_atlas();
1331    }
1332
1333    /// How many atlas regions this renderer has declined to write or clear.
1334    ///
1335    /// Zero on every sound frame: residency allocates inside the budget the
1336    /// array is created at, and the array is grown to the depth the frame
1337    /// reports before its regions are written, so a refusal means those two
1338    /// went out of agreement. The count exists so that disagreement is
1339    /// measurable rather than silent — a refused write is a region the frame
1340    /// believed it had filled.
1341    #[must_use]
1342    pub fn refused_atlas_regions(&self) -> u64 {
1343        self.resources.refused_regions
1344    }
1345
1346    /// Finishes pipeline warm-up on the calling thread, returning only once
1347    /// every engine pipeline exists.
1348    ///
1349    /// Warm-up is started in the background by [`Self::new`] and normally
1350    /// needs no attention. Two callers want it forced: a host that must not
1351    /// let the *first* frame pay for a compile, and anything about to drop the
1352    /// `wgpu::Device` shortly after building a renderer — the warm-up worker
1353    /// holds its own handle on that device, and tearing it down while the
1354    /// worker is mid-compile is a driver-level hazard rather than a clean
1355    /// cancellation.
1356    ///
1357    /// Each variant is requested through the cache, which builds a queued one
1358    /// inline and waits for one the worker has already started, so nothing is
1359    /// compiled twice and nothing is left for the worker to claim afterwards.
1360    ///
1361    /// "Every" is the warm-up list: the nine Uber states, plus — unless the
1362    /// Uber program is forced — the warmed paint classes and the surface's
1363    /// layer-composite program (see [`warm_up_list`]). A class variant outside
1364    /// that list is created on the first frame that draws a run of it.
1365    ///
1366    /// Idempotent: once the list is built every request is a cache hit, so a
1367    /// second call compiles nothing. It encodes and submits nothing either —
1368    /// the first frame afterwards is still the first frame.
1369    pub fn finish_warm_up(&mut self, device: &wgpu::Device) {
1370        for desc in warm_up_list(&self.shaders, self.format, self.routing.uber) {
1371            let _ = self.pipelines.get_or_create(device, &desc);
1372        }
1373    }
1374
1375    /// Waits up to `timeout` for the background warm-up [`Self::new`] started
1376    /// to build every pipeline on its list, and returns whether it did.
1377    ///
1378    /// The counterpart of [`Self::finish_warm_up`] that compiles nothing on
1379    /// the calling thread: the worker keeps building at its own pace and this
1380    /// only parks until it is done, so a thread that must not spend its own
1381    /// time compiling can still keep the first frame from being the one that
1382    /// waits on a pipeline. Encodes and submits nothing. Returns at once when
1383    /// the warm-up is already done ([`Self::warm_up_done`]).
1384    pub fn wait_warm_up(&mut self, timeout: Duration) -> bool {
1385        self.pipelines.wait_warm_up(timeout)
1386    }
1387
1388    /// Whether every pipeline on the warm-up list has been built (or failed
1389    /// and been left to its first request), by whichever thread built it.
1390    #[must_use]
1391    pub fn warm_up_done(&self) -> bool {
1392        self.pipelines.warm_up_done()
1393    }
1394
1395    /// Forces every strip run through the whole (Uber) strip program, or
1396    /// lets passes split into paint-class runs again — the programmatic
1397    /// counterpart of `FRUST_ENGINE_UBER_PROGRAM`, which only sets the value a
1398    /// renderer starts with.
1399    ///
1400    /// Takes effect from the next frame. With it set a frame draws exactly
1401    /// what it drew before paint classes existed — one pipeline per pass, the
1402    /// whole `strip.wgsl` program of this renderer's family — which is what a
1403    /// caller comparing the two needs without a process-global environment
1404    /// variable.
1405    pub fn set_uber_program(&mut self, uber: bool) {
1406        self.routing.uber = uber;
1407    }
1408
1409    /// Whether every strip run is drawn with the Uber program.
1410    #[must_use]
1411    pub fn uber_program(&self) -> bool {
1412        self.routing.uber
1413    }
1414
1415    /// Draws a glyph run resolved entirely out of the glyph atlas as glyph
1416    /// quads, or the general way — the programmatic counterpart of
1417    /// `FRUST_ENGINE_NO_GLYPH_QUADS`, which only sets the value a renderer
1418    /// starts with (see [`SceneCompiler::set_glyph_quads`]).
1419    ///
1420    /// Takes effect from the next frame. Both arms draw the same pixels, which
1421    /// is what a caller comparing them in one process checks.
1422    pub fn set_glyph_quads(&mut self, enabled: bool) {
1423        self.compiler.set_glyph_quads(enabled);
1424    }
1425
1426    /// Whether resident glyph runs are drawn as glyph quads.
1427    #[must_use]
1428    pub fn glyph_quads(&self) -> bool {
1429        self.compiler.glyph_quads_enabled()
1430    }
1431
1432    /// Sets the shortest paint-class run worth a pipeline switch of its own
1433    /// (see [`merge_class_runs`]); shorter runs are merged and drawn with the
1434    /// Uber program. `FRUST_ENGINE_RUN_SPLIT_MIN` only sets the value a
1435    /// renderer starts with.
1436    pub fn set_run_split_min(&mut self, min_run: u32) {
1437        self.routing.min_run = min_run;
1438    }
1439
1440    /// The run threshold this renderer splits passes at.
1441    #[must_use]
1442    pub fn run_split_min(&self) -> u32 {
1443        self.routing.min_run
1444    }
1445
1446    /// Records the surface opening as one pass, or as the clear and the
1447    /// opaque strips apart from the first surface round — the programmatic
1448    /// counterpart of `FRUST_ENGINE_NO_PASS_MERGE`, which only sets the value a
1449    /// renderer starts with.
1450    ///
1451    /// Takes effect from the next frame. Unmerged, a frame records the clear
1452    /// and the opaque strips as passes of their own ahead of every round (see
1453    /// the module doc's *The frame's passes*) and draws the same pixels, which
1454    /// is what a caller comparing the two arms in one process needs.
1455    pub fn set_pass_merge(&mut self, merge: bool) {
1456        self.pass_merge = merge;
1457    }
1458
1459    /// Whether the surface opening is recorded as one pass.
1460    #[must_use]
1461    pub fn pass_merge(&self) -> bool {
1462        self.pass_merge
1463    }
1464
1465    /// Routes the frame's per-frame uploads through the staging ring, or
1466    /// through queue writes — the programmatic counterpart of
1467    /// `FRUST_ENGINE_NO_STAGING_RING`, which only sets the value a renderer
1468    /// starts with.
1469    ///
1470    /// Takes effect from the next frame. Both routes upload the same bytes to
1471    /// the same resources, so both draw the same pixels, which is what a caller
1472    /// comparing the two arms in one process checks. Turning the ring off
1473    /// drops its slots; a slot still in flight is kept alive by `wgpu` until
1474    /// its submission retires.
1475    pub fn set_staging_ring(&mut self, enabled: bool) {
1476        match (enabled, self.staging.is_some()) {
1477            (true, false) => self.staging = Some(StagingRing::new()),
1478            (false, true) => self.staging = None,
1479            _ => {}
1480        }
1481    }
1482
1483    /// Whether the frame's per-frame uploads go through the staging ring.
1484    #[must_use]
1485    pub fn staging_ring(&self) -> bool {
1486        self.staging.is_some()
1487    }
1488
1489    /// Bytes the last [`Self::encode`] placed in its staging slot, alignment
1490    /// padding included — zero when the frame's uploads took the queue-write
1491    /// route (the ring off, every slot still in flight, or a frame larger than
1492    /// a slot may hold). A diagnostic: what a test pins to tell the two routes
1493    /// apart.
1494    #[must_use]
1495    pub fn last_frame_staged_bytes(&self) -> u64 {
1496        self.last_staged
1497    }
1498
1499    /// The staging and bind-group counters as they stand now, for the trace's
1500    /// per-frame deltas. A ring that is off reads as zero.
1501    #[cfg(feature = "perf-trace")]
1502    fn trace_counters(&self) -> TraceCounters {
1503        let staging = self.staging_stats().unwrap_or_default();
1504        TraceCounters {
1505            busy: staging.busy,
1506            too_large: staging.too_large,
1507            buffers_created: staging.buffers_created,
1508            bind_groups: self.bind_groups_created(),
1509        }
1510    }
1511
1512    /// What the staging ring has done so far, or `None` with the ring off.
1513    #[must_use]
1514    pub fn staging_stats(&self) -> Option<frust_gpu::staging::StagingStats> {
1515        self.staging.as_ref().map(StagingRing::stats)
1516    }
1517
1518    /// How many bind groups this renderer's frame path has created so far —
1519    /// the strip, external, page-round, composite, backdrop, filter and
1520    /// region-copy groups. A steady frame over unchanged pages creates none, so
1521    /// a test diffs this across two frames to pin that. The glyph-atlas
1522    /// replay's own groups are not counted.
1523    #[must_use]
1524    pub fn bind_groups_created(&self) -> u64 {
1525        self.resources
1526            .groups_created
1527            .saturating_add(self.resources.source_groups.created())
1528            .saturating_add(self.region_copy_groups)
1529            .saturating_add(
1530                self.filters
1531                    .as_ref()
1532                    .map_or(0, |filters| filters.groups_created),
1533            )
1534    }
1535
1536    /// The passes the last [`Self::encode`] recorded into the caller's
1537    /// encoder, in recording order.
1538    ///
1539    /// Empty after a refused frame, which records no pass at all. A
1540    /// diagnostic: how many passes a frame costs is the number a tile-based
1541    /// GPU pays a whole-framebuffer load and store for, so it is what a test
1542    /// pins and a capture is read against.
1543    #[must_use]
1544    pub fn last_frame_passes(&self) -> &[FramePass] {
1545        &self.frame_passes
1546    }
1547
1548    /// How many render pipelines this renderer has compiled so far.
1549    ///
1550    /// Counts *distinct* pipelines, which is why it is a diagnostic rather
1551    /// than something to wait on: two entries of [`EnginePipeline::ALL`] that
1552    /// differ only in their colour format describe the same pipeline whenever
1553    /// the frame's target format happens to equal
1554    /// [`crate::gpu::pipelines::INTERMEDIATE_FORMAT`], and the cache compiles
1555    /// that one variant once. Use [`Self::finish_warm_up`] to wait.
1556    #[must_use]
1557    pub fn compiled_pipelines(&self) -> u64 {
1558        self.pipelines.compiled_variants()
1559    }
1560
1561    /// States whether a caller-supplied depth attachment already holds the
1562    /// depth this frame should test against — see
1563    /// [`DepthAttachment::set_pre_cleared`].
1564    pub fn set_depth_pre_cleared(&mut self, pre_cleared: bool) {
1565        self.depth.set_pre_cleared(pre_cleared);
1566    }
1567
1568    /// Whether a caller-supplied depth attachment is treated as already
1569    /// populated.
1570    ///
1571    /// The statement is sticky and set once, so a host driving several
1572    /// surfaces (or re-establishing one after a device loss) can read back
1573    /// what this renderer is on rather than tracking it a second time.
1574    #[must_use]
1575    pub fn depth_pre_cleared(&self) -> bool {
1576        self.depth.is_pre_cleared()
1577    }
1578
1579    /// Registers a caller-owned texture so a
1580    /// [`Command::SceneTexture`](frust_scene::Command::SceneTexture) naming
1581    /// `id` draws it, returning whatever was registered under `id` before.
1582    ///
1583    /// `id` is the [`SceneTextureId`] the texture minted for itself
1584    /// (`frust_gpu::Texture::as_scene_texture`), and `size` is its extent in
1585    /// texels — the rectangle a display list's destination is mapped onto.
1586    /// `view` must be a non-array 2D view of a float-sampleable texture
1587    /// carrying `wgpu::TextureUsages::TEXTURE_BINDING`; `wgpu` rejects
1588    /// anything else when the frame's bind group is built.
1589    ///
1590    /// Both halves of the registration land here: the view a pass samples and
1591    /// the extent the compiler composes a paint transform against. Registering
1592    /// is idempotent — re-registering the same id replaces the view and drops
1593    /// the bind groups naming the old one.
1594    ///
1595    /// An extent past `u16::MAX` on either axis, or a zero one, registers
1596    /// nothing and answers `None`: the record the shader reads packs the
1597    /// source region into `u16` halves, so there is no honest rectangle to
1598    /// name. Scenes drawing that id go on drawing nothing.
1599    pub fn bind_texture(
1600        &mut self,
1601        id: SceneTextureId,
1602        size: (u32, u32),
1603        view: wgpu::TextureView,
1604    ) -> Option<wgpu::TextureView> {
1605        self.resources.forget_external(id.get());
1606        if !self.compiler.bind_external_texture(id.get(), size) {
1607            return self.textures.unbind(id);
1608        }
1609        self.textures.bind(id, view)
1610    }
1611
1612    /// Removes the texture registered under `id`, returning its view.
1613    pub fn unbind_texture(&mut self, id: SceneTextureId) -> Option<wgpu::TextureView> {
1614        self.compiler.unbind_external_texture(id.get());
1615        self.resources.forget_external(id.get());
1616        self.textures.unbind(id)
1617    }
1618
1619    /// The view registered under `id`, if any.
1620    #[must_use]
1621    pub fn bound_texture(&self, id: SceneTextureId) -> Option<&wgpu::TextureView> {
1622        self.textures.get(id)
1623    }
1624
1625    /// How many external textures are currently bound.
1626    #[must_use]
1627    pub fn bound_texture_count(&self) -> usize {
1628        self.textures.len()
1629    }
1630
1631    /// Releases everything sized against the old surface extent and
1632    /// re-establishes what the next frame needs at the new one.
1633    ///
1634    /// Pipelines, shader modules, the gradient cache and the resource textures
1635    /// are all extent-independent and deliberately survive: a resize must not
1636    /// cost a pipeline rebuild or a ramp re-bake. What goes is the intermediate
1637    /// pool's parked entries — every one keyed on an extent nothing will ask
1638    /// for again — and the engine-owned depth attachment, which has to match
1639    /// its colour attachment exactly. Reallocating the depth buffer here rather
1640    /// than on the next frame keeps it off the frame path.
1641    ///
1642    /// Every retained page goes too: a page holds its body as it was laid out
1643    /// for the old extent, and the resident-key ledger published after the
1644    /// next frame reports the empty store, so every body is painted (and
1645    /// re-recorded) at the new one.
1646    pub fn resize(&mut self, device: &wgpu::Device, width: u32, height: u32) {
1647        self.targets.drop_parked();
1648        // A cached group would otherwise keep a dropped page's texture alive.
1649        self.resources.source_groups.clear();
1650        self.depth.resize(device, width, height);
1651        self.retained.clear();
1652    }
1653
1654    /// The retained pages this renderer holds resident after its last frame —
1655    /// the renderer's half of the resident-key ledger (see
1656    /// [`crate::cache::retained`]), in no particular order.
1657    pub fn retained_ledger(&self) -> impl Iterator<Item = RetainedEntry> + '_ {
1658        self.retained.entries()
1659    }
1660
1661    /// The retained store's frame clock — how many frames it has planned —
1662    /// for a ledger's diagnostic frame stamp.
1663    #[must_use]
1664    pub fn retained_frame(&self) -> u64 {
1665        self.retained.frame()
1666    }
1667
1668    /// A counter advanced whenever what [`Self::retained_ledger`] reports
1669    /// changes; a host publishing the ledger only when it moved publishes
1670    /// every change and nothing else.
1671    #[must_use]
1672    pub fn retained_generation(&self) -> u64 {
1673        self.retained.generation()
1674    }
1675
1676    /// Whether a frame was refused with [`EngineError::RetainedKeyMissing`]
1677    /// (and the store cleared) since the last call, clearing the flag. A host
1678    /// that sees `true` owes the widget tree one more paint against the
1679    /// now-empty ledger: the refused frame presented nothing.
1680    pub fn take_retained_refusal(&mut self) -> bool {
1681        self.retained.take_refusal()
1682    }
1683
1684    /// Drop every retained page — what a surface recreate or a device-scale
1685    /// change calls for. The next ledger read reports the empty store.
1686    pub fn clear_retained(&mut self) {
1687        // A cached group would otherwise keep a dropped page's texture alive.
1688        self.resources.source_groups.clear();
1689        self.retained.clear();
1690    }
1691
1692    /// The byte budget the retained store is held to.
1693    #[must_use]
1694    pub fn retained_budget(&self) -> RetainedBudget {
1695        self.retained.budget()
1696    }
1697
1698    /// Re-budget the retained store, evicting least recently used pages until
1699    /// what it holds fits.
1700    pub fn set_retained_budget(&mut self, budget: RetainedBudget) {
1701        // A cached group would otherwise keep an evicted page's texture alive.
1702        self.resources.source_groups.clear();
1703        self.retained.set_budget(budget);
1704    }
1705
1706    /// Set whether this renderer records retained brackets at all — the
1707    /// programmatic counterpart of `FRUST_ENGINE_NO_LAYERS` (see
1708    /// [`SceneCompiler::set_retained_layers`]). Turning it off clears the
1709    /// store, since nothing would ever draw from it again.
1710    pub fn set_retained_layers(&mut self, enabled: bool) {
1711        self.compiler.set_retained_layers(enabled);
1712        if !enabled {
1713            self.retained.clear();
1714        }
1715    }
1716
1717    /// Whether this renderer records retained brackets.
1718    #[must_use]
1719    pub fn retained_layers(&self) -> bool {
1720        self.compiler.retained_layers_enabled()
1721    }
1722
1723    /// Takes the time this renderer's pipeline cache has spent resolving
1724    /// variants it did not already hold (waiting on the warm-up worker's
1725    /// in-flight build, or building one inline) since the last call.
1726    ///
1727    /// `perf-trace`-only: it feeds the first frame's
1728    /// `first_submit_pipeline_wait` startup span. Time spent in
1729    /// [`Self::wait_warm_up`] counts too.
1730    #[cfg(feature = "perf-trace")]
1731    pub fn take_pipeline_wait(&mut self) -> Duration {
1732        self.pipelines.take_resolve_wait()
1733    }
1734
1735    /// How many pipelines the warm-up worker has finished building.
1736    #[must_use]
1737    pub fn warm_up_finished(&self) -> u64 {
1738        self.pipelines.warm_up_finished()
1739    }
1740
1741    /// Closes the frame out: ages the intermediate pool by one frame and
1742    /// evicts the gradient cache down to its capacity.
1743    ///
1744    /// Call once per frame, after the frame's commands have been submitted and
1745    /// including frames that drew nothing — those are the frames a parked
1746    /// intermediate ages on. Gradient eviction compacts the packed LUT buffer
1747    /// and rewrites the offsets of the survivors, which is why it belongs at
1748    /// the frame boundary rather than mid-frame, where it would invalidate
1749    /// offsets the frame's own encoded paints already carry.
1750    ///
1751    /// `_queue` is part of the signature because the end-of-frame maintenance
1752    /// this method owns grows queue writes as the engine does (an atlas region
1753    /// cleared after the frame that consumed it, in the reference renderer);
1754    /// it has none of them yet.
1755    pub fn end_frame(&mut self, _queue: &wgpu::Queue) {
1756        self.targets.end_frame();
1757        self.gradients.maintain();
1758        self.resources.source_groups.sweep();
1759    }
1760
1761    /// Compiles `scene` and records the frame's passes into `encoder`.
1762    ///
1763    /// `root` is applied ahead of every command's own transform and
1764    /// `base_color` is what the target is cleared to before anything is drawn.
1765    /// Neither the encoder nor the queue is submitted — see the module header.
1766    ///
1767    /// # Errors
1768    ///
1769    /// [`EngineError::TargetTooLarge`] for a target outside the `u16` device
1770    /// grid the strip pipeline addresses, [`EngineError::InvalidTransform`] for
1771    /// a non-finite transform, [`EngineError::InvalidGeometry`] for non-finite
1772    /// command geometry (rect extents, radii, path points, stroke or dash
1773    /// values), [`EngineError::SchedulerEscalation`] for a layer shape the
1774    /// engine's scheduler does not serve,
1775    /// [`EngineError::IntermediateTextureTooLarge`] for a layer no page can be
1776    /// sized to, [`EngineError::AlphaCapacity`] when a frame's coverage
1777    /// outgrows the alpha texture, and [`EngineError::PaintCapacity`] when its
1778    /// encoded paints or colour ramps outgrow theirs. Every one of them is
1779    /// returned before anything is recorded, uploaded, allocated or submitted,
1780    /// so a refused frame leaves `encoder` exactly as it was found and the
1781    /// renderer's own resources — the atlas array included — exactly as they
1782    /// were. That is what lets the caller skip the frame cleanly (nothing is
1783    /// presented and the previously presented content persists) rather than
1784    /// present it half-drawn, and what keeps a refused frame's image uploads
1785    /// alive for the next frame that is not refused.
1786    #[expect(
1787        clippy::too_many_arguments,
1788        reason = "the seam frust-render drives: device, queue, encoder, scene, \
1789                  target, base colour and root transform are each supplied by a \
1790                  different owner, so bundling them would only move the \
1791                  assembly to every call site"
1792    )]
1793    pub fn encode(
1794        &mut self,
1795        device: &wgpu::Device,
1796        queue: &wgpu::Queue,
1797        encoder: &mut wgpu::CommandEncoder,
1798        scene: &Scene,
1799        target: EngineTarget<'_>,
1800        base_color: Color,
1801        root: Affine,
1802    ) -> Result<(), EngineError> {
1803        self.encode_traced(
1804            device,
1805            queue,
1806            encoder,
1807            scene,
1808            target,
1809            base_color,
1810            root,
1811            FrameTimestamps::inert(),
1812        )
1813    }
1814
1815    /// [`Self::encode`], with each pass's GPU time stamped into `timestamps`.
1816    ///
1817    /// The one difference is the sink: every pass this records asks
1818    /// `timestamps` for its own `timestamp_writes` and takes `None` for an
1819    /// answer, so a frame encoded with [`FrameTimestamps::inert`] — which is
1820    /// exactly what [`Self::encode`] passes — records byte-identical work.
1821    /// Which pass is charged to which span is [`EngineSpan`]'s own
1822    /// documentation; the host owns the ring behind the sink and reads the
1823    /// frame's spans back out of it some frames later (see
1824    /// [`frust_gpu::diag::TimestampRing`]).
1825    ///
1826    /// # Errors
1827    ///
1828    /// Exactly [`Self::encode`]'s, on exactly its terms — a refused frame has
1829    /// recorded no pass, so it has taken no timestamp either and the host
1830    /// abandons the ring's slot rather than mapping it.
1831    #[expect(
1832        clippy::too_many_arguments,
1833        reason = "[`Self::encode`]'s argument list plus the timestamp sink, \
1834                  each still supplied by a different owner"
1835    )]
1836    pub fn encode_traced(
1837        &mut self,
1838        device: &wgpu::Device,
1839        queue: &wgpu::Queue,
1840        encoder: &mut wgpu::CommandEncoder,
1841        scene: &Scene,
1842        target: EngineTarget<'_>,
1843        base_color: Color,
1844        root: Affine,
1845        timestamps: FrameTimestamps<'_>,
1846    ) -> Result<(), EngineError> {
1847        // The CPU half of the frame's accounting, lapped phase by phase
1848        // through to the end of the call (see [`EncodeSpans`]). Free without
1849        // `perf-trace` — no clock is read at all.
1850        let mut clock = PhaseClock::start();
1851        // Emptied first, so a frame refused below reports the nothing it
1852        // recorded rather than the last frame's passes. The atlas report is
1853        // the same kind of record: a frame that errors before its replay
1854        // would otherwise carry the last replay's numbers forward.
1855        self.frame_passes.clear();
1856        self.atlas_report = AtlasRenderReport::default();
1857        let size = grid_size(target.width, target.height)?;
1858        let mut frame = self.compiler.compile(scene, root, size)?;
1859        let compile_total = clock.lap();
1860
1861        // The frame's pass plan, settled before anything is allocated or
1862        // recorded: a layer shape this scheduler does not serve, or one larger
1863        // than a page can be sized to, refuses the whole frame here so the
1864        // caller can skip it cleanly rather than present it half-drawn.
1865        //
1866        // The retained store decides first, because its answers are roles the
1867        // scheduler serves: which bodies render into a kept page, where each
1868        // `DrawRetained` composites — or that one cannot be served, which
1869        // refuses the frame here and empties the store (see
1870        // `crate::cache::retained`). Nothing it decides is applied until the
1871        // frame is past its last fallible step.
1872        let retained_plan = self
1873            .retained
1874            .plan(&frame.retained, &mut frame.recorder.layers)?;
1875        retained_roles(&frame, &retained_plan, &mut self.retained_layers);
1876        // Whether the frame's backdrop blurs can be served at all: each one
1877        // reads its region of this very target back (see `RegionCopyMode`).
1878        // A target that can be neither copied nor sampled serves none — every
1879        // backdrop's rectangle is then the transparent draw the compiler
1880        // recorded, drawing nothing, and is scheduled as the ordinary layer it
1881        // is — rather than refusing a frame it can otherwise draw whole.
1882        let copy_mode = RegionCopyMode::of(target.view.texture());
1883        let serve_backdrops = copy_mode != RegionCopyMode::Unavailable;
1884        if !frame.backdrops.is_empty() && !serve_backdrops {
1885            note_backdrop_unreadable(target.view.texture());
1886        }
1887        let backdrops: &[Backdrop] = if serve_backdrops {
1888            &frame.backdrops
1889        } else {
1890            &[]
1891        };
1892        // The largest region a texture-mode frame transfers, which the
1893        // staging texture is grown to before the first pass is recorded.
1894        let staging_extent = backdrops.iter().fold((0_u32, 0_u32), |(w, h), backdrop| {
1895            (
1896                w.max(u32::from(backdrop.region.width())),
1897                h.max(u32::from(backdrop.region.height())),
1898            )
1899        });
1900        let rounds = Schedule::build_frame(
1901            &frame.recorder,
1902            &self.caps,
1903            &self.pages,
1904            &self.retained_layers,
1905            backdrops,
1906        )?;
1907        let ceiling = self.targets.max_texture_size();
1908        for round in &rounds {
1909            if let Some(page) = round.page()
1910                && (page.size.width > ceiling || page.size.height > ceiling)
1911            {
1912                return Err(EngineError::IntermediateTextureTooLarge);
1913            }
1914        }
1915
1916        // Depth is available only when there is an attachment to use and the
1917        // kill switch is off. Settling that before a single instance is built
1918        // is what keeps the opaque/alpha split and the pass shape in agreement.
1919        let depth_enabled = !config::depth_disabled();
1920        if depth_enabled && target.depth.is_none() {
1921            self.depth.ensure(device, target.width, target.height);
1922        }
1923        // Cloned out of the renderer rather than borrowed from it: an engine-
1924        // owned attachment lives in `self.depth`, and recording the frame needs
1925        // `self` mutably (the page pool, the pipeline cache). A `wgpu`
1926        // texture view is a reference-counted handle, so the clone is a
1927        // refcount bump once per frame rather than an allocation.
1928        let depth_view = depth_enabled
1929            .then(|| target.depth.or_else(|| self.depth.owned_view()))
1930            .flatten()
1931            .cloned();
1932        let depth_view = depth_view.as_ref();
1933
1934        // Destination-out erases colour as well as alpha, so a target whose
1935        // alpha is disregarded would take a black rectangle where the display
1936        // list says nothing changes. A frame cleared to an opaque base colour
1937        // is exactly that target — every pixel of it presents opaquely — and is
1938        // the only such statement `encode` is handed, so it is what the skip
1939        // `compile::clear`'s contract calls for is decided on.
1940        let punches = !frame.clears.is_empty() && !is_opaque(base_color);
1941        let schedule_span = clock.lap();
1942
1943        // Paints are resolved before instances are built: an instance names
1944        // its paint by the texel its record starts at, which only exists once
1945        // the frame's ramps are resident and its records are laid out.
1946        let atlas_budget = self.compiler.images().budget();
1947        self.resources
1948            .resolve_paints(&frame, &mut self.gradients, atlas_budget, &self.textures);
1949        let paints_span = clock.lap();
1950        self.scratch.routing = self.routing;
1951        self.scratch.build(
1952            &frame,
1953            &rounds,
1954            depth_view.is_some(),
1955            punches,
1956            &self.resources.paint_slots,
1957            if serve_backdrops {
1958                &frame.backdrops
1959            } else {
1960                &[]
1961            },
1962        );
1963        let instances_span = clock.lap();
1964
1965        // Everything that can fail does so here, ahead of the first
1966        // `begin_render_pass` AND ahead of the first thing this frame changes
1967        // about the renderer's frame-visible state: apart from the engine's
1968        // own depth attachment (re-sized above, an internal resource no pass
1969        // has read yet), a frame refused below has allocated nothing, replaced
1970        // no texture and submitted nothing, which is what lets the caller
1971        // skip it cleanly.
1972        let dim = self.caps.resource_texture_dim;
1973        let alphas_grown = gpu::grow_alpha_texture_height(
1974            self.resources.alphas.height,
1975            frame.alphas().len(),
1976            dim,
1977        )?;
1978        let paints_grown = gpu::paint_texture::grow_encoded_paints_texture_height(
1979            self.resources.paints.height,
1980            self.resources.paint_texels(),
1981            dim,
1982        )?;
1983        let gradients_grown = self.resources.grown_gradient_height(&self.gradients)?;
1984
1985        // Past the last fallible step. The retained store applies its plan
1986        // first — evictions, then a page for every body a round really renders
1987        // into — so every retained page the frame's rounds name exists before
1988        // they are recorded. The evictions include every key the frame
1989        // re-recorded whose page will not hold that recording (refused,
1990        // nested or unrendered), so the ledger published after this frame
1991        // never names a stale page.
1992        let rendered = retained_rendered(&frame, &rounds);
1993        self.retained.commit(
1994            device,
1995            &frame.retained,
1996            &frame.retained_superseded,
1997            &retained_plan,
1998            &rendered,
1999        );
2000
2001        // The atlas is created or grown first, so
2002        // the growth copy's own submit precedes the frame's atlas writes below
2003        // (see `gpu::atlas`) and the array is deep enough for every region they
2004        // name.
2005        self.resources
2006            .ensure_atlas(device, queue, atlas_budget, frame.atlas_layers);
2007
2008        // The glyph atlas, in the order [`crate::gpu::atlas`] documents: the
2009        // rectangles last frame's eviction freed are zeroed first, ahead of
2010        // every write this frame issues, so a rectangle handed straight back
2011        // out cannot be erased after its new occupant landed in it.
2012        //
2013        // Acknowledged only once the writes were really issued — the same
2014        // re-offer contract the image plan keeps below. A frame refused before
2015        // this point leaves every rectangle pending, so the next frame that
2016        // gets here still zeroes it.
2017        if self.clear_glyph_rects(device, queue, &frame) {
2018            self.compiler.acknowledge_glyph_clears();
2019        }
2020
2021        self.resources.resize_alphas(device, alphas_grown);
2022        self.resources.resize_paints(device, paints_grown);
2023        self.resources.resize_gradients(device, gradients_grown);
2024        let resize_span = clock.lap();
2025        // The frame's staging slot, opened only now that nothing can refuse
2026        // the frame: the per-frame uploads below — and each page round's
2027        // viewport uniform in `record_frame` — are written into it and copied
2028        // out of it into `encoder`, ahead of every pass this frame records
2029        // that reads them. `None` sends them through queue writes instead.
2030        let mut stage = self.begin_staging(device);
2031        let atlas_serviced = self.resources.upload(
2032            queue,
2033            encoder,
2034            stage.as_mut(),
2035            &mut frame,
2036            &mut self.gradients,
2037            size,
2038            dim,
2039        );
2040        self.resources
2041            .upload_instances(device, queue, encoder, stage.as_mut(), &self.scratch);
2042
2043        // Residency is committed exactly here: the frame passed every fallible
2044        // step and its evictions and uploads have reached the array, so the
2045        // compiler may stop re-offering them. A frame that returned early above
2046        // never gets here, and its plan is re-offered on the next frame that
2047        // does (see `cache::images`).
2048        if atlas_serviced {
2049            self.compiler.acknowledge_image_plan();
2050        }
2051        let upload_span = clock.lap();
2052
2053        // The glyph pixels themselves, last of the atlas work and strictly
2054        // before the scene pass: every page `glifo` dirtied this frame is
2055        // lowered to strips and drawn into its own array layer, on an encoder
2056        // this call owns and submits (the sanctioned exception to the encode
2057        // contract — see this module's header and `gpu::atlas`). The queue
2058        // writes issued above are flushed ahead of that submit, so the pass
2059        // composites onto a layer whose clears and image uploads have landed.
2060        //
2061        // Driven by the atlas's own pending work, never by this frame's
2062        // surviving draws: `glifo` dirties a page when it *inserts* an entry,
2063        // so a run culled away behind a clip records fills while drawing
2064        // nothing, and a draw-gated replay would leave those commands recorded
2065        // until some later frame happened to run one — by which time eviction
2066        // may have re-let the rectangles they name.
2067        //
2068        // Acknowledged only when the pass really ran, the same way the clears
2069        // above are: acknowledging is what lifts the eviction deferral, so an
2070        // acknowledgement for a replay that returned early would let `glifo`
2071        // free and re-let the very rectangles those commands still name.
2072        #[cfg(feature = "perf-trace")]
2073        let mut replayed = false;
2074        if self.compiler.glyph_replay_pending()
2075            && self.replay_glyph_pages(device, queue, timestamps)
2076        {
2077            self.compiler.acknowledge_glyph_replay();
2078            #[cfg(feature = "perf-trace")]
2079            {
2080                replayed = true;
2081            }
2082        }
2083
2084        let replay_span = clock.lap();
2085
2086        let format = target.format;
2087        let pipelines = self.frame_pipelines(device, format, depth_view.is_some(), copy_mode);
2088        if pipelines.copy.is_some() {
2089            let copies = self
2090                .region_copies
2091                .get_or_insert_with(|| RegionCopyResources::new(device));
2092            if copy_mode == RegionCopyMode::Texture {
2093                copies.ensure_staging(device, staging_extent.0, staging_extent.1);
2094            }
2095        }
2096
2097        // The filter rounds' own resources, created by the first frame that
2098        // schedules one. Only the parameter blocks are uploaded here: a pass's
2099        // instance names the extent the *pool* quantized its destination page
2100        // up to, which only the round that acquires it knows, so the instances
2101        // are written round by round in `record_frame`.
2102        if let Some(pipeline) = pipelines.filter.as_ref() {
2103            let scratch = &self.scratch;
2104            self.filters
2105                .get_or_insert_with(|| FilterResources::new(device))
2106                .prepare(
2107                    device,
2108                    queue,
2109                    pipeline,
2110                    &scratch.filter_blocks,
2111                    scratch.filter_passes,
2112                );
2113        }
2114
2115        let pipelines_span = clock.lap();
2116
2117        self.record_frame(
2118            device,
2119            queue,
2120            encoder,
2121            stage.as_mut(),
2122            &target,
2123            depth_view,
2124            base_color,
2125            &pipelines,
2126            copy_mode,
2127            timestamps,
2128        );
2129        // Unmapped and fenced on `encoder`: the slot is handed out again only
2130        // once the submission carrying this frame has finished on the GPU.
2131        self.last_staged = stage.as_ref().map_or(0, StagingFrame::used);
2132        if let Some(stage) = stage {
2133            stage.finish(encoder);
2134        }
2135
2136        // The laps above still run without `perf-trace` — `PhaseClock` reads
2137        // no clock there, so each one is `Duration::ZERO` — but nothing records
2138        // them, and the trace they would have fed is not compiled at all.
2139        #[cfg(not(feature = "perf-trace"))]
2140        let _ = (
2141            compile_total,
2142            schedule_span,
2143            paints_span,
2144            instances_span,
2145            resize_span,
2146            upload_span,
2147            replay_span,
2148            pipelines_span,
2149            clock.lap(),
2150        );
2151
2152        // Last, so the window's own formatting is charged to no phase it
2153        // reports. A frame refused above records nothing: its phases are a
2154        // partial encode and would drag every percentile toward a frame that
2155        // was never presented.
2156        #[cfg(feature = "perf-trace")]
2157        let now = self.trace_counters();
2158        #[cfg(feature = "perf-trace")]
2159        self.encode_trace.record(
2160            &EncodeSpans {
2161                compile: frame.compile_spans,
2162                compile_total,
2163                schedule: schedule_span,
2164                paints: paints_span,
2165                instances: instances_span,
2166                resize: resize_span,
2167                upload: upload_span,
2168                replay: replay_span,
2169                pipelines: pipelines_span,
2170                record: clock.lap(),
2171                counts: EncodeCounts::of(
2172                    &frame,
2173                    replayed,
2174                    &self.atlas_report,
2175                    self.compiler.glyph_atlas_trace(),
2176                    self.last_staged,
2177                ),
2178            },
2179            now,
2180        );
2181
2182        Ok(())
2183    }
2184
2185    /// [`Self::encode_traced`] over the commands of `scene` in `range` alone,
2186    /// cut out by [`project_command_range`] — the overlay spike's way of
2187    /// drawing one segment of a frame into a target of its own (render-model
2188    /// Phase 4, decision D10).
2189    ///
2190    /// The range is projected into a scene this renderer keeps for the
2191    /// purpose (its allocation reused from call to call), carrying
2192    /// `DrawRetained`s only when this renderer retains
2193    /// ([`Self::retained_layers`]) — a renderer with retention off refuses a
2194    /// frame carrying one. The bracket rule is [`project_command_range`]'s.
2195    /// `target`, `base_color` (an overlay passes transparent), `root` and
2196    /// `timestamps` mean exactly what they mean to [`Self::encode_traced`].
2197    ///
2198    /// A range covering the whole scene is still projected (and re-recorded),
2199    /// so a caller that draws the whole frame calls [`Self::encode_traced`]
2200    /// instead; this method is never on a path that does.
2201    ///
2202    /// # Errors
2203    ///
2204    /// Exactly [`Self::encode_traced`]'s, over the projected scene.
2205    #[expect(
2206        clippy::too_many_arguments,
2207        reason = "[`Self::encode_traced`]'s argument list plus the range, each \
2208                  still supplied by a different owner"
2209    )]
2210    pub fn record_range(
2211        &mut self,
2212        device: &wgpu::Device,
2213        queue: &wgpu::Queue,
2214        encoder: &mut wgpu::CommandEncoder,
2215        scene: &Scene,
2216        range: Range<usize>,
2217        target: EngineTarget<'_>,
2218        base_color: Color,
2219        root: Affine,
2220        timestamps: FrameTimestamps<'_>,
2221    ) -> Result<RangeCut, EngineError> {
2222        // Taken out for the call (and always put back): `encode_traced`
2223        // borrows the renderer mutably while it reads the scene.
2224        let mut projected = std::mem::take(&mut self.range_scene);
2225        let options = RangeOptions {
2226            keep_draw_retained: self.retained_layers(),
2227        };
2228        let cut = project_command_range(scene, range, options, &mut projected);
2229        let result = self.encode_traced(
2230            device, queue, encoder, &projected, target, base_color, root, timestamps,
2231        );
2232        self.range_scene = projected;
2233        result.map(|()| cut)
2234    }
2235
2236    /// Opens this frame's staging slot, sized for every upload the frame routes
2237    /// through it, or `None` when the ring is off or refuses the frame (every
2238    /// slot still in flight, or more bytes than a slot may hold) — in which
2239    /// case each of those uploads is a queue write, as without the ring.
2240    fn begin_staging(&mut self, device: &wgpu::Device) -> Option<StagingFrame> {
2241        let bytes = self.staging_bytes();
2242        self.staging.as_mut()?.begin(device, bytes)
2243    }
2244
2245    /// An upper bound on the bytes this frame stages, alignment padding
2246    /// included: the whole alpha texture, the whole encoded-paint texture
2247    /// when the frame has paints, the viewport uniform, the instances, and
2248    /// one viewport uniform per scheduled round (more than the page rounds
2249    /// that take one, which is all an upper bound needs).
2250    fn staging_bytes(&self) -> u64 {
2251        let resources = &self.resources;
2252        let texture = |texture: &ResourceTexture| {
2253            gpu::resource_texture_bytes(texture.width, texture.height)
2254                .saturating_add(TEXTURE_COPY_ALIGNMENT)
2255        };
2256        let buffer = |len: u64| len.saturating_add(BUFFER_COPY_ALIGNMENT);
2257        let config = buffer(size_of::<GpuConfig>() as u64);
2258        let (opaque, alpha) = self.scratch.instance_bytes();
2259        let paints = if resources.paints_data.is_empty() {
2260            0
2261        } else {
2262            texture(&resources.paints)
2263        };
2264        let rounds = self.scratch.rounds.len() as u64;
2265        texture(&resources.alphas)
2266            .saturating_add(paints)
2267            .saturating_add(config)
2268            .saturating_add(buffer((opaque.len() + alpha.len()) as u64))
2269            .saturating_add(config.saturating_mul(rounds))
2270    }
2271
2272    /// Creates the render-to-atlas pass on first use.
2273    fn ensure_atlas_renderer(&mut self, device: &wgpu::Device) {
2274        if self.atlas_glyphs.is_none() {
2275            self.atlas_glyphs = Some(AtlasRenderer::new(device, &self.caps));
2276        }
2277    }
2278
2279    /// Zero every atlas rectangle an earlier frame's glyph eviction freed,
2280    /// answering whether they were serviced.
2281    ///
2282    /// Queue writes, issued before this frame's image uploads and before the
2283    /// replay pass's submit — the first of the three orderings
2284    /// [`crate::gpu::atlas`] states. A rectangle the array will not take is
2285    /// counted rather than dropped silently, on the same terms an image region
2286    /// it refuses is.
2287    ///
2288    /// `true` means every rectangle was *offered* to the array — including one
2289    /// it refused, which no later frame could place either — so the caller may
2290    /// stop re-offering them. `false` means there was no array to write to at
2291    /// all, which is the one case where trying again later can succeed. An
2292    /// empty list is serviced trivially.
2293    fn clear_glyph_rects(
2294        &mut self,
2295        device: &wgpu::Device,
2296        queue: &wgpu::Queue,
2297        frame: &CompiledFrame,
2298    ) -> bool {
2299        if frame.glyph_clears.is_empty() {
2300            return true;
2301        }
2302        if self.resources.atlas.is_none() {
2303            return false;
2304        }
2305        self.ensure_atlas_renderer(device);
2306
2307        let refused = {
2308            let Self {
2309                atlas_glyphs,
2310                resources,
2311                ..
2312            } = self;
2313            let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_ref(), resources.atlas.as_ref())
2314            else {
2315                return false;
2316            };
2317            frame
2318                .glyph_clears
2319                .iter()
2320                .filter(|rect| !glyphs.clear_rect(queue, atlas, **rect))
2321                .count()
2322        };
2323
2324        if refused > 0 {
2325            self.resources.note_refused_regions(refused as u64);
2326        }
2327        true
2328    }
2329
2330    /// Draw every atlas page `glifo` dirtied this frame into its own array
2331    /// layer.
2332    ///
2333    /// The pixels of a newly cached glyph, and the last atlas work before the
2334    /// scene pass. Each page's recorded commands are lowered to strips by
2335    /// [`lower_atlas_page`] and drawn through the pipeline
2336    /// [`atlas_strip_desc`] describes; a page the lowering declines is left
2337    /// undrawn and counted, so a glyph whose shape this tier cannot express
2338    /// goes *missing* rather than landing half-painted.
2339    ///
2340    /// Answers whether the pass really ran, on the same terms
2341    /// [`clear_glyph_rects`](Self::clear_glyph_rects) does and for the same
2342    /// reason: `false` means there was no atlas array to draw into at all, so
2343    /// the recorded commands are still recorded and the caller must go on
2344    /// offering them. A page the lowering *declined* is not a `false` — it was
2345    /// offered to the array and counted refused, and no later frame could lower
2346    /// it either.
2347    ///
2348    /// `timestamps` is passed straight through to
2349    /// [`gpu::atlas::AtlasRenderer::render_pending`], which charges each dirty
2350    /// page's own pass to [`EngineSpan::Prepass`] — this is the frame's own
2351    /// Prepass recording site.
2352    #[must_use]
2353    fn replay_glyph_pages(
2354        &mut self,
2355        device: &wgpu::Device,
2356        queue: &wgpu::Queue,
2357        timestamps: FrameTimestamps<'_>,
2358    ) -> bool {
2359        if self.resources.atlas.is_none() {
2360            return false;
2361        }
2362        self.ensure_atlas_renderer(device);
2363        let pipeline = self
2364            .pipelines
2365            .get_or_create(device, &atlas_strip_desc(&self.shaders))
2366            .clone();
2367
2368        let report = {
2369            let Self {
2370                atlas_glyphs,
2371                atlas_lowering,
2372                resources,
2373                compiler,
2374                ..
2375            } = self;
2376            let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_mut(), resources.atlas.as_ref())
2377            else {
2378                return false;
2379            };
2380
2381            let (width, height) = atlas.size();
2382            let page = (
2383                u16::try_from(width).unwrap_or(u16::MAX),
2384                u16::try_from(height).unwrap_or(u16::MAX),
2385            );
2386            let lowering = atlas_lowering.get_or_insert_with(|| SceneCompiler::new(page.0, page.1));
2387
2388            glyphs.render_pending(
2389                device,
2390                queue,
2391                &pipeline,
2392                atlas,
2393                compiler.glyph_atlas_mut(),
2394                timestamps,
2395                |recorder, buffers| lower_atlas_page(recorder, buffers, lowering, page),
2396            )
2397        };
2398
2399        if report.refused > 0 {
2400            self.resources
2401                .note_refused_regions(u64::from(report.refused));
2402        }
2403        self.atlas_report = report;
2404        true
2405    }
2406
2407    /// What the last frame's render-to-atlas pass serviced.
2408    ///
2409    /// Zero across the board on a steady-state frame: text that hit the cache
2410    /// on every glyph frees no rectangle, queues no pixmap and dirties no page.
2411    #[must_use]
2412    pub fn atlas_render_report(&self) -> AtlasRenderReport {
2413        self.atlas_report
2414    }
2415
2416    /// Builds (or takes from the cache) every pipeline this frame's passes
2417    /// need, and the bind groups each of them will be bound through.
2418    ///
2419    /// All of it happens before the first `begin_render_pass`: a pipeline
2420    /// compiled mid-recording would be the very stall the warm-up exists to
2421    /// avoid. Each pass resolves exactly the paint classes its runs name
2422    /// ([`Scratch`]'s class sets), folded to what its state serves; the bind
2423    /// groups are built once per state, against any one of its class
2424    /// pipelines — every strip pipeline shares one explicit layout, so a group
2425    /// built against one binds under all of them.
2426    fn frame_pipelines(
2427        &mut self,
2428        device: &wgpu::Device,
2429        format: wgpu::TextureFormat,
2430        depth: bool,
2431        copy_mode: RegionCopyMode,
2432    ) -> FramePipelines {
2433        let alpha_variant = if depth {
2434            EnginePipeline::StripDepthAlpha
2435        } else {
2436            EnginePipeline::StripAlpha
2437        };
2438        let punch_variant = if depth {
2439            EnginePipeline::StripDepthDestOut
2440        } else {
2441            EnginePipeline::StripDestOut
2442        };
2443        let uber = ClassSet::only(PaintClass::Uber);
2444        if self.resolved.format != Some(format) {
2445            self.resolved = ResolvedPipelines {
2446                format: Some(format),
2447                ..ResolvedPipelines::default()
2448            };
2449            self.resources.source_groups.clear();
2450            // Derived from the copy pipeline built for the old format.
2451            if let Some(copies) = self.region_copies.as_mut() {
2452                copies.staging_group = None;
2453            }
2454        }
2455
2456        // The surface's own pass is unconditional — it is what the frame's
2457        // bind groups are built against even when it draws nothing — so a frame
2458        // with no surface run still resolves one pipeline for it: the Uber one,
2459        // which the warm-up already built.
2460        let surface_classes = if self.scratch.surface_classes.is_empty() {
2461            uber
2462        } else {
2463            self.scratch.surface_classes
2464        };
2465        let mut frame = FramePipelines {
2466            alpha: (
2467                alpha_variant,
2468                self.class_pipelines(device, format, alpha_variant, surface_classes),
2469            ),
2470            opaque: None,
2471            page: None,
2472            punch: None,
2473            filter: None,
2474            copy: None,
2475        };
2476        if depth && !self.scratch.opaque.is_empty() {
2477            frame.opaque = Some(self.class_pipelines(
2478                device,
2479                format,
2480                EnginePipeline::StripOpaque,
2481                self.scratch.opaque_classes,
2482            ));
2483        }
2484        if self.scratch.page_rounds() > 0 {
2485            let page_classes = if self.scratch.page_classes.is_empty() {
2486                uber
2487            } else {
2488                self.scratch.page_classes
2489            };
2490            frame.page = Some(self.class_pipelines(
2491                device,
2492                format,
2493                EnginePipeline::StripIntermediate,
2494                page_classes,
2495            ));
2496        }
2497        if self.scratch.punches() {
2498            frame.punch = Some((
2499                punch_variant,
2500                self.class_pipelines(
2501                    device,
2502                    format,
2503                    punch_variant,
2504                    ClassSet::only(self.routing.punch_class()),
2505                ),
2506            ));
2507        }
2508        if self.scratch.filter_passes > 0 {
2509            // Takes the frame's format like every other variant and ignores
2510            // it: a filter pass only ever writes a pooled page, so its own
2511            // description is pinned to `INTERMEDIATE_FORMAT`. One filter
2512            // pipeline therefore serves a renderer for its whole life,
2513            // whatever its surface is reconfigured to.
2514            if self.resolved.filter.is_none() {
2515                self.resolved.filter = Some(
2516                    self.pipelines
2517                        .get_or_create(device, &EnginePipeline::Filter.desc(&self.shaders, format))
2518                        .clone(),
2519                );
2520            }
2521            frame.filter.clone_from(&self.resolved.filter);
2522        }
2523
2524        if self.scratch.copies > 0 && copy_mode != RegionCopyMode::Unavailable {
2525            // Pinned to `INTERMEDIATE_FORMAT` like the filter pipeline — a copy
2526            // only ever writes a pooled page — so one serves the renderer for
2527            // its whole life whatever the frame's own format.
2528            if self.resolved.copy.is_none() {
2529                self.resolved.copy = Some(
2530                    self.pipelines
2531                        .get_or_create(device, &EnginePipeline::Copy.desc(&self.shaders, format))
2532                        .clone(),
2533                );
2534            }
2535            frame.copy.clone_from(&self.resolved.copy);
2536        }
2537
2538        if let Some(pipeline) = frame.alpha.1.any() {
2539            self.resources
2540                .ensure_bind_groups(device, frame.alpha.0, pipeline, format);
2541            self.resources
2542                .ensure_external_groups(device, frame.alpha.0, pipeline, &self.textures);
2543        }
2544        if let Some(pipeline) = frame.opaque.as_ref().and_then(ClassPipelines::any) {
2545            self.resources.ensure_bind_groups(
2546                device,
2547                EnginePipeline::StripOpaque,
2548                pipeline,
2549                format,
2550            );
2551        }
2552        if let Some(pipeline) = frame.page.as_ref().and_then(ClassPipelines::any) {
2553            self.resources.ensure_bind_groups(
2554                device,
2555                EnginePipeline::StripIntermediate,
2556                pipeline,
2557                format,
2558            );
2559            // A layer's own round draws through this variant, so an external
2560            // texture inside an isolated layer needs its group here too.
2561            self.resources.ensure_external_groups(
2562                device,
2563                EnginePipeline::StripIntermediate,
2564                pipeline,
2565                &self.textures,
2566            );
2567        }
2568        if let Some((variant, pipelines)) = frame.punch.as_ref()
2569            && let Some(pipeline) = pipelines.any()
2570        {
2571            self.resources
2572                .ensure_bind_groups(device, *variant, pipeline, format);
2573        }
2574        self.resources
2575            .ensure_page_configs(device, self.scratch.page_rounds());
2576
2577        frame
2578    }
2579
2580    /// The pipelines `state` draws `classes` with — each class folded to what
2581    /// the state serves — resolved through this renderer's memo first and the
2582    /// pipeline cache (warm-up, steal, or an inline compile on a class's first
2583    /// use) only on a miss.
2584    fn class_pipelines(
2585        &mut self,
2586        device: &wgpu::Device,
2587        format: wgpu::TextureFormat,
2588        state: EnginePipeline,
2589        classes: ClassSet,
2590    ) -> ClassPipelines {
2591        let mut out = ClassPipelines::default();
2592        for class in classes.iter() {
2593            let class = state.fold_class(class);
2594            if out.get(class).is_some() {
2595                continue;
2596            }
2597            let slot = ResolvedPipelines::slot(state, class);
2598            let pipeline = match self.resolved.entries.get(slot).and_then(Option::as_ref) {
2599                Some(pipeline) => pipeline.clone(),
2600                None => {
2601                    let pipeline = self
2602                        .pipelines
2603                        .get_or_create(device, &state.class_desc(&self.shaders, format, class))
2604                        .clone();
2605                    if let Some(entry) = self.resolved.entries.get_mut(slot) {
2606                        *entry = Some(pipeline.clone());
2607                    }
2608                    pipeline
2609                }
2610            };
2611            out.set(class, pipeline);
2612        }
2613        out
2614    }
2615
2616    /// Records the frame's surface opening, every round's own pass, and the
2617    /// hole punch — the pass list the module doc's *The frame's passes*
2618    /// states — logging each pass it begins into [`Self::frame_passes`].
2619    ///
2620    /// Every pass opened here is ended before the method returns, which is the
2621    /// half of the encode contract a caller cannot check for itself.
2622    ///
2623    /// Each pass names the [`EngineSpan`] it is charged to: the frame's own
2624    /// surface passes are [`EngineSpan::Main`], a layer page round and a
2625    /// filter pass are [`EngineSpan::Composite`]. Several passes per span is
2626    /// the ordinary case and they sum. A pass takes its query pair only when
2627    /// it is really begun, so no pair is ever taken for a pass that was not
2628    /// recorded.
2629    #[expect(
2630        clippy::too_many_arguments,
2631        reason = "one frame's full recording state, each piece owned by a \
2632                  different part of the renderer; bundling them would move the \
2633                  same assembly one call up"
2634    )]
2635    fn record_frame(
2636        &mut self,
2637        device: &wgpu::Device,
2638        queue: &wgpu::Queue,
2639        encoder: &mut wgpu::CommandEncoder,
2640        mut stage: Option<&mut StagingFrame>,
2641        target: &EngineTarget<'_>,
2642        depth_view: Option<&wgpu::TextureView>,
2643        base_color: Color,
2644        pipelines: &FramePipelines,
2645        copy_mode: RegionCopyMode,
2646        timestamps: FrameTimestamps<'_>,
2647    ) {
2648        let merge = self.pass_merge;
2649        let clear = clear_color(base_color, target.output);
2650        let depth_load = self.depth.load_op(target.depth.is_some());
2651
2652        // Under the kill switch the clear is a pass of its own, recorded first
2653        // and drawing nothing, with the opaque strips in a pass of their own
2654        // after it. It is timed alongside the frame's other surface
2655        // passes even though a backend that samples its counters at the
2656        // vertex/fragment stage boundaries may write nothing for it — an
2657        // untimed pass is a measurement gap, not a wrong measurement (see
2658        // `frust_gpu::diag`).
2659        if !merge {
2660            drop(encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
2661                label: Some(FramePass::Clear.label()),
2662                color_attachments: &[Some(color_attachment(
2663                    target.view,
2664                    wgpu::LoadOp::Clear(clear),
2665                ))],
2666                depth_stencil_attachment: depth_view.map(|view| depth_attachment(view, depth_load)),
2667                timestamp_writes: timestamps.writes(EngineSpan::Main),
2668                occlusion_query_set: None,
2669                multiview_mask: None,
2670            }));
2671            self.frame_passes.push(FramePass::Clear);
2672        }
2673
2674        // Every field below is reached through `self.<field>` rather than
2675        // through a method: the page pool is borrowed mutably for the whole
2676        // walk while the instance buffer, the bind groups and the plan are
2677        // borrowed immutably, and only disjoint field borrows let those
2678        // coexist.
2679        let Some(instances) = self.resources.instances.as_ref() else {
2680            // No instance has ever been uploaded, so the frame draws nothing
2681            // at all — and still resolves to a clean surface: the opening with
2682            // nothing to draw is exactly the clear.
2683            if merge {
2684                SurfaceOpening {
2685                    view: target.view,
2686                    depth: depth_view,
2687                    clear,
2688                    depth_load,
2689                    opaque: None,
2690                }
2691                .record(
2692                    encoder,
2693                    None,
2694                    timestamps.writes(EngineSpan::Main),
2695                    &mut self.frame_passes,
2696                );
2697            }
2698            #[cfg(debug_assertions)]
2699            self.check_pass_list(merge, false, false, false);
2700            return;
2701        };
2702        let dim = self.caps.resource_texture_dim;
2703        let opaque_count = self.scratch.opaque.len() as u32;
2704        // The alpha region starts where the opaque one ends, so every segment's
2705        // own index is relative to that.
2706        let base = opaque_count;
2707
2708        // The opaque strips, once for the whole frame and ahead of every
2709        // surface draw: the depth they write is what every blended instance
2710        // the frame draws onto its own target, composites included, is then
2711        // tested against.
2712        //
2713        // Once rather than ahead of each surface round, because a frame can
2714        // take several of those (the scheduler cuts one short to hand a page
2715        // group back) and a second recording would re-draw opaque coverage at
2716        // equal stored depth over composites the round before it had already
2717        // blended. Ahead of the layer rounds or after them makes no difference
2718        // to those: a layer round writes a pooled page and reads neither this
2719        // target nor the depth attachment.
2720        let opaque = if opaque_count > 0
2721            && let Some(opaque) = pipelines.opaque.as_ref()
2722            && let Some(opaque_groups) =
2723                self.resources.bind_groups.get(&EnginePipeline::StripOpaque)
2724        {
2725            Some(PassPlan {
2726                pass: FramePass::Opaque,
2727                view: target.view,
2728                load: wgpu::LoadOp::Load,
2729                depth: depth_view,
2730                state: EnginePipeline::StripOpaque,
2731                pipelines: opaque,
2732                groups: opaque_groups,
2733                resources: &opaque_groups.resources,
2734                composites: &[],
2735                // An external paint is never claimed opaque, so the
2736                // depth-writing pass never holds a run.
2737                external_keys: &[],
2738                instances,
2739                base: 0,
2740                // The whole opaque region, `(0, opaque_count)`, cut into its
2741                // own class runs when the frame was planned.
2742                segments: &self.scratch.opaque_segments,
2743                composite_class: self.routing.composite_class(),
2744            })
2745        } else {
2746            None
2747        };
2748        let opaque_drawn = opaque.is_some();
2749
2750        // The surface opening: the clear, then the opaque strips, then — when
2751        // one reaches the surface before anything else does — the first
2752        // surface round's own strips, all in ONE pass. Owed until the first
2753        // pass that touches the surface takes it: the first surface round
2754        // that draws, or a punch issued ahead of any (which takes it as a
2755        // pass of its own, drawing no round). A page or filter round never
2756        // takes it — neither reads nor writes the surface or its depth — so
2757        // the opening simply waits past them. Whatever is still owed at the
2758        // end of the walk is recorded there, which is how a frame whose
2759        // surface draws nothing still resolves to a clean surface.
2760        //
2761        // Under the kill switch the clear was recorded above and the opaque
2762        // strips are recorded here, as passes of their own, and nothing is
2763        // owed.
2764        let mut opening = if merge {
2765            Some(SurfaceOpening {
2766                view: target.view,
2767                depth: depth_view,
2768                clear,
2769                depth_load,
2770                opaque,
2771            })
2772        } else {
2773            if let Some(opaque) = opaque.as_ref() {
2774                record_pass(
2775                    encoder,
2776                    opaque,
2777                    timestamps.writes(EngineSpan::Main),
2778                    &mut self.frame_passes,
2779                );
2780            }
2781            None
2782        };
2783
2784        // The destination-out pass's own recording state, resolved once for the
2785        // frame: a punch can now land at any of the walk's cuts, and every one
2786        // of them erases the same target through the same pipeline.
2787        let punch_pass = pipelines.punch.as_ref().and_then(|(variant, punch)| {
2788            Some(PunchPass {
2789                view: target.view,
2790                depth: depth_view,
2791                state: *variant,
2792                pipelines: punch,
2793                class: self.routing.punch_class(),
2794                groups: self.resources.bind_groups.get(variant)?,
2795                instances,
2796                base,
2797            })
2798        });
2799        let punch_drawn = punch_pass.is_some();
2800
2801        // The frame's live pages — the two ping-pong groups and the one spill
2802        // page beside them — each holding the finished page a later round
2803        // composites (see [`crate::schedule`]).
2804        let mut live: [Option<PooledTexture>; MAX_LIVE_PAGES] = [const { None }; MAX_LIVE_PAGES];
2805        let mut page_slot = 0_usize;
2806        // The fragment region copies' own instance slots, one per copy round.
2807        let mut copy_slot = 0_u32;
2808
2809        for plan in &self.scratch.rounds {
2810            let own = match plan.page {
2811                None => None,
2812                // A round continuing a page an earlier round of the same layer
2813                // opened takes that very texture back out of its group: a fresh
2814                // one from the pool would hold the previous holder's pixels
2815                // instead of the half already drawn.
2816                Some(page) if page.continued => match live[page.parity.index()].take() {
2817                    Some(pooled) => Some((page.parity, pooled)),
2818                    // Unreachable: the round that opened the page put it in
2819                    // this group, and no round between the two releases it.
2820                    None => continue,
2821                },
2822                Some(page) => match self.targets.acquire(
2823                    device,
2824                    page.size.width,
2825                    page.size.height,
2826                    PAGE_LABEL,
2827                ) {
2828                    IntermediateTexture::Texture(pooled) => Some((page.parity, pooled)),
2829                    // Unreachable: every page extent was checked against this
2830                    // pool's own ceiling before the first pass was recorded.
2831                    // Skipping the round draws less rather than taking a device
2832                    // error mid-frame.
2833                    IntermediateTexture::TooLarge { .. } => continue,
2834                },
2835            };
2836
2837            // A region-copy round seeds its page with a region of the frame's
2838            // own target, read back after everything painted before it in
2839            // painter order — so the opening, which clears the target and
2840            // draws the opaque strips, is recorded first if it is still owed
2841            // (the scheduler already cut the surface round ahead of it).
2842            if let Some(source) = plan.copy {
2843                if let Some((_, pooled)) = own.as_ref() {
2844                    record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
2845                    // The staging texture's group is stable until the texture
2846                    // grows, so a texture-mode copy builds it once.
2847                    if copy_mode == RegionCopyMode::Texture
2848                        && let (Some(copies), Some(pipeline)) =
2849                            (self.region_copies.as_mut(), pipelines.copy.as_ref())
2850                        && copies.staging_group.is_none()
2851                        && let Some(staging) = copies.staging.as_ref()
2852                    {
2853                        let group = copy_source_group(device, pipeline, &staging.view);
2854                        copies.staging_group = Some(group);
2855                        self.region_copy_groups = self.region_copy_groups.saturating_add(1);
2856                    }
2857                    let built = RegionCopyPass {
2858                        mode: copy_mode,
2859                        target: target.view,
2860                        page: pooled,
2861                        source,
2862                        pipeline: pipelines.copy.as_ref(),
2863                        resources: self.region_copies.as_ref(),
2864                        slot: copy_slot,
2865                    }
2866                    .record(
2867                        device,
2868                        queue,
2869                        encoder,
2870                        timestamps.writes(EngineSpan::Composite),
2871                    );
2872                    self.region_copy_groups = self.region_copy_groups.saturating_add(built);
2873                    copy_slot = copy_slot.saturating_add(1);
2874                    self.frame_passes.push(FramePass::Copy);
2875                }
2876                settle_pages(&mut self.targets, &mut live, plan.released, own);
2877                continue;
2878            }
2879
2880            // A filter round is only a filter pass: no strip instance, no
2881            // composite, no viewport uniform of its own — the pass maps NDC
2882            // against the destination extent its own instance carries, which is
2883            // why that instance is written here, where the pool's quantized
2884            // extent is finally known.
2885            if let Some(filter) = plan.filter {
2886                if let Some((_, pooled)) = own.as_ref()
2887                    && let Some(pipeline) = pipelines.filter.as_ref()
2888                    && let Some(filters) = self.filters.as_ref()
2889                {
2890                    let (width, height) = pooled.size();
2891                    filters.write_instance(
2892                        queue,
2893                        filter.instance,
2894                        &FilterInstanceData::new(
2895                            &filter.step,
2896                            filter.data_offset,
2897                            // Both pages hold the layer at their own origin, so
2898                            // neither region is offset within its page.
2899                            (0, 0),
2900                            (0, 0),
2901                            SizeU16::from_wh(
2902                                u16::try_from(width).unwrap_or(u16::MAX),
2903                                u16::try_from(height).unwrap_or(u16::MAX),
2904                            ),
2905                            filter.original,
2906                        ),
2907                    );
2908                    // A group with no live page samples the transparent
2909                    // placeholder, which filters nothing — the same "draw less,
2910                    // never wrong" answer an unresolvable paint gets.
2911                    // Unreachable: the round that wrote this pass's source is
2912                    // the one before it, and nothing between the two releases
2913                    // that group.
2914                    let (source, key) = match live[filter.source.index()].as_ref() {
2915                        Some(pooled) => (pooled.view(), pooled_source(pooled)),
2916                        None => (
2917                            &self.resources.placeholders.layer_input,
2918                            GroupSource::Placeholder,
2919                        ),
2920                    };
2921                    let source_group = self.resources.source_groups.get_or_build(
2922                        SourceGroupKey {
2923                            kind: GroupKind::FilterSource,
2924                            source: key,
2925                        },
2926                        || filters.source_group(device, pipeline, source),
2927                    );
2928                    filters.record_pass_with(
2929                        device,
2930                        encoder,
2931                        &FilterPassPlan {
2932                            label: FILTER_LABEL,
2933                            pipeline,
2934                            dest: pooled.view(),
2935                            source,
2936                            instance: filter.instance,
2937                        },
2938                        timestamps.writes(EngineSpan::Composite),
2939                        Some(&source_group),
2940                    );
2941                    self.frame_passes.push(FramePass::Filter);
2942                }
2943
2944                settle_pages(&mut self.targets, &mut live, plan.released, own);
2945                continue;
2946            }
2947
2948            // The off-screen texture this round draws into, when it is not the
2949            // surface: the pooled page acquired above, or the retained page the
2950            // store bound to this round's body (`crate::cache::retained`). A
2951            // retained round whose page is missing — unreachable, the store
2952            // committed one for every body a round renders — draws nothing
2953            // rather than drawing onto the surface.
2954            let retained_page = plan
2955                .retained
2956                .and_then(|retained| self.retained.page_for(retained.bracket));
2957            if plan.retained.is_some() && retained_page.is_none() {
2958                settle_pages(&mut self.targets, &mut live, plan.released, own);
2959                continue;
2960            }
2961            let offscreen = own
2962                .as_ref()
2963                .map(|(_, pooled)| (pooled.view(), pooled.size()))
2964                .or(retained_page);
2965            let continued = plan.page.is_some_and(|page| page.continued)
2966                || plan.retained.is_some_and(|retained| retained.continued);
2967
2968            // The round's viewport uniform. A page's is written here rather
2969            // than with the frame's other uploads because only the pool knows
2970            // the extent it quantized the request up to, and NDC is computed
2971            // against the attachment's real extent. Distinct buffers, so the
2972            // write ordering against the frame's own config never matters.
2973            // Staged, its copy lands in the encoder here — between passes,
2974            // immediately ahead of the one round that reads it.
2975
2976            // The page-config buffer this round binds; the surface's own
2977            // config has no slot and is named by the maximum.
2978            let config_slot = offscreen.map_or(u32::MAX, |_| {
2979                u32::try_from(page_slot).unwrap_or(u32::MAX - 1)
2980            });
2981            let config = match offscreen {
2982                None => &self.resources.config,
2983                Some((_, (width, height))) => {
2984                    let Some(config) = self.resources.page_configs.get(page_slot) else {
2985                        continue;
2986                    };
2987                    let bytes = GpuConfig::new(width, height, dim, dim);
2988                    let bytes = bytemuck::bytes_of(&bytes);
2989                    let staged = stage
2990                        .as_deref_mut()
2991                        .is_some_and(|stage| stage.copy_to_buffer(encoder, bytes, config, 0));
2992                    if !staged {
2993                        queue.write_buffer(config, 0, bytes);
2994                    }
2995                    page_slot = page_slot.saturating_add(1);
2996                    config
2997                }
2998            };
2999
3000            let variant = match offscreen {
3001                None => pipelines.alpha.0,
3002                Some(_) => EnginePipeline::StripIntermediate,
3003            };
3004            let class_pipelines = match (offscreen, pipelines.page.as_ref()) {
3005                (None, _) => &pipelines.alpha.1,
3006                (Some(_), Some(page)) => page,
3007                (Some(_), None) => continue,
3008            };
3009            // The pipeline group 0 is built against: any of the pass's class
3010            // pipelines will do, since every strip pipeline shares one layout.
3011            let Some(pipeline) = class_pipelines.any() else {
3012                continue;
3013            };
3014            let Some(groups) = self.resources.bind_groups.get(&variant) else {
3015                continue;
3016            };
3017
3018            // A page round binds its own viewport uniform; the root round's is
3019            // already the one the shared set carries. Cached by the config
3020            // slot: the buffers are grown-only, so the group stays valid until
3021            // the coverage view it also names is recreated.
3022            let page_resources = offscreen.map(|_| {
3023                self.resources.source_groups.get_or_build(
3024                    SourceGroupKey {
3025                        kind: GroupKind::Resources { slot: config_slot },
3026                        source: GroupSource::Placeholder,
3027                    },
3028                    || {
3029                        resources_bind_group(
3030                            device,
3031                            pipeline,
3032                            &self.resources.alphas.view,
3033                            config,
3034                            &self.resources.placeholders.layer_input,
3035                        )
3036                    },
3037                )
3038            });
3039            let resources = page_resources.as_ref().unwrap_or(&groups.resources);
3040
3041            let segments = self
3042                .scratch
3043                .segments
3044                .get(plan.segments.clone())
3045                .unwrap_or(&[]);
3046            let mut composites = std::mem::take(&mut self.composite_groups);
3047            composites.clear();
3048            for segment in segments {
3049                // A backdrop's draws sample the page holding its blurred
3050                // region through the same group-0 layer input a composite
3051                // reads, so its group is built here, in segment order, beside
3052                // the composites' own.
3053                if let Segment::Backdrop(_, _, parity) = *segment {
3054                    let (view, source) = match live[parity.index()].as_ref() {
3055                        Some(pooled) => (pooled.view(), pooled_source(pooled)),
3056                        None => (
3057                            &self.resources.placeholders.layer_input,
3058                            GroupSource::Placeholder,
3059                        ),
3060                    };
3061                    composites.push(self.resources.source_groups.get_or_build(
3062                        SourceGroupKey {
3063                            kind: GroupKind::Resources { slot: config_slot },
3064                            source,
3065                        },
3066                        || {
3067                            resources_bind_group(
3068                                device,
3069                                pipeline,
3070                                &self.resources.alphas.view,
3071                                config,
3072                                view,
3073                            )
3074                        },
3075                    ));
3076                }
3077                if let Segment::Composite(_, source) = *segment {
3078                    // A group with no live page — or a retained bracket with no
3079                    // page bound — samples the transparent placeholder, which
3080                    // composites nothing: the same "draw less, never wrong"
3081                    // answer an unresolvable paint gets.
3082                    let placeholder = (
3083                        &self.resources.placeholders.layer_input,
3084                        GroupSource::Placeholder,
3085                    );
3086                    let (view, source) = match source {
3087                        CompositeSource::Page(parity) => live[parity.index()]
3088                            .as_ref()
3089                            .map_or(placeholder, |pooled| (pooled.view(), pooled_source(pooled))),
3090                        CompositeSource::Retained(bracket) => self
3091                            .retained
3092                            .page_for(bracket)
3093                            .map_or(placeholder, |(view, size)| {
3094                                (
3095                                    view,
3096                                    GroupSource::Retained {
3097                                        view: std::ptr::from_ref(view) as usize,
3098                                        generation: self.retained.generation(),
3099                                        size,
3100                                    },
3101                                )
3102                            }),
3103                    };
3104                    composites.push(self.resources.source_groups.get_or_build(
3105                        SourceGroupKey {
3106                            kind: GroupKind::Resources { slot: config_slot },
3107                            source,
3108                        },
3109                        || {
3110                            resources_bind_group(
3111                                device,
3112                                pipeline,
3113                                &self.resources.alphas.view,
3114                                config,
3115                                view,
3116                            )
3117                        },
3118                    ));
3119                }
3120            }
3121
3122            // A page round draws into an off-screen layer page, the frame's own
3123            // rounds into the surface — the split the two spans name.
3124            let span = if offscreen.is_some() {
3125                EngineSpan::Composite
3126            } else {
3127                EngineSpan::Main
3128            };
3129
3130            let (view, pass, load, depth) = match offscreen {
3131                Some((view, _)) => (
3132                    view,
3133                    FramePass::Page,
3134                    // A pooled texture holds whatever its last holder left
3135                    // there, and a retained one whatever this body's last frame
3136                    // left, so a layer's first round into a page clears it —
3137                    // and a round continuing that same page loads it, because
3138                    // clearing again would wipe the half already drawn.
3139                    if continued {
3140                        wgpu::LoadOp::Load
3141                    } else {
3142                        wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT)
3143                    },
3144                    None,
3145                ),
3146                None => (
3147                    target.view,
3148                    FramePass::Surface,
3149                    wgpu::LoadOp::Load,
3150                    depth_view,
3151                ),
3152            };
3153
3154            // A surface plan with nothing of its own to draw records no pass:
3155            // loading and storing the target unchanged is exactly nothing. A
3156            // punch cut falling at the very start of a round — a `ClearRect`
3157            // recorded before anything the round draws — is how one arises, and
3158            // so is a surface whose every span went to the opaque strips; the
3159            // opening stays owed past it either way. A *page* plan with no
3160            // segments still records its pass, because that is what clears the
3161            // pooled page.
3162            if !segments.is_empty() || offscreen.is_some() {
3163                let round = PassPlan {
3164                    pass,
3165                    view,
3166                    load,
3167                    depth,
3168                    state: variant,
3169                    pipelines: class_pipelines,
3170                    groups,
3171                    resources,
3172                    composites: &composites,
3173                    external_keys: self.resources.external_runs.keys(),
3174                    instances,
3175                    base,
3176                    segments,
3177                    composite_class: self.routing.composite_class(),
3178                };
3179                // The first surface round that draws takes the opening: one
3180                // pass clears, draws the opaque strips, then this round's own
3181                // segments in order — the pipeline switching between them
3182                // inside the pass — and ends where this round's pass would
3183                // have ended.
3184                let opening = if offscreen.is_none() {
3185                    opening.take()
3186                } else {
3187                    None
3188                };
3189                match opening {
3190                    Some(opening) => opening.record(
3191                        encoder,
3192                        Some(&round),
3193                        timestamps.writes(span),
3194                        &mut self.frame_passes,
3195                    ),
3196                    None => record_pass(
3197                        encoder,
3198                        &round,
3199                        timestamps.writes(span),
3200                        &mut self.frame_passes,
3201                    ),
3202                }
3203            }
3204
3205            // Handed back empty so the buffer is reused and no cached group is
3206            // held past the round.
3207            composites.clear();
3208            self.composite_groups = composites;
3209
3210            settle_pages(&mut self.targets, &mut live, plan.released, own);
3211
3212            // The cut this plan ends at, erased once the ops before it have
3213            // been recorded and before the plan after it draws a thing. Inert
3214            // on every plan the walk did not cut, which is every plan of a
3215            // frame that punches nothing. A punch issued before the opening
3216            // was taken has to erase a cleared surface, so the opening is
3217            // recorded first, as a pass of its own.
3218            if let Some(punch) = punch_pass.as_ref()
3219                && plan.punch.1 > 0
3220            {
3221                record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
3222                punch.record(encoder, plan.punch, timestamps, &mut self.frame_passes);
3223            }
3224        }
3225
3226        // An opening no surface round took — a frame whose surface draws
3227        // nothing of its own past the opaque strips, or nothing at all — is
3228        // recorded here, ahead of the trailing punch it must precede.
3229        record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
3230
3231        // The punches past every op of the frame, in the position the pass held
3232        // unconditionally before painter order was restored.
3233        if let Some(punch) = punch_pass.as_ref() {
3234            punch.record(
3235                encoder,
3236                self.scratch.punch,
3237                timestamps,
3238                &mut self.frame_passes,
3239            );
3240        }
3241
3242        for page in live.into_iter().flatten() {
3243            self.targets.release(page);
3244        }
3245
3246        #[cfg(debug_assertions)]
3247        self.check_pass_list(merge, true, opaque_drawn, punch_drawn);
3248        #[cfg(not(debug_assertions))]
3249        let _ = (opaque_drawn, punch_drawn);
3250    }
3251
3252    /// Checks that the passes the walk just recorded are exactly the ones
3253    /// [`Scratch::plan_passes`] says a frame of this shape records — the
3254    /// host-testable statement of the pass list, held to the recording on
3255    /// every frame a debug build encodes.
3256    #[cfg(debug_assertions)]
3257    fn check_pass_list(&self, merge: bool, instances: bool, opaque: bool, punch: bool) {
3258        let mut planned = Vec::new();
3259        self.scratch.plan_passes(
3260            &PassShape {
3261                merge,
3262                instances,
3263                opaque,
3264                punch,
3265            },
3266            &mut planned,
3267        );
3268        debug_assert_eq!(
3269            planned, self.frame_passes,
3270            "the recorded pass list disagrees with the planned one"
3271        );
3272    }
3273}
3274
3275/// The cache key naming `pooled`'s texture as a group's layer input.
3276fn pooled_source(pooled: &PooledTexture) -> GroupSource {
3277    GroupSource::Pooled(pooled.texture().as_scene_texture())
3278}
3279
3280/// One render pass a frame recorded into the caller's encoder — what
3281/// [`EngineRenderer::last_frame_passes`] lists, in recording order.
3282///
3283/// The module doc's *The frame's passes* is the contract these name. The
3284/// glyph-atlas replay's passes are not among them: they are recorded on an
3285/// encoder of the engine's own, ahead of the frame (see [`crate::gpu::atlas`]).
3286#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
3287pub enum FramePass {
3288    /// The surface opening: clears the colour target (and the depth
3289    /// attachment, unless pre-cleared), then draws the opaque strips and the
3290    /// first surface round that draws, whichever of the two the frame has.
3291    /// The first pass over the surface whenever passes are merged.
3292    Opening,
3293    /// The clear alone, drawing nothing — the first pass of a frame recorded
3294    /// with passes unmerged.
3295    Clear,
3296    /// The opaque strips alone, depth-writing, loading what the clear left —
3297    /// recorded only with passes unmerged.
3298    Opaque,
3299    /// A surface round's blended strips, loading the surface.
3300    Surface,
3301    /// A layer page round, rendering into a pooled page.
3302    Page,
3303    /// One pass of a filter's sequence, between two pooled pages.
3304    Filter,
3305    /// A region copy: a region of the frame's own target read back into a
3306    /// pooled page by one pass that clears the page and draws the region into
3307    /// it — preceded, on a target that can only be copied, by a transfer into
3308    /// the renderer's staging texture (see [`RegionCopyMode`]) — ahead of a
3309    /// backdrop blur's filter passes.
3310    Copy,
3311    /// The destination-out hole punch, at a cut or after the last round.
3312    Punch,
3313}
3314
3315impl FramePass {
3316    /// The label the pass is recorded under, as a GPU capture shows it.
3317    #[must_use]
3318    pub const fn label(self) -> &'static str {
3319        match self {
3320            FramePass::Opening => "frust-engine surface opening",
3321            FramePass::Clear => "frust-engine clear",
3322            FramePass::Opaque => "frust-engine opaque strips",
3323            FramePass::Surface => "frust-engine alpha strips",
3324            FramePass::Page => PAGE_LABEL,
3325            FramePass::Filter => FILTER_LABEL,
3326            FramePass::Copy => "frust-engine region copy",
3327            FramePass::Punch => "frust-engine hole punch",
3328        }
3329    }
3330
3331    /// Whether the pass renders into the frame's own target rather than into
3332    /// a pooled page.
3333    #[must_use]
3334    pub const fn targets_surface(self) -> bool {
3335        !matches!(self, FramePass::Page | FramePass::Filter | FramePass::Copy)
3336    }
3337}
3338
3339/// What a frame's pass list depends on beyond its plan, as
3340/// [`Scratch::plan_passes`] reads it.
3341#[cfg(any(test, debug_assertions))]
3342#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3343struct PassShape {
3344    /// Whether the surface opening is merged (the renderer's
3345    /// [`EngineRenderer::pass_merge`]).
3346    merge: bool,
3347    /// Whether the frame has an instance buffer to draw from at all.
3348    instances: bool,
3349    /// Whether the frame draws opaque strips — depth in play, opaque spans to
3350    /// route, and their pipelines resolved.
3351    opaque: bool,
3352    /// Whether the frame records its hole punches.
3353    punch: bool,
3354}
3355
3356/// The pipelines one frame's passes are recorded with, resolved once before the
3357/// first `begin_render_pass`.
3358///
3359/// Only `alpha` is unconditional: the rest exist exactly when the frame has
3360/// work for them, so a plain frame compiles and binds nothing it will not draw.
3361#[derive(Debug)]
3362struct FramePipelines {
3363    /// The blended pass over the frame's own target, with its variant — which
3364    /// of the two it is depends on whether depth is in play.
3365    alpha: (EnginePipeline, ClassPipelines),
3366    /// The depth-writing pass, when depth is available and the frame has
3367    /// fully covered spans to route into it.
3368    opaque: Option<ClassPipelines>,
3369    /// The pass a layer page is rendered through, when the frame has one.
3370    page: Option<ClassPipelines>,
3371    /// The destination-out pass, with its variant, when the frame punches.
3372    punch: Option<(EnginePipeline, ClassPipelines)>,
3373    /// The pass one filter round runs through, when the frame filters a layer.
3374    filter: Option<wgpu::RenderPipeline>,
3375    /// The pass a fragment region copy runs through, when the frame copies a
3376    /// region of a target it can only sample ([`RegionCopyMode::Fragment`]).
3377    copy: Option<wgpu::RenderPipeline>,
3378}
3379
3380/// How a renderer cuts its passes into paint-class runs.
3381///
3382/// Settled from [`crate::config`] when the renderer is built
3383/// (`FRUST_ENGINE_UBER_PROGRAM`, `FRUST_ENGINE_RUN_SPLIT_MIN`) and overridable
3384/// per renderer, so a caller can compare the two arms in one process.
3385#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3386struct ClassRouting {
3387    /// The kill switch: every run is drawn with [`PaintClass::Uber`] and no
3388    /// segment is split.
3389    uber: bool,
3390    /// The shortest run worth a pipeline switch of its own.
3391    min_run: u32,
3392}
3393
3394impl Default for ClassRouting {
3395    fn default() -> Self {
3396        Self {
3397            uber: false,
3398            min_run: crate::gpu::strips::RUN_SPLIT_MIN,
3399        }
3400    }
3401}
3402
3403impl ClassRouting {
3404    /// The routing a renderer starts with, from the process configuration.
3405    fn from_config() -> Self {
3406        Self {
3407            uber: config::uber_program_forced(),
3408            min_run: config::run_split_min(),
3409        }
3410    }
3411
3412    /// The class a composite quad is drawn with.
3413    fn composite_class(self) -> PaintClass {
3414        if self.uber {
3415            PaintClass::Uber
3416        } else {
3417            PaintClass::LayerComposite
3418        }
3419    }
3420
3421    /// The class a hole punch is drawn with.
3422    fn punch_class(self) -> PaintClass {
3423        if self.uber {
3424            PaintClass::Uber
3425        } else {
3426            PaintClass::SolidCoverage
3427        }
3428    }
3429}
3430
3431/// One strip state's pipelines, by paint class — the ones a pass resolved for
3432/// the classes its runs name.
3433#[derive(Debug, Default)]
3434struct ClassPipelines {
3435    by_class: [Option<wgpu::RenderPipeline>; PaintClass::ALL.len()],
3436}
3437
3438impl ClassPipelines {
3439    /// The pipeline drawing `class`, if the frame resolved one.
3440    fn get(&self, class: PaintClass) -> Option<&wgpu::RenderPipeline> {
3441        self.by_class.get(class.index()).and_then(Option::as_ref)
3442    }
3443
3444    fn set(&mut self, class: PaintClass, pipeline: wgpu::RenderPipeline) {
3445        if let Some(slot) = self.by_class.get_mut(class.index()) {
3446            *slot = Some(pipeline);
3447        }
3448    }
3449
3450    /// Any one of the pipelines — what a bind group is built against, every
3451    /// strip pipeline sharing one explicit layout.
3452    fn any(&self) -> Option<&wgpu::RenderPipeline> {
3453        self.by_class.iter().flatten().next()
3454    }
3455}
3456
3457/// The strip pipelines a renderer has resolved so far, by `(state, class)`,
3458/// for the target format they were built against.
3459///
3460/// A memo in front of the pipeline cache: a steady-state frame takes every
3461/// pipeline it draws with from here, by indexing a dense `state x class` table,
3462/// rather than rebuilding each description (a vertex-layout allocation) and
3463/// packing its cache key — which keeps the frame's `pipelines` phase from
3464/// growing with the number of classes it draws.
3465#[derive(Debug)]
3466struct ResolvedPipelines {
3467    /// The format every entry was built for; a change drops them all.
3468    format: Option<wgpu::TextureFormat>,
3469    /// Indexed by [`Self::slot`].
3470    entries: [Option<wgpu::RenderPipeline>; RESOLVED_SLOTS],
3471    /// The filter pass's pipeline; its description is pinned to one format, so
3472    /// it is built once per renderer rather than once per filtering frame.
3473    filter: Option<wgpu::RenderPipeline>,
3474    /// The fragment region copy's pipeline, memoised on the same terms.
3475    copy: Option<wgpu::RenderPipeline>,
3476}
3477
3478/// One [`ResolvedPipelines`] entry per render state and paint class.
3479const RESOLVED_SLOTS: usize = EnginePipeline::ALL.len() * PaintClass::ALL.len();
3480
3481impl Default for ResolvedPipelines {
3482    fn default() -> Self {
3483        Self {
3484            format: None,
3485            entries: [const { None }; RESOLVED_SLOTS],
3486            filter: None,
3487            copy: None,
3488        }
3489    }
3490}
3491
3492impl ResolvedPipelines {
3493    /// The table slot of `state` drawing `class`.
3494    const fn slot(state: EnginePipeline, class: PaintClass) -> usize {
3495        state.index() * PaintClass::ALL.len() + class.index()
3496    }
3497}
3498
3499/// One pass's full recording state, assembled before the pass is begun.
3500///
3501/// A struct rather than an argument list because the composite groups have to
3502/// be built (and so borrowed) before `begin_render_pass` takes the encoder, and
3503/// naming them together is what makes that ordering obvious at the call site.
3504///
3505/// The attachment half (`pass`, `view`, `load`, `depth`) describes the pass the
3506/// plan begins when it is recorded on its own; the rest is what it draws, which
3507/// is all the [surface opening](SurfaceOpening) reads of the plans it draws
3508/// inside its own pass.
3509struct PassPlan<'a> {
3510    /// Which of the frame's passes this is — its label, and its entry in
3511    /// [`EngineRenderer::last_frame_passes`].
3512    pass: FramePass,
3513    view: &'a wgpu::TextureView,
3514    load: wgpu::LoadOp<wgpu::Color>,
3515    depth: Option<&'a wgpu::TextureView>,
3516    /// The render state the pass draws with, which folds each run's class to
3517    /// one it serves.
3518    state: EnginePipeline,
3519    /// The state's pipeline per paint class — at least every class the pass's
3520    /// segments name, folded.
3521    pipelines: &'a ClassPipelines,
3522    /// The pipeline variant's own groups 1-3.
3523    groups: &'a StripBindGroups,
3524    /// Group 0 for the pass's ordinary strip segments.
3525    resources: &'a wgpu::BindGroup,
3526    /// Group 0 per composite segment, in the order the segments name them.
3527    composites: &'a [wgpu::BindGroup],
3528    /// The externally bound texture each [`Segment::External`] slot names,
3529    /// indexed by slot — the frame's own
3530    /// [`ExternalRuns::keys`](crate::gpu::bindings::ExternalRuns::keys).
3531    /// Empty for a pass that draws no external texture, which is every pass of
3532    /// every frame that records no `SceneTexture`.
3533    external_keys: &'a [u64],
3534    instances: &'a wgpu::Buffer,
3535    /// The instance index every segment's own index is relative to.
3536    base: u32,
3537    segments: &'a [Segment],
3538    /// The class a [`Segment::Composite`] is drawn with —
3539    /// [`PaintClass::LayerComposite`], or Uber under the kill switch.
3540    composite_class: PaintClass,
3541}
3542
3543/// Records one pass: load the colour target, then draw each segment in order.
3544///
3545/// `timestamps` is the query pair the pass's GPU time is stamped into, `None`
3546/// for an untimed pass — which is every pass of every frame encoded without a
3547/// timestamp sink (see [`crate::diag`]). The pass is logged into `log` as it
3548/// is begun.
3549fn record_pass(
3550    encoder: &mut wgpu::CommandEncoder,
3551    plan: &PassPlan<'_>,
3552    timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3553    log: &mut Vec<FramePass>,
3554) {
3555    let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3556        label: Some(plan.pass.label()),
3557        color_attachments: &[Some(color_attachment(plan.view, plan.load))],
3558        depth_stencil_attachment: plan
3559            .depth
3560            .map(|view| depth_attachment(view, wgpu::LoadOp::Load)),
3561        timestamp_writes: timestamps,
3562        occlusion_query_set: None,
3563        multiview_mask: None,
3564    });
3565    log.push(plan.pass);
3566    draw_segments(&mut pass, plan);
3567}
3568
3569/// Draws `plan`'s segments, in order, into an already begun `pass`.
3570///
3571/// Two groups are re-bound per segment. Group 0, because a composite reads its
3572/// page through it while an ordinary strip reads the placeholder; and group 1,
3573/// because a run of instances sampling an externally bound texture needs that
3574/// texture bound where the atlas placeholder otherwise sits. Groups 2-3 are the
3575/// variant's own and never change within a plan. A plan with one segment —
3576/// every frame that records no layer and no external texture — sets them all
3577/// exactly once.
3578///
3579/// A segment naming a slot with no bind group behind it draws nothing rather
3580/// than drawing with whatever group 1 last held: the texture was unbound
3581/// between the frame's paint resolution and its recording, and a wrongly
3582/// sampled run is worse than a missing one.
3583///
3584/// Each segment is one paint-class run, and the pipeline is set only where
3585/// the class changes from the segment before — so a plan of one class (every
3586/// plan under the Uber kill switch) sets its pipeline exactly once. A segment
3587/// whose class has no resolved pipeline draws nothing, for the same reason; it
3588/// cannot occur, since the frame resolves every class its segments name before
3589/// recording.
3590///
3591/// Everything a plan draws with is set here, its first pipeline included, so
3592/// two plans drawn back to back into one pass — the opening's opaque strips
3593/// and then its round's — each start from their own state rather than from
3594/// whatever the plan before them left bound.
3595fn draw_segments(pass: &mut wgpu::RenderPass<'_>, plan: &PassPlan<'_>) {
3596    pass.set_vertex_buffer(0, plan.instances.slice(..));
3597
3598    let mut composite = 0_usize;
3599    let mut bound: Option<PaintClass> = None;
3600    for segment in plan.segments {
3601        // Claimed before the class lookup, so a composite whose pipeline is
3602        // missing still consumes its own group and the next one pairs up.
3603        let class = plan.state.fold_class(match *segment {
3604            Segment::Strips(_, _, class) | Segment::External(_, _, _, class) => class,
3605            Segment::Composite(..) | Segment::Backdrop(..) => plan.composite_class,
3606        });
3607        let (group, images, range) = match *segment {
3608            Segment::Strips(first, count, _) => {
3609                if count == 0 {
3610                    continue;
3611                }
3612                (
3613                    plan.resources,
3614                    None,
3615                    GpuStrip::instance_range(plan.base.saturating_add(first), count),
3616                )
3617            }
3618            Segment::External(first, count, slot, _) => {
3619                if count == 0 {
3620                    continue;
3621                }
3622                let Some(images) = plan
3623                    .external_keys
3624                    .get(slot as usize)
3625                    .and_then(|key| plan.groups.externals.get(key))
3626                else {
3627                    continue;
3628                };
3629                (
3630                    plan.resources,
3631                    Some(images),
3632                    GpuStrip::instance_range(plan.base.saturating_add(first), count),
3633                )
3634            }
3635            Segment::Composite(first, _) => {
3636                let Some(group) = plan.composites.get(composite) else {
3637                    continue;
3638                };
3639                composite = composite.saturating_add(1);
3640                (
3641                    group,
3642                    None,
3643                    GpuStrip::instance_range(plan.base.saturating_add(first), 1),
3644                )
3645            }
3646            // Its group is claimed before its count is read, like a
3647            // composite's, so the groups after it stay paired.
3648            Segment::Backdrop(first, count, _) => {
3649                let Some(group) = plan.composites.get(composite) else {
3650                    continue;
3651                };
3652                composite = composite.saturating_add(1);
3653                if count == 0 {
3654                    continue;
3655                }
3656                (
3657                    group,
3658                    None,
3659                    GpuStrip::instance_range(plan.base.saturating_add(first), count),
3660                )
3661            }
3662        };
3663        if bound != Some(class) {
3664            let Some(pipeline) = plan.pipelines.get(class) else {
3665                continue;
3666            };
3667            pass.set_pipeline(pipeline);
3668            bound = Some(class);
3669        }
3670        plan.groups.bind_with(pass, group, images);
3671        pass.draw(GpuStrip::vertex_range(), range);
3672    }
3673}
3674
3675/// The pass that opens the frame's own surface: the clear, the opaque strips,
3676/// and the first surface round that draws, in one render pass.
3677///
3678/// Merging is pixel-identical to recording the three apart, because nothing
3679/// between them differs but the pass boundary: the clear becomes the pass's
3680/// `LoadOp::Clear` on colour (and on depth, unless the caller stated its
3681/// attachment is already populated), and the two strip plans draw in the order
3682/// their passes ran, each through its own pipelines and bind groups, against
3683/// the one colour and depth attachment both already shared. What it saves is
3684/// the pass boundary itself: on a tile-based GPU every pass over the surface
3685/// is a whole-framebuffer load into tile memory and a store back out, and the
3686/// clear and the opaque pass were two of those that drew little or nothing.
3687///
3688/// Recorded exactly once per frame, by whichever comes first of the first
3689/// surface round that draws (which it then carries) and the first punch
3690/// (ahead of which it is recorded alone) — or at the end of the walk when
3691/// neither came. See [`EngineRenderer::set_pass_merge`] for the switch that
3692/// records the three apart again.
3693struct SurfaceOpening<'a> {
3694    view: &'a wgpu::TextureView,
3695    depth: Option<&'a wgpu::TextureView>,
3696    /// The frame's base colour, already in the target's alpha convention.
3697    clear: wgpu::Color,
3698    /// The depth clear, or a load when the caller's attachment is pre-cleared
3699    /// (see [`crate::gpu::depth`]).
3700    depth_load: wgpu::LoadOp<f32>,
3701    /// The opaque strips, when the frame has any to draw.
3702    opaque: Option<PassPlan<'a>>,
3703}
3704
3705impl SurfaceOpening<'_> {
3706    /// Records the opening, drawing `round`'s segments after the opaque strips
3707    /// when a surface round is folded into it.
3708    fn record(
3709        &self,
3710        encoder: &mut wgpu::CommandEncoder,
3711        round: Option<&PassPlan<'_>>,
3712        timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3713        log: &mut Vec<FramePass>,
3714    ) {
3715        let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3716            label: Some(FramePass::Opening.label()),
3717            color_attachments: &[Some(color_attachment(
3718                self.view,
3719                wgpu::LoadOp::Clear(self.clear),
3720            ))],
3721            depth_stencil_attachment: self
3722                .depth
3723                .map(|view| depth_attachment(view, self.depth_load)),
3724            timestamp_writes: timestamps,
3725            occlusion_query_set: None,
3726            multiview_mask: None,
3727        });
3728        log.push(FramePass::Opening);
3729        if let Some(opaque) = self.opaque.as_ref() {
3730            draw_segments(&mut pass, opaque);
3731        }
3732        if let Some(round) = round {
3733            draw_segments(&mut pass, round);
3734        }
3735    }
3736}
3737
3738/// Records the surface opening as a pass of its own if it is still owed, and
3739/// marks it taken.
3740fn record_owed_opening(
3741    opening: &mut Option<SurfaceOpening<'_>>,
3742    encoder: &mut wgpu::CommandEncoder,
3743    timestamps: FrameTimestamps<'_>,
3744    log: &mut Vec<FramePass>,
3745) {
3746    if let Some(opening) = opening.take() {
3747        opening.record(encoder, None, timestamps.writes(EngineSpan::Main), log);
3748    }
3749}
3750
3751/// Everything the frame's hole-punch passes are recorded with, resolved once
3752/// before the round walk begins.
3753///
3754/// A frame can now record several: the punches are issued at the painter-order
3755/// positions they were hoisted from, so a surface round carrying two of them is
3756/// cut twice and each cut erases through this same pipeline and these same
3757/// groups (see [`crate::compile::clear`]). Resolving them once is what keeps
3758/// that from becoming a per-cut lookup, and holding the borrows in one value is
3759/// what keeps them out of the page pool's way — the walk holds that mutably
3760/// throughout.
3761struct PunchPass<'a> {
3762    view: &'a wgpu::TextureView,
3763    depth: Option<&'a wgpu::TextureView>,
3764    /// Which destination-out state the punch draws with.
3765    state: EnginePipeline,
3766    /// That state's pipelines, holding at least [`Self::class`].
3767    pipelines: &'a ClassPipelines,
3768    /// The class every punch run is drawn with: the solid program, or Uber
3769    /// under the kill switch.
3770    class: PaintClass,
3771    /// The destination-out variant's own groups 1-3.
3772    groups: &'a StripBindGroups,
3773    instances: &'a wgpu::Buffer,
3774    /// The instance index the punch's own `(first, count)` is relative to — the
3775    /// alpha region's start, the same one every round's segments use.
3776    base: u32,
3777}
3778
3779impl PunchPass<'_> {
3780    /// Records `punch`'s instances as one destination-out pass, or nothing at
3781    /// all when the cut issued none — which is every cut of every frame that
3782    /// records no `ClearRect`. A pass that is not recorded takes no query pair.
3783    fn record(
3784        &self,
3785        encoder: &mut wgpu::CommandEncoder,
3786        punch: (u32, u32),
3787        timestamps: FrameTimestamps<'_>,
3788        log: &mut Vec<FramePass>,
3789    ) {
3790        let (first, count) = punch;
3791        if count == 0 {
3792            return;
3793        }
3794        record_pass(
3795            encoder,
3796            &PassPlan {
3797                pass: FramePass::Punch,
3798                view: self.view,
3799                load: wgpu::LoadOp::Load,
3800                depth: self.depth,
3801                state: self.state,
3802                pipelines: self.pipelines,
3803                groups: self.groups,
3804                resources: &self.groups.resources,
3805                composites: &[],
3806                // A punch erases with a solid source; it samples nothing.
3807                external_keys: &[],
3808                instances: self.instances,
3809                base: self.base,
3810                // Every punch instance is the same solid source, so the run
3811                // is one class whole — no split to plan.
3812                segments: &[Segment::Strips(first, count, self.class)],
3813                composite_class: self.class,
3814            },
3815            timestamps.writes(EngineSpan::Main),
3816            log,
3817        );
3818    }
3819}
3820
3821/// Hands back the page groups a finished round consumed, and parks the page it
3822/// wrote in its own group.
3823///
3824/// The same bookkeeping after every round, strip and filter alike: a page is
3825/// free the moment the pass that sampled it ends, which is what bounds a chain
3826/// of any depth — and a filter layer's own pair of pages — to the two ping-pong
3827/// groups, and every shape this scheduler serves to
3828/// [`MAX_LIVE_PAGES`](crate::schedule::MAX_LIVE_PAGES) live intermediates.
3829///
3830/// A round that is not continuing a page of its own takes a *fresh* texture out
3831/// of the pool rather than the group's current occupant, and the occupant it
3832/// displaces goes back here. That is what keeps a filter pass from ever holding
3833/// one texture as both its attachment and its source: a filter round never
3834/// continues a page — it clears — so what it writes is always a different
3835/// texture from the one the pass before it wrote and this pass reads.
3836fn settle_pages(
3837    targets: &mut IntermediateTargets,
3838    live: &mut [Option<PooledTexture>; MAX_LIVE_PAGES],
3839    released: [bool; MAX_LIVE_PAGES],
3840    own: Option<(PageParity, PooledTexture)>,
3841) {
3842    for (index, slot) in live.iter_mut().enumerate() {
3843        if released.get(index).copied().unwrap_or(false)
3844            && let Some(page) = slot.take()
3845        {
3846            targets.release(page);
3847        }
3848    }
3849    if let Some((parity, pooled)) = own
3850        && let Some(previous) = live[parity.index()].replace(pooled)
3851    {
3852        targets.release(previous);
3853    }
3854}
3855
3856/// The GPU resources a frame's [filter](crate::filters) rounds are executed
3857/// with, beyond the two pooled pages they ping-pong between.
3858///
3859/// Three of them, and each is the engine's only one of its kind: the
3860/// filter-data texture holding every filter in the frame's 48-byte parameter
3861/// block, the bilinear sampler
3862/// ([`filter_sampler`](crate::gpu::targets::filter_sampler)) the blur kernels
3863/// read their source page through, and the instance buffer one quad per pass is
3864/// drawn from.
3865///
3866/// Public because this *is* executing a filter pass — the renderer holds one
3867/// and drives it over the pages the scheduler named, and `tests/filters.rs`
3868/// drives the same type over pages of its own on real hardware. That second
3869/// caller is not a convenience: `frust_scene` carries no *layer*-filter
3870/// command, so a filter layer cannot reach [`EngineRenderer::encode`] through a
3871/// `Scene` at all, and driving this type directly is how the drop shadow's
3872/// WGSL is exercised on a device. The blur's is also reached through `encode`
3873/// itself, by a backdrop blur's pass rounds.
3874#[derive(Debug)]
3875pub struct FilterResources {
3876    /// Every filter in the frame's parameter block, back to back.
3877    data: ResourceTexture,
3878    /// Group 0, naming [`Self::data`]'s view. Rebuilt whenever that texture is,
3879    /// and valid for the renderer's whole life otherwise: a filter pipeline's
3880    /// description does not depend on the frame's target format, so there is
3881    /// only ever one layout to have derived it from.
3882    data_group: Option<wgpu::BindGroup>,
3883    sampler: wgpu::Sampler,
3884    instances: Option<wgpu::Buffer>,
3885    instance_capacity: u64,
3886    /// Reusable staging for the parameter-block upload, padded to the
3887    /// texture's own footprint.
3888    staging: Vec<u8>,
3889    /// Bind groups [`Self::prepare`] has built, over this value's life.
3890    groups_created: u64,
3891}
3892
3893impl FilterResources {
3894    /// A renderer's filter resources, with nothing uploaded yet.
3895    #[must_use]
3896    pub fn new(device: &wgpu::Device) -> Self {
3897        Self {
3898            data: ResourceTexture::new(
3899                device,
3900                &filter_data_texture_descriptor(gpu::MIN_RESOURCE_TEXTURE_HEIGHT),
3901            ),
3902            data_group: None,
3903            sampler: filter_sampler(device),
3904            instances: None,
3905            instance_capacity: 0,
3906            staging: Vec::new(),
3907            groups_created: 0,
3908        }
3909    }
3910
3911    /// Uploads this frame's parameter `blocks` and reserves room for `passes`
3912    /// pass instances, growing either resource if the frame outgrew it.
3913    ///
3914    /// `pipeline` is the one [`EnginePipeline::Filter`] describes; it is needed
3915    /// because wgpu derives a pipeline's bind-group layouts from its shader
3916    /// module, so the group naming the filter-data texture can only be built
3917    /// against the pipeline that will bind it.
3918    ///
3919    /// A block count no filter-data texture could hold (see
3920    /// [`filter_data_texture_height`]) leaves the group unbuilt, which leaves
3921    /// every filter pass of the frame issuing no draw — the page is still
3922    /// cleared, so the layer composites as transparent rather than as whatever
3923    /// its page last held. Unreachable in practice, and "draw less, never
3924    /// wrong" when it is not.
3925    pub fn prepare(
3926        &mut self,
3927        device: &wgpu::Device,
3928        queue: &wgpu::Queue,
3929        pipeline: &wgpu::RenderPipeline,
3930        blocks: &[GpuFilterData],
3931        passes: u32,
3932    ) {
3933        let Some(height) = filter_data_texture_height(blocks.len()) else {
3934            self.data_group = None;
3935            return;
3936        };
3937        if height > self.data.height {
3938            self.data = ResourceTexture::new(device, &filter_data_texture_descriptor(height));
3939            self.data_group = None;
3940        }
3941        if self.data_group.is_none() {
3942            self.groups_created = self.groups_created.saturating_add(1);
3943            self.data_group = Some(device.create_bind_group(&wgpu::BindGroupDescriptor {
3944                label: Some("frust-engine filter data"),
3945                layout: &pipeline.get_bind_group_layout(0),
3946                entries: &[wgpu::BindGroupEntry {
3947                    binding: 0,
3948                    resource: wgpu::BindingResource::TextureView(&self.data.view),
3949                }],
3950            }));
3951        }
3952
3953        // A queue write covers the whole texture extent, so the blocks are
3954        // padded out to its footprint; the trailing texels are never addressed,
3955        // because a pass names its own block by a texel offset the host handed
3956        // it.
3957        let footprint = gpu::resource_texture_bytes(self.data.width, self.data.height);
3958        self.staging.clear();
3959        self.staging
3960            .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
3961        let bytes: &[u8] = bytemuck::cast_slice(blocks);
3962        if let Some(head) = self.staging.get_mut(..bytes.len()) {
3963            head.copy_from_slice(bytes);
3964        }
3965        queue.write_texture(
3966            self.data.copy_target(),
3967            &self.staging,
3968            resource_layout(
3969                gpu::resource_bytes_per_row(self.data.width),
3970                self.data.height,
3971            ),
3972            self.data.extent(),
3973        );
3974
3975        self.reserve_instances(device, passes);
3976    }
3977
3978    /// Grows the instance buffer if this frame's pass count outgrew it.
3979    fn reserve_instances(&mut self, device: &wgpu::Device, passes: u32) {
3980        let stride = size_of::<FilterInstanceData>() as u64;
3981        let required = u64::from(passes).saturating_mul(stride).max(stride);
3982        if self.instances.is_some() && self.instance_capacity >= required {
3983            return;
3984        }
3985        let capacity = required
3986            .checked_next_power_of_two()
3987            .unwrap_or(required)
3988            .max(MIN_FILTER_INSTANCE_CAPACITY.saturating_mul(stride));
3989        self.instance_capacity = capacity;
3990        self.instances = Some(device.create_buffer(&wgpu::BufferDescriptor {
3991            label: Some("frust-engine filter instances"),
3992            size: capacity,
3993            usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
3994            mapped_at_creation: false,
3995        }));
3996    }
3997
3998    /// Writes one pass's instance into slot `index` of the instance buffer.
3999    ///
4000    /// Separate from [`Self::prepare`] because a pass's `dest_texture_size` is
4001    /// the extent the texture pool *quantized* its destination page up to, and
4002    /// NDC is computed against the attachment's real extent — only the round
4003    /// that acquires the page knows it. A queue write issued while the frame is
4004    /// being recorded still lands ahead of the command buffers it is submitted
4005    /// with, which is the same ordering a page's viewport uniform already
4006    /// relies on.
4007    ///
4008    /// A slot past the reserved capacity is dropped rather than written, which
4009    /// leaves that pass drawing whatever the slot last held; unreachable, since
4010    /// `prepare` reserved one slot per pass of this very frame.
4011    pub fn write_instance(&self, queue: &wgpu::Queue, index: u32, instance: &FilterInstanceData) {
4012        let Some(buffer) = self.instances.as_ref() else {
4013            return;
4014        };
4015        let stride = size_of::<FilterInstanceData>() as u64;
4016        let offset = u64::from(index).saturating_mul(stride);
4017        if offset.saturating_add(stride) > self.instance_capacity {
4018            return;
4019        }
4020        queue.write_buffer(buffer, offset, bytemuck::bytes_of(instance));
4021    }
4022
4023    /// Records one filter pass into `encoder`: clear the destination page, then
4024    /// draw the one instanced quad that filters `plan`'s source into it.
4025    ///
4026    /// The clear is unconditional and the draw is not. A filter pass writes only
4027    /// the region its step names — a decimated one a quarter of the texels the
4028    /// pass before it did — and the kernels sample past that region without
4029    /// bounds checks, so whatever surrounds it has to be transparent rather than
4030    /// a previous holder's pixels. That has to hold even on the path where the
4031    /// pass itself cannot be issued, or the layer's composite would sample the
4032    /// page's previous tenant instead of nothing.
4033    ///
4034    /// The pass is opened and closed here, on the caller's own encoder: a filter
4035    /// round is not an exception to [`EngineRenderer::encode`]'s contract.
4036    pub fn record_pass(
4037        &self,
4038        device: &wgpu::Device,
4039        encoder: &mut wgpu::CommandEncoder,
4040        plan: &FilterPassPlan<'_>,
4041    ) {
4042        self.record_pass_timed(device, encoder, plan, None);
4043    }
4044
4045    /// [`Self::record_pass`], stamping the pass's GPU time into `timestamps`.
4046    ///
4047    /// A separate method rather than a field on [`FilterPassPlan`]: the plan is
4048    /// public and built by struct literal outside this crate, so a new required
4049    /// field would break every one of those call sites to serve a diagnostic
4050    /// they do not use. `None` records exactly what [`Self::record_pass`] does.
4051    pub fn record_pass_timed(
4052        &self,
4053        device: &wgpu::Device,
4054        encoder: &mut wgpu::CommandEncoder,
4055        plan: &FilterPassPlan<'_>,
4056        timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
4057    ) {
4058        self.record_pass_with(device, encoder, plan, timestamps, None);
4059    }
4060
4061    /// The group 1 a pass samples `source` through: the page and this
4062    /// resource's sampler. Built against `pipeline`'s layout, which every
4063    /// filter pass shares.
4064    fn source_group(
4065        &self,
4066        device: &wgpu::Device,
4067        pipeline: &wgpu::RenderPipeline,
4068        source: &wgpu::TextureView,
4069    ) -> wgpu::BindGroup {
4070        device.create_bind_group(&wgpu::BindGroupDescriptor {
4071            label: Some("frust-engine filter source"),
4072            layout: &pipeline.get_bind_group_layout(1),
4073            entries: &[
4074                wgpu::BindGroupEntry {
4075                    binding: 0,
4076                    resource: wgpu::BindingResource::TextureView(source),
4077                },
4078                wgpu::BindGroupEntry {
4079                    binding: 1,
4080                    resource: wgpu::BindingResource::Sampler(&self.sampler),
4081                },
4082            ],
4083        })
4084    }
4085
4086    /// [`Self::record_pass_timed`] over a source group the caller already
4087    /// holds (`Some`, built by [`Self::source_group`] for `plan.source`), or one
4088    /// built here (`None`).
4089    fn record_pass_with(
4090        &self,
4091        device: &wgpu::Device,
4092        encoder: &mut wgpu::CommandEncoder,
4093        plan: &FilterPassPlan<'_>,
4094        timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
4095        source: Option<&wgpu::BindGroup>,
4096    ) {
4097        let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
4098            label: Some(plan.label),
4099            color_attachments: &[Some(color_attachment(
4100                plan.dest,
4101                wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
4102            ))],
4103            // A pooled page carries no depth attachment, which is also why the
4104            // filter pipeline declares no depth state.
4105            depth_stencil_attachment: None,
4106            timestamp_writes: timestamps,
4107            occlusion_query_set: None,
4108            multiview_mask: None,
4109        });
4110
4111        let (Some(data_group), Some(instances)) =
4112            (self.data_group.as_ref(), self.instances.as_ref())
4113        else {
4114            return;
4115        };
4116
4117        let built;
4118        let source = match source {
4119            Some(source) => source,
4120            None => {
4121                built = self.source_group(device, plan.pipeline, plan.source);
4122                &built
4123            }
4124        };
4125
4126        pass.set_pipeline(plan.pipeline);
4127        pass.set_vertex_buffer(0, instances.slice(..));
4128        pass.set_bind_group(0, data_group, &[]);
4129        pass.set_bind_group(1, source, &[]);
4130        pass.draw(
4131            0..FILTER_QUAD_VERTICES,
4132            plan.instance..plan.instance.saturating_add(1),
4133        );
4134    }
4135}
4136
4137/// One filter pass's full recording state.
4138///
4139/// The two pages are named as views rather than as parities because
4140/// [`FilterResources`] holds no pool: which texture each group is holding is the
4141/// caller's bookkeeping, whether that caller is the frame path or a test.
4142#[derive(Debug)]
4143pub struct FilterPassPlan<'a> {
4144    /// Label for captures and validation messages.
4145    pub label: &'a str,
4146    /// The pipeline [`EnginePipeline::Filter`] describes.
4147    pub pipeline: &'a wgpu::RenderPipeline,
4148    /// The page this pass writes. Cleared to transparent before it is written.
4149    pub dest: &'a wgpu::TextureView,
4150    /// The page this pass reads — the one the pass before it wrote.
4151    pub source: &'a wgpu::TextureView,
4152    /// Which instance of the filter instance buffer this pass draws.
4153    pub instance: u32,
4154}
4155
4156/// A colour attachment over `view` with `load`, keeping what it stores.
4157fn color_attachment(
4158    view: &wgpu::TextureView,
4159    load: wgpu::LoadOp<wgpu::Color>,
4160) -> wgpu::RenderPassColorAttachment<'_> {
4161    wgpu::RenderPassColorAttachment {
4162        view,
4163        depth_slice: None,
4164        resolve_target: None,
4165        ops: wgpu::Operations {
4166            load,
4167            store: wgpu::StoreOp::Store,
4168        },
4169    }
4170}
4171
4172/// A depth attachment over `view` with `load`, keeping what it stores so the
4173/// pass after it tests against the same buffer.
4174fn depth_attachment(
4175    view: &wgpu::TextureView,
4176    load: wgpu::LoadOp<f32>,
4177) -> wgpu::RenderPassDepthStencilAttachment<'_> {
4178    wgpu::RenderPassDepthStencilAttachment {
4179        view,
4180        depth_ops: Some(wgpu::Operations {
4181            load,
4182            store: wgpu::StoreOp::Store,
4183        }),
4184        stencil_ops: None,
4185    }
4186}
4187
4188/// The frame's base colour as a clear value.
4189///
4190/// The strip pipelines blend premultiplied and the shader emits premultiplied
4191/// colour, so a [`OutputAlpha::Premultiplied`] target's clear value has to be
4192/// premultiplied too — otherwise the background sits in a different alpha
4193/// convention from everything drawn over it. `Straight` is honoured here and
4194/// only here: the pipelines themselves are fixed premultiplied, so a straight
4195/// target's *drawn* content is premultiplied regardless.
4196fn clear_color(base_color: Color, output: OutputAlpha) -> wgpu::Color {
4197    let components = match output {
4198        OutputAlpha::Premultiplied => base_color.premultiply().components,
4199        OutputAlpha::Straight => base_color.components,
4200    };
4201    wgpu::Color {
4202        r: f64::from(components[0]),
4203        g: f64::from(components[1]),
4204        b: f64::from(components[2]),
4205        a: f64::from(components[3]),
4206    }
4207}
4208
4209/// Whether a frame cleared to `base_color` presents opaquely, and so whether
4210/// its alpha channel carries anything a hole punch could reveal.
4211///
4212/// The engine's only statement about the target's own alpha handling:
4213/// [`EngineTarget`] describes how the alpha it produces is *interpreted*
4214/// (premultiplied or straight), never whether it is used at all. A base colour
4215/// at full alpha seals every pixel of the surface, which is exactly the
4216/// presentation [`crate::compile::clear`]'s contract says to skip the
4217/// destination-out pass on.
4218fn is_opaque(base_color: Color) -> bool {
4219    base_color.components[3] >= 1.0
4220}
4221
4222/// The target extent on the `u16` device grid the strip pipeline addresses.
4223fn grid_size(width: u32, height: u32) -> Result<(u16, u16), EngineError> {
4224    let width = u16::try_from(width).map_err(|_| EngineError::TargetTooLarge)?;
4225    let height = u16::try_from(height).map_err(|_| EngineError::TargetTooLarge)?;
4226    Ok((width, height))
4227}
4228
4229/// One unit of drawing inside a round's pass, in execution order.
4230///
4231/// Instance indices are relative to the alpha region of the shared instance
4232/// buffer, which is why [`PassPlan::base`] exists rather than the indices being
4233/// absolute: the opaque region is laid out first and its own segment addresses
4234/// from zero.
4235///
4236/// Every strip segment is also one paint-class run: it names the
4237/// [`PaintClass`] whose program draws it, and a pass sets that class's
4238/// pipeline where the class changes (see the module doc's *Paint-class runs*).
4239#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4240enum Segment {
4241    /// Ordinary strip instances, as `(first, count, class)`, drawn with the
4242    /// frame's own atlas binding.
4243    Strips(u32, u32, PaintClass),
4244    /// Strip instances sampling an externally bound texture, as
4245    /// `(first, count, slot, class)` — a *run*, in the sense
4246    /// [`crate::gpu::bindings::ExternalRuns`] gives the word: the maximal span
4247    /// of consecutive instances that share one texture, drawn with that
4248    /// texture's own group 1 bound.
4249    External(u32, u32, u32, PaintClass),
4250    /// One composite quad at `first`, sampling the finished page it names.
4251    Composite(u32, CompositeSource),
4252    /// A backdrop blur's own strip instances, as `(first, count, parity)`:
4253    /// its rectangle's coverage, each instance sampling the blurred page in
4254    /// `parity`'s group at its own position — drawn through the composite
4255    /// class with that page bound as group 0's layer input.
4256    Backdrop(u32, u32, PageParity),
4257}
4258
4259/// The page a [`Segment::Composite`] samples.
4260#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4261enum CompositeSource {
4262    /// The live pooled page in this group.
4263    Page(PageParity),
4264    /// The retained page of this retained bracket (see
4265    /// [`crate::schedule::RetainedSource`]).
4266    Retained(u32),
4267}
4268
4269/// The retained page one round renders a body into.
4270#[derive(Debug, Clone, Copy)]
4271struct RetainedRound {
4272    /// The retained bracket the page belongs to.
4273    bracket: u32,
4274    /// Whether an earlier round of the same body already rendered into the
4275    /// page this frame, so this one loads it rather than clearing it.
4276    continued: bool,
4277}
4278
4279/// One scheduled round, resolved to the instances and target it draws with.
4280///
4281/// One *plan* rather than one scheduled round: a surface round carrying a hole
4282/// punch is cut into a plan per span between its punches, so the punch pass can
4283/// be recorded at the painter-order position it was hoisted from (see
4284/// [`Scratch::cut_for_punches`]).
4285#[derive(Debug, Clone)]
4286struct RoundPlan {
4287    /// The page this round renders into, or `None` for the frame's own target
4288    /// and for a retained page.
4289    page: Option<PagePlan>,
4290    /// The retained page this round renders a body into, if it does — an
4291    /// off-screen round like a page one, into the store's texture.
4292    retained: Option<RetainedRound>,
4293    /// This round's slice of [`Scratch::segments`], in execution order.
4294    segments: Range<usize>,
4295    /// Page groups this round consumed, indexed by
4296    /// [`PageParity::index`]; each returns to the pool once the pass ends.
4297    released: [bool; MAX_LIVE_PAGES],
4298    /// The filter pass this round runs, on a filter round — which draws no
4299    /// strip and composites nothing, so its `segments` range is empty.
4300    filter: Option<FilterPlan>,
4301    /// The region of the frame's own target this round copies into its page,
4302    /// on a region-copy round — which, like a filter round, draws no strip.
4303    copy: Option<RectU16>,
4304    /// The hole punches to erase with once this plan's own pass has been
4305    /// recorded, as `(first, count)` into the alpha region — the cut this plan
4306    /// ends at.
4307    ///
4308    /// Only a *surface* plan ever carries one: a punch recorded inside an
4309    /// isolated layer was hoisted out of it to the frame root (see
4310    /// [`crate::compile::clear`]), so a page round is never cut and a filter
4311    /// round — which draws nothing of the frame's own — never is either.
4312    punch: (u32, u32),
4313}
4314
4315/// One filter pass, resolved to the instance slot it draws and the page group
4316/// it reads.
4317#[derive(Debug, Clone, Copy)]
4318struct FilterPlan {
4319    /// Which pass of the filter's sequence this is, and at what extents.
4320    step: FilterStep,
4321    /// The group holding the page this pass reads.
4322    source: PageParity,
4323    /// The texel this filter's parameter block starts at in the filter-data
4324    /// texture — where the fragment stage reads its kernel from.
4325    data_offset: u32,
4326    /// The filter layer's own extent, before any decimation; it bounds the
4327    /// transparent border a decimated pass overdraws.
4328    original: SizeU16,
4329    /// This pass's slot in the frame's filter instance buffer.
4330    instance: u32,
4331}
4332
4333/// The pooled page one round renders into.
4334#[derive(Debug, Clone, Copy)]
4335struct PagePlan {
4336    parity: PageParity,
4337    size: PageSize,
4338    /// Whether an earlier round of the same layer already rendered into this
4339    /// page, so this round takes that texture back rather than acquiring a
4340    /// fresh one, and loads it rather than clearing it.
4341    continued: bool,
4342}
4343
4344/// The frame's instance buffers and pass plan, retained so a steady-state frame
4345/// refills them rather than reallocating them.
4346///
4347/// The segments of every round live in one flat vector rather than a vector per
4348/// round, so a frame with layers costs no allocation once the first one has
4349/// grown these buffers.
4350#[derive(Debug, Default)]
4351struct Scratch {
4352    /// Fully-covered spans of the opaque draws targeting the frame's own
4353    /// surface, drawn unblended with depth write in one pass ahead of every
4354    /// round — including the surface's own, of which a frame may have several.
4355    opaque: Vec<GpuStrip>,
4356    /// Everything else, drawn premultiplied-blended in painter order and laid
4357    /// out round by round in execution order.
4358    alpha: Vec<GpuStrip>,
4359    /// Every round's segments, back to back; [`RoundPlan::segments`] slices it.
4360    segments: Vec<Segment>,
4361    /// The frame's rounds, each layer's before the round that composites it and
4362    /// the surface's last round last.
4363    rounds: Vec<RoundPlan>,
4364    /// The instances of the punches no cut reached — those recorded past every
4365    /// op of the frame — as `(first, count)` into the alpha region.
4366    ///
4367    /// Their pass is the last thing the frame records, which is where a
4368    /// `ClearRect` recorded last belongs in painter order anyway. Punches the
4369    /// walk *did* cut at are held on their own [`RoundPlan::punch`] instead.
4370    ///
4371    /// No round's segments name a punch instance, wherever it sits in the
4372    /// buffer, so dropping the punch passes leaves the frame exactly as the
4373    /// display list would read without the clear.
4374    punch: (u32, u32),
4375    /// The deepest painter's-order index inside each recorded layer, which is
4376    /// the depth its composite carries. Filled as the rounds are walked, which
4377    /// is sound because a layer's own round always precedes the round that
4378    /// composites it.
4379    layer_depth: Vec<u32>,
4380    /// One parameter block per *filter layer* of the frame, in the order the
4381    /// rounds first named them — which is the order the texel offsets in
4382    /// [`FilterPlan::data_offset`] were taken from.
4383    filter_blocks: Vec<GpuFilterData>,
4384    /// The layers `filter_blocks` holds, parallel to it, so a filter's second
4385    /// and later passes reuse the block its first one packed rather than
4386    /// repacking one per pass.
4387    filter_layers: Vec<u32>,
4388    /// How many filter passes this frame runs, and so how many instances its
4389    /// filter instance buffer has to hold.
4390    filter_passes: u32,
4391    /// How many region-copy rounds this frame runs — one per backdrop blur
4392    /// the scheduler served.
4393    copies: u32,
4394    /// How segments are cut into paint-class runs this frame — copied in from
4395    /// the renderer ahead of [`Self::build`].
4396    routing: ClassRouting,
4397    /// The opaque region's own segments: `(0, opaque.len())` cut into class
4398    /// runs, drawn by the one depth-writing pass.
4399    opaque_segments: Vec<Segment>,
4400    /// The classes the opaque pass's runs name.
4401    opaque_classes: ClassSet,
4402    /// The classes the frame's surface rounds name — composites included.
4403    surface_classes: ClassSet,
4404    /// The classes the frame's page rounds name — composites included.
4405    page_classes: ClassSet,
4406    /// Whether the round being planned renders into a page, which decides the
4407    /// class set [`Self::push_run`] records a run's class in.
4408    planning_page: bool,
4409    /// Reusable output of [`merge_class_runs`].
4410    class_runs: Vec<ClassRun>,
4411    /// Where the paint class changes in [`Self::alpha`], as `(first instance,
4412    /// class)` in ascending order — one entry per draw whose class differs
4413    /// from the draw before it. Every instance of a draw shares its paint, so
4414    /// these are the only places a class run can start, and a segment is split
4415    /// by walking them instead of re-reading every instance.
4416    alpha_marks: Vec<(u32, PaintClass)>,
4417    /// The same marks for [`Self::opaque`].
4418    opaque_marks: Vec<(u32, PaintClass)>,
4419}
4420
4421impl Scratch {
4422    /// Turns a scheduled frame into instances and a per-round pass plan,
4423    /// routing each instance to the pass that can draw it.
4424    ///
4425    /// A draw's anti-aliased spans always land in the blended buffer: partial
4426    /// coverage is not opaque however opaque the paint is. Its fully-covered
4427    /// spans land in the opaque buffer only when the paint is opaque, depth is
4428    /// available to re-establish their ordering against the blended ones, *and*
4429    /// the draw targets the frame's own surface — a page has no depth
4430    /// attachment, so a page round is a plain painter's-algorithm walk.
4431    ///
4432    /// `backdrops` is the frame's served backdrop blurs — `frame.backdrops`,
4433    /// or none at all on a target no region can be read back from. A draw the
4434    /// scheduler placed at a backdrop's own depth is that backdrop's rectangle
4435    /// and samples its blurred page instead of its recorded paint; and no draw
4436    /// at or past the first backdrop's depth takes the depth-writing pass,
4437    /// which is recorded ahead of every surface round — so ahead of the copy
4438    /// that reads the target back, which must not see what is painted over the
4439    /// backdrop.
4440    fn build(
4441        &mut self,
4442        frame: &CompiledFrame,
4443        rounds: &[Round],
4444        depth_active: bool,
4445        punches: bool,
4446        paint_slots: &[Option<ResolvedPaint>],
4447        backdrops: &[Backdrop],
4448    ) {
4449        self.opaque.clear();
4450        self.alpha.clear();
4451        self.segments.clear();
4452        self.rounds.clear();
4453        self.punch = (0, 0);
4454        self.layer_depth.clear();
4455        self.layer_depth.resize(frame.recorder.layers.len(), 0);
4456        // A `DrawRetained`'s layer holds no draw, so its composite's depth is
4457        // the slot the command itself was recorded at rather than the deepest
4458        // index inside it.
4459        for bracket in &frame.retained {
4460            if let RetainedKind::Draw { depth, .. } = bracket.kind
4461                && let Some(slot) = self.layer_depth.get_mut(bracket.layer as usize)
4462            {
4463                *slot = depth;
4464            }
4465        }
4466        self.filter_blocks.clear();
4467        self.filter_layers.clear();
4468        self.filter_passes = 0;
4469        self.copies = 0;
4470        self.opaque_segments.clear();
4471        self.alpha_marks.clear();
4472        self.opaque_marks.clear();
4473        self.opaque_classes = ClassSet::EMPTY;
4474        self.surface_classes = ClassSet::EMPTY;
4475        self.page_classes = ClassSet::EMPTY;
4476
4477        let draws = frame.draws();
4478        let strips = frame.strip_buf();
4479        // The punches still to be issued, in the order the compiler hoisted
4480        // them — which is depth order, since each took its own index from the
4481        // frame's monotonic painter-order counter. Empty on a frame with no
4482        // clear and on a target that disregards alpha (`punches`), and then
4483        // nothing below ever cuts: a non-punching frame is planned exactly as
4484        // it always was.
4485        let mut pending: &[ClearPunch] = if punches { &frame.clears } else { &[] };
4486        // Every draw before the first backdrop may take the depth-writing pass;
4487        // none from it on. `u32::MAX` on a frame with no backdrop, which is
4488        // then routed exactly as it always was.
4489        let opaque_limit = backdrops
4490            .first()
4491            .map_or(u32::MAX, |backdrop| backdrop.depth);
4492
4493        for round in rounds {
4494            let page = round.page();
4495            let retained = round.retained();
4496            let offscreen = page.is_some() || retained.is_some();
4497
4498            // A region-copy round draws nothing: it is planned as its page and
4499            // the region it copies. Any punch recorded ahead of the backdrop
4500            // it serves is erased first, as a cut of its own, so the copy reads
4501            // the target the display list says lies beneath the backdrop.
4502            if let Some(copy) = round.region_copy() {
4503                if let Some(page) = page {
4504                    if !pending.is_empty()
4505                        && let Some(backdrop) = backdrops
4506                            .iter()
4507                            .find(|backdrop| backdrop.layer == page.layer)
4508                    {
4509                        let mut first_segment = self.segments.len();
4510                        let mut run_start = self.alpha.len() as u32;
4511                        let mut run_external = None;
4512                        self.cut_for_punches(
4513                            &mut pending,
4514                            strips,
4515                            backdrop.depth,
4516                            &mut first_segment,
4517                            &mut run_start,
4518                            &mut run_external,
4519                        );
4520                    }
4521                    self.copies = self.copies.saturating_add(1);
4522                    self.rounds.push(RoundPlan {
4523                        page: Some(PagePlan {
4524                            parity: page.parity,
4525                            size: page.size,
4526                            continued: false,
4527                        }),
4528                        retained: None,
4529                        segments: self.segments.len()..self.segments.len(),
4530                        released: released_groups(&round.released),
4531                        filter: None,
4532                        copy: Some(copy.source),
4533                        punch: (0, 0),
4534                    });
4535                }
4536                continue;
4537            }
4538
4539            let filter = self.plan_filter(frame, round, backdrops);
4540            // A page holds its layer — or one column band of it — at the page's
4541            // own origin, so every instance of the round is shifted into that
4542            // column and clipped to it (see `PageWindow`). A retained page holds
4543            // its body the same way.
4544            let window = match (page, retained) {
4545                (Some(page), _) => PageWindow::of(page),
4546                (None, Some(retained)) => PageWindow::of_retained(retained),
4547                (None, None) => PageWindow::ROOT,
4548            };
4549            let split_opaque = !offscreen && depth_active;
4550            self.planning_page = offscreen;
4551            // Only the frame's own target is punched — a punch inside an
4552            // isolated layer was hoisted out of it — so only a surface round is
4553            // ever cut.
4554            let cuts = !offscreen;
4555
4556            let mut first_segment = self.segments.len();
4557            let mut deepest = 0_u32;
4558            let mut run_start = self.alpha.len() as u32;
4559            // The external texture the open run is drawn with, `None` while it
4560            // is drawn with the frame's own atlas binding. A draw that names a
4561            // different one closes the run: there is a single external binding
4562            // to set (see [`crate::gpu::bindings`]).
4563            let mut run_external: Option<u32> = None;
4564
4565            for op in &round.ops {
4566                match op {
4567                    RoundOp::Draws(range) => {
4568                        let batch = draws
4569                            .get(range.start as usize..range.end as usize)
4570                            .unwrap_or(&[]);
4571                        for draw in batch {
4572                            deepest = deepest.max(draw.depth);
4573                            // Ahead of the paint lookup rather than after it, so
4574                            // the cut follows the order the display list was
4575                            // recorded in rather than the subset of it that
4576                            // survived lowering.
4577                            if cuts && !pending.is_empty() {
4578                                self.cut_for_punches(
4579                                    &mut pending,
4580                                    strips,
4581                                    draw.depth,
4582                                    &mut first_segment,
4583                                    &mut run_start,
4584                                    &mut run_external,
4585                                );
4586                            }
4587                            // A backdrop's rectangle: drawn from the blurred
4588                            // page this round samples rather than from the
4589                            // transparent paint it was recorded with.
4590                            if let Ok(at) = backdrops
4591                                .binary_search_by_key(&draw.depth, |backdrop| backdrop.depth)
4592                            {
4593                                if let Some(backdrop) = backdrops.get(at)
4594                                    && let Some(sample) = round
4595                                        .samples
4596                                        .iter()
4597                                        .find(|sample| sample.layer == backdrop.layer)
4598                                    && let Some(run) = strips.get(draw.strip_range.clone())
4599                                {
4600                                    let end = self.alpha.len() as u32;
4601                                    self.push_run(run_start, end, run_external);
4602                                    run_external = None;
4603                                    self.push_backdrop(
4604                                        backdrop,
4605                                        sample.parity,
4606                                        draw.depth,
4607                                        run,
4608                                        window,
4609                                    );
4610                                    run_start = self.alpha.len() as u32;
4611                                }
4612                                continue;
4613                            }
4614                            let Some(paint) = pack_paint(&draw.paint, draw.depth, paint_slots)
4615                            else {
4616                                continue;
4617                            };
4618                            if paint.glyph_quad {
4619                                // One rect instance, from the quad alone: the
4620                                // draw's strips are its rectangle's coverage,
4621                                // which the instance covers whole.
4622                                let Some(quad) = glyph_quad(&frame.glyph_quads, draw.depth) else {
4623                                    continue;
4624                                };
4625                                if paint.external != run_external {
4626                                    let end = self.alpha.len() as u32;
4627                                    self.push_run(run_start, end, run_external);
4628                                    run_start = end;
4629                                    run_external = paint.external;
4630                                }
4631                                mark_class(
4632                                    &mut self.alpha_marks,
4633                                    self.alpha.len() as u32,
4634                                    paint.class,
4635                                );
4636                                if let Some(instance) = window.place_quad(quad, paint) {
4637                                    self.alpha.push(instance);
4638                                }
4639                                continue;
4640                            }
4641                            let Some(run) = strips.get(draw.strip_range.clone()) else {
4642                                continue;
4643                            };
4644                            // Before the first of this draw's instances lands,
4645                            // so the run that closes holds exactly the
4646                            // instances drawn with the texture it names.
4647                            if paint.external != run_external {
4648                                let end = self.alpha.len() as u32;
4649                                self.push_run(run_start, end, run_external);
4650                                run_start = end;
4651                                run_external = paint.external;
4652                            }
4653                            let to_opaque =
4654                                paint.opaque && split_opaque && draw.depth < opaque_limit;
4655                            let class = paint.class;
4656                            mark_class(&mut self.alpha_marks, self.alpha.len() as u32, class);
4657                            if to_opaque {
4658                                mark_class(&mut self.opaque_marks, self.opaque.len() as u32, class);
4659                            }
4660
4661                            // A generation's last strip is its sentinel, which
4662                            // is what carries the preceding strip's extent — so
4663                            // every instance comes from a pair, and the
4664                            // sentinel itself never becomes one.
4665                            for pair in run.windows(2) {
4666                                self.push_span(&pair[0], &pair[1], paint, to_opaque, window);
4667                            }
4668                        }
4669                    }
4670                    RoundOp::Composite(composite) => {
4671                        let depth = self
4672                            .layer_depth
4673                            .get(composite.layer as usize)
4674                            .copied()
4675                            .unwrap_or(0);
4676                        // A composite carries the deepest index inside its
4677                        // layer, so a layer recorded after a punch is cut
4678                        // against it exactly like a draw would be — which is
4679                        // what keeps a translucent layer over the slot from
4680                        // being erased by it.
4681                        if cuts && !pending.is_empty() {
4682                            self.cut_for_punches(
4683                                &mut pending,
4684                                strips,
4685                                depth,
4686                                &mut first_segment,
4687                                &mut run_start,
4688                                &mut run_external,
4689                            );
4690                        }
4691
4692                        // Consecutive draw batches merge into one segment; a
4693                        // composite is what breaks the run, because it binds a
4694                        // different page as its colour source.
4695                        let end = self.alpha.len() as u32;
4696                        self.push_run(run_start, end, run_external);
4697
4698                        deepest = deepest.max(depth);
4699                        let source = match composite.retained {
4700                            Some(retained) => CompositeSource::Retained(retained.bracket),
4701                            None => CompositeSource::Page(composite.parity),
4702                        };
4703                        self.segments.push(Segment::Composite(end, source));
4704                        self.note_class(self.routing.composite_class());
4705                        self.alpha
4706                            .push(composite_instance(composite, window.origin(), depth));
4707                        run_start = self.alpha.len() as u32;
4708                        // A composite draws through group 1's own atlas
4709                        // binding, so the run after it starts un-bound again.
4710                        run_external = None;
4711                    }
4712                }
4713            }
4714
4715            let end = self.alpha.len() as u32;
4716            self.push_run(run_start, end, run_external);
4717
4718            // Accumulated rather than assigned: a layer whose round was cut
4719            // renders in several rounds, and the depth its composite carries is
4720            // the deepest index across all of them.
4721            let own_layer = page
4722                .map(|page| page.layer)
4723                .or(retained.map(|retained| retained.layer));
4724            if let Some(layer) = own_layer
4725                && let Some(slot) = self.layer_depth.get_mut(layer as usize)
4726            {
4727                *slot = (*slot).max(deepest);
4728            }
4729
4730            let released = released_groups(&round.released);
4731
4732            self.rounds.push(RoundPlan {
4733                page: page.map(|page| PagePlan {
4734                    parity: page.parity,
4735                    size: page.size,
4736                    continued: page.continued,
4737                }),
4738                retained: retained.map(|retained| RetainedRound {
4739                    bracket: retained.bracket,
4740                    continued: retained.continued,
4741                }),
4742                segments: first_segment..self.segments.len(),
4743                released,
4744                filter,
4745                copy: None,
4746                // The round's last plan draws to its own end; a punch past
4747                // every op of the frame is issued after the whole walk instead.
4748                punch: (0, 0),
4749            });
4750        }
4751
4752        // Whatever no op was recorded after: a `ClearRect` recorded last, which
4753        // is the ordinary shape of a platform-view slot.
4754        if !pending.is_empty() {
4755            self.punch = self.push_punches(pending, strips);
4756        }
4757
4758        // The opaque region is one segment of the one depth-writing pass, cut
4759        // into class runs like any other.
4760        self.class_runs.clear();
4761        merge_class_runs(
4762            marked_runs(&self.opaque_marks, 0, self.opaque.len() as u32),
4763            self.routing.min_run,
4764            self.routing.uber,
4765            &mut self.class_runs,
4766        );
4767        debug_assert!(
4768            runs_draw(&self.class_runs, &self.opaque),
4769            "the opaque marks disagree with the instances they mark"
4770        );
4771        for run in &self.class_runs {
4772            self.opaque_segments
4773                .push(Segment::Strips(run.first, run.count, run.class));
4774            self.opaque_classes.insert(run.class);
4775        }
4776    }
4777
4778    /// Records that the round being planned draws a run of `class`.
4779    fn note_class(&mut self, class: PaintClass) {
4780        if self.planning_page {
4781            self.page_classes.insert(class);
4782        } else {
4783            self.surface_classes.insert(class);
4784        }
4785    }
4786
4787    /// Closes the open span of a surface round at `depth`, issuing every punch
4788    /// recorded before it as a pass of its own.
4789    ///
4790    /// This is the whole painter-order restoration: the ops recorded before the
4791    /// punch become a plan that ends here, the punch's own instances follow
4792    /// them in the buffer, and the ops recorded after it start a plan of their
4793    /// own — so the erase lands between the two rather than after both. A punch
4794    /// pass is a pass of its own because it draws through a different pipeline
4795    /// (destination-out) than the strips around it, not because of what it
4796    /// covers.
4797    ///
4798    /// Does nothing when no pending punch is shallower than `depth`, which is
4799    /// every op of every frame that records no clear.
4800    fn cut_for_punches(
4801        &mut self,
4802        pending: &mut &[ClearPunch],
4803        strips: &[Strip],
4804        depth: u32,
4805        first_segment: &mut usize,
4806        run_start: &mut u32,
4807        run_external: &mut Option<u32>,
4808    ) {
4809        let cut = pending
4810            .iter()
4811            .take_while(|punch| punch.depth < depth)
4812            .count();
4813        if cut == 0 {
4814            return;
4815        }
4816        let (issued, rest) = pending.split_at(cut);
4817        *pending = rest;
4818
4819        // Close the run this cut interrupts, so the plan's segments name only
4820        // what was recorded before the punch.
4821        let end = self.alpha.len() as u32;
4822        self.push_run(*run_start, end, *run_external);
4823        // The punch's own instances go in next and are drawn solid, so the run
4824        // resumed after the cut starts with nothing bound.
4825        *run_external = None;
4826
4827        let punch = self.push_punches(issued, strips);
4828        self.rounds.push(RoundPlan {
4829            page: None,
4830            retained: None,
4831            segments: *first_segment..self.segments.len(),
4832            // The pages a cut round consumed are handed back when its LAST plan
4833            // ends: nothing between two plans of one round acquires a page, and
4834            // a composite in an earlier plan still has to sample the page it
4835            // names.
4836            released: [false; MAX_LIVE_PAGES],
4837            filter: None,
4838            copy: None,
4839            punch,
4840        });
4841        *first_segment = self.segments.len();
4842        *run_start = self.alpha.len() as u32;
4843    }
4844
4845    /// Records the open run of instances `first..end` as segments drawn with
4846    /// the texture in `external` bound — one per paint-class run the span
4847    /// splits into — or nothing at all when the run is empty.
4848    ///
4849    /// The one place a strip segment is created, so both run-breaking rules —
4850    /// a segment holds instances sharing one group-1 binding, and one paint
4851    /// class ([`merge_class_runs`]) — are stated once rather than at each of
4852    /// the four points a run can close. The class runs tile the span in order,
4853    /// so the segments still draw `first..end` front to back exactly as one
4854    /// segment did.
4855    fn push_run(&mut self, first: u32, end: u32, external: Option<u32>) {
4856        if end.checked_sub(first).filter(|count| *count > 0).is_none() {
4857            return;
4858        }
4859        let Some(span) = self.alpha.get(first as usize..end as usize) else {
4860            return;
4861        };
4862        self.class_runs.clear();
4863        merge_class_runs(
4864            marked_runs(&self.alpha_marks, first, end),
4865            self.routing.min_run,
4866            self.routing.uber,
4867            &mut self.class_runs,
4868        );
4869        debug_assert!(
4870            runs_draw(&self.class_runs, span),
4871            "the alpha marks disagree with the instances they mark"
4872        );
4873        for index in 0..self.class_runs.len() {
4874            let run = self.class_runs[index];
4875            let at = first.saturating_add(run.first);
4876            self.segments.push(match external {
4877                Some(slot) => Segment::External(at, run.count, slot, run.class),
4878                None => Segment::Strips(at, run.count, run.class),
4879            });
4880            self.note_class(run.class);
4881        }
4882    }
4883
4884    /// Appends a backdrop blur's rectangle — the strip run `run` of its one
4885    /// draw, at painter-order `depth` — as instances sampling the blurred page
4886    /// in `parity`'s group, and the [`Segment::Backdrop`] drawing them.
4887    ///
4888    /// Each instance reads the page at its own position relative to the
4889    /// backdrop's region, which the page holds at its origin, so every pixel
4890    /// of the rectangle takes the blur of the backdrop under that very pixel;
4891    /// the run's own coverage — the rectangle's edges, and any coverage-mask
4892    /// clip it was generated under — weights it exactly as it would a fill.
4893    /// Never opaque: the blur of a backdrop reaching transparent edges is not.
4894    fn push_backdrop(
4895        &mut self,
4896        backdrop: &Backdrop,
4897        parity: PageParity,
4898        depth: u32,
4899        run: &[Strip],
4900        window: PageWindow,
4901    ) {
4902        let class = self.routing.composite_class();
4903        let paint = PackedPaint {
4904            payload: PaintPayload::Layer(backdrop.region.x0, backdrop.region.y0),
4905            paint: LAYER_PAINT_SOURCE | u32::from(u8::MAX),
4906            depth_index: depth,
4907            opaque: false,
4908            external: None,
4909            class,
4910            glyph_quad: false,
4911        };
4912        let first = self.alpha.len() as u32;
4913        for pair in run.windows(2) {
4914            self.push_span(&pair[0], &pair[1], paint, false, window);
4915        }
4916        let count = (self.alpha.len() as u32).saturating_sub(first);
4917        if count > 0 {
4918            self.segments.push(Segment::Backdrop(first, count, parity));
4919            self.note_class(class);
4920        }
4921    }
4922
4923    /// Turns `punches` into destination-out instances, appended to the alpha
4924    /// region and named by no round's segments, as `(first, count)`.
4925    ///
4926    /// Each punch is a strip run like any other, drawn with an opaque source so
4927    /// the blend state's `1 − src.a` reaches zero exactly where the coverage is
4928    /// full, and carrying the painter-order depth it was hoisted from — which
4929    /// still orders it against the frame's one depth-writing pass, whose opaque
4930    /// coverage is recorded once ahead of every round and so cannot be cut.
4931    fn push_punches(&mut self, punches: &[ClearPunch], strips: &[Strip]) -> (u32, u32) {
4932        let first = self.alpha.len() as u32;
4933
4934        for punch in punches {
4935            let paint = PackedPaint {
4936                payload: PaintPayload::Solid(PUNCH_SOURCE),
4937                paint: SOLID_PAINT,
4938                depth_index: punch.depth,
4939                // An erase never joins the depth-writing pass: it establishes
4940                // no colour for a later fragment to be rejected against.
4941                opaque: false,
4942                external: None,
4943                class: PaintClass::SolidCoverage,
4944                glyph_quad: false,
4945            };
4946            let Some(run) = strips.get(punch.strip_range.clone()) else {
4947                continue;
4948            };
4949            for pair in run.windows(2) {
4950                // Only the frame's own target is punched, so a punch's
4951                // instances are never shifted and never clipped.
4952                self.push_span(&pair[0], &pair[1], paint, false, PageWindow::ROOT);
4953            }
4954        }
4955
4956        (first, (self.alpha.len() as u32).saturating_sub(first))
4957    }
4958
4959    /// Emits the instances the `strip`/`next` pair describes: the strip's own
4960    /// alpha-sampled span, plus the solid span filling the gap to `next` when
4961    /// the winding between them says there is one.
4962    ///
4963    /// `window` shifts each instance's *geometry* into the round's target and
4964    /// clips it to the column that target holds, while the paint is still
4965    /// sampled at the scene position the strip was rasterized at — a layer's
4966    /// contents move into its page, the gradient or image painting them does
4967    /// not. An instance the window culls entirely is not emitted at all.
4968    fn push_span(
4969        &mut self,
4970        strip: &Strip,
4971        next: &Strip,
4972        paint: PackedPaint,
4973        to_opaque: bool,
4974        window: PageWindow,
4975    ) {
4976        let values = paint.values_at(strip.x, strip.y);
4977        let mut span = GpuStrip::from_strip_pair(strip, next, values);
4978        if window.place(&mut span, paint) {
4979            self.alpha.push(span);
4980        }
4981        // A gap starts where the strip ends rather than where it begins, so a
4982        // position-sampled paint is re-evaluated at the gap's own origin — the
4983        // gap is a different piece of the scene, not a continuation of the
4984        // span's sampling. `place` re-evaluates it once more if the window's
4985        // own left edge moves the instance again.
4986        if let Some(mut gap) = GpuStrip::gap_fill(strip, next, values) {
4987            gap.payload = paint.payload_at(gap.x, gap.y);
4988            if window.place(&mut gap, paint) {
4989                if to_opaque {
4990                    self.opaque.push(gap);
4991                } else {
4992                    self.alpha.push(gap);
4993                }
4994            }
4995        }
4996    }
4997
4998    /// Resolves `round`'s filter pass, if it has one, packing the layer's
4999    /// parameter block on first sight and claiming the pass's instance slot.
5000    ///
5001    /// `None` for an ordinary round, and also for the two shapes that cannot
5002    /// occur: a filter round with no page (the scheduler always gives one a
5003    /// page) and a recorded kind [`served_filter`] does not recognise (the
5004    /// scheduler refuses one before it plans a round). Both answer by leaving
5005    /// the round's pass unissued — its
5006    /// page is still cleared — rather than by asserting (E17).
5007    fn plan_filter(
5008        &mut self,
5009        frame: &CompiledFrame,
5010        round: &Round,
5011        backdrops: &[Backdrop],
5012    ) -> Option<FilterPlan> {
5013        let pass = round.filter_pass()?;
5014        let page = round.page()?;
5015        let recorded = frame.recorder.layers.get(pass.layer as usize)?;
5016        // A backdrop's layer is a regular one: its blur is the backdrop
5017        // record's, not the recording's.
5018        let data_offset = match backdrops
5019            .iter()
5020            .find(|backdrop| backdrop.layer == pass.layer)
5021        {
5022            Some(backdrop) => self.pack_filter_block(pass.layer, || {
5023                Some(GpuFilterData::from(GpuGaussianBlur::from(&backdrop.blur())))
5024            })?,
5025            None => self.filter_block(pass.layer, &recorded.kind)?,
5026        };
5027
5028        let instance = self.filter_passes;
5029        self.filter_passes = self.filter_passes.saturating_add(1);
5030        Some(FilterPlan {
5031            step: pass.step,
5032            source: pass.source,
5033            data_offset,
5034            original: SizeU16::from(page.bounds),
5035            instance,
5036        })
5037    }
5038
5039    /// The texel offset of `layer`'s parameter block, packing the block on
5040    /// first sight.
5041    ///
5042    /// A linear scan rather than a map: a frame's filter layers are counted in
5043    /// ones (a filter layer is served only directly under the surface), and one
5044    /// allocation-free vector beats a hash map that would have to be cleared
5045    /// every frame.
5046    fn filter_block(&mut self, layer: u32, kind: &RecordedLayerKind) -> Option<u32> {
5047        // The one dispatch every filter-recognising site in this crate shares
5048        // (`schedule::layer_role`/`filter_rounds`), so the block packed here is
5049        // for the same filter the scheduler planned the passes of.
5050        self.pack_filter_block(layer, || {
5051            Some(match served_filter(layer, kind).ok()? {
5052                ServedFilter::Blur(blur) => GpuFilterData::from(GpuGaussianBlur::from(&blur)),
5053                ServedFilter::DropShadow(shadow) => {
5054                    GpuFilterData::from(GpuDropShadow::from(&shadow))
5055                }
5056            })
5057        })
5058    }
5059
5060    /// The texel offset of `layer`'s parameter block, packing the one `make`
5061    /// answers on first sight — [`Self::filter_block`]'s bookkeeping, shared
5062    /// with a backdrop blur, whose block comes from its own record.
5063    fn pack_filter_block(
5064        &mut self,
5065        layer: u32,
5066        make: impl FnOnce() -> Option<GpuFilterData>,
5067    ) -> Option<u32> {
5068        let index = match self.filter_layers.iter().position(|id| *id == layer) {
5069            Some(index) => index,
5070            None => {
5071                let block = make()?;
5072                self.filter_layers.push(layer);
5073                self.filter_blocks.push(block);
5074                self.filter_layers.len().saturating_sub(1)
5075            }
5076        };
5077
5078        u32::try_from(index)
5079            .ok()?
5080            .checked_mul(GpuFilterData::SIZE_TEXELS)
5081    }
5082
5083    /// Whether this frame records a hole-punch pass at all — at a cut inside a
5084    /// surface round, or after every round of the frame.
5085    ///
5086    /// Derived rather than counted alongside the instances: the two places a
5087    /// punch can land are the two places its instances are recorded from, and
5088    /// one answer read off both is one fewer field to keep in step.
5089    fn punches(&self) -> bool {
5090        self.punch.1 > 0 || self.rounds.iter().any(|plan| plan.punch.1 > 0)
5091    }
5092
5093    /// How many of this frame's rounds render *strips* into a pooled page, and
5094    /// so how many viewport uniforms of their own the frame needs.
5095    ///
5096    /// A filter round targets a page too and is deliberately not counted: it
5097    /// binds no viewport uniform at all, mapping NDC against the destination
5098    /// extent its own instance carries.
5099    fn page_rounds(&self) -> usize {
5100        self.rounds
5101            .iter()
5102            .filter(|plan| {
5103                (plan.page.is_some() || plan.retained.is_some())
5104                    && plan.filter.is_none()
5105                    && plan.copy.is_none()
5106            })
5107            .count()
5108    }
5109
5110    /// The passes a frame planned like this one records, in order, into
5111    /// `out` — the pass list of the module doc's *The frame's passes*, stated
5112    /// over the plan alone so it can be checked without a device.
5113    ///
5114    /// `record_frame` is what actually records them, and a debug build holds
5115    /// every frame it records to this list (`EngineRenderer::check_pass_list`).
5116    /// The two agree by construction on every shape the walk serves; the
5117    /// unreachable skips the walk guards against (a page the pool refuses, a
5118    /// pipeline the frame never resolved) are the only places they could part.
5119    #[cfg(any(test, debug_assertions))]
5120    fn plan_passes(&self, shape: &PassShape, out: &mut Vec<FramePass>) {
5121        out.clear();
5122        if !shape.merge {
5123            out.push(FramePass::Clear);
5124        }
5125        if !shape.instances {
5126            if shape.merge {
5127                out.push(FramePass::Opening);
5128            }
5129            return;
5130        }
5131        if !shape.merge && shape.opaque {
5132            out.push(FramePass::Opaque);
5133        }
5134
5135        let mut owed = shape.merge;
5136        for plan in &self.rounds {
5137            // A copy reads the surface back, so the opening — the clear and
5138            // the opaque strips — is recorded ahead of it.
5139            if plan.copy.is_some() {
5140                if owed {
5141                    out.push(FramePass::Opening);
5142                    owed = false;
5143                }
5144                out.push(FramePass::Copy);
5145                continue;
5146            }
5147            if plan.filter.is_some() {
5148                out.push(FramePass::Filter);
5149                continue;
5150            }
5151            if plan.page.is_some() || plan.retained.is_some() {
5152                out.push(FramePass::Page);
5153            } else if !plan.segments.is_empty() {
5154                out.push(if owed {
5155                    FramePass::Opening
5156                } else {
5157                    FramePass::Surface
5158                });
5159                owed = false;
5160            }
5161            if shape.punch && plan.punch.1 > 0 {
5162                if owed {
5163                    out.push(FramePass::Opening);
5164                    owed = false;
5165                }
5166                out.push(FramePass::Punch);
5167            }
5168        }
5169        if owed {
5170            out.push(FramePass::Opening);
5171        }
5172        if shape.punch && self.punch.1 > 0 {
5173            out.push(FramePass::Punch);
5174        }
5175    }
5176
5177    /// The instance bytes, opaque buffer first, so one vertex buffer serves
5178    /// every pass and each is a contiguous instance range into it.
5179    fn instance_bytes(&self) -> (&[u8], &[u8]) {
5180        (
5181            bytemuck::cast_slice(&self.opaque),
5182            bytemuck::cast_slice(&self.alpha),
5183        )
5184    }
5185}
5186
5187/// Records that the instances from `at` on carry `class`, unless the marks
5188/// already say so.
5189///
5190/// A draw that emitted nothing leaves its mark at the same index as the next
5191/// draw's, which then overwrites it — and drops it altogether when that leaves
5192/// two neighbouring marks of one class.
5193fn mark_class(marks: &mut Vec<(u32, PaintClass)>, at: u32, class: PaintClass) {
5194    match marks.last_mut() {
5195        Some(last) if last.1 == class => {}
5196        Some(last) if last.0 == at => {
5197            last.1 = class;
5198            let len = marks.len();
5199            if len >= 2 && marks[len - 2].1 == class {
5200                marks.pop();
5201            }
5202        }
5203        _ => marks.push((at, class)),
5204    }
5205}
5206
5207/// The class runs `marks` says cover instances `first..end`, relative to
5208/// `first` — the per-draw input [`merge_class_runs`] merges.
5209///
5210/// Every instance of a planned segment was emitted after its draw's mark, so
5211/// the mark at or before `first` names its class; a composite or punch
5212/// instance between two marks is never inside a segment's range.
5213fn marked_runs(
5214    marks: &[(u32, PaintClass)],
5215    first: u32,
5216    end: u32,
5217) -> impl Iterator<Item = ClassRun> + '_ {
5218    let from = marks
5219        .partition_point(|(at, _)| *at <= first)
5220        .saturating_sub(1);
5221    let tail = marks.get(from..).unwrap_or(&[]);
5222    tail.iter()
5223        .enumerate()
5224        .take_while(move |(_, (at, _))| *at < end)
5225        .filter_map(move |(index, (at, class))| {
5226            let start = (*at).max(first);
5227            let stop = tail.get(index + 1).map_or(end, |next| next.0.min(end));
5228            (stop > start).then(|| ClassRun {
5229                first: start - first,
5230                count: stop - start,
5231                class: *class,
5232            })
5233        })
5234}
5235
5236/// Whether `runs` tile `instances` front to back and each run's program draws
5237/// every instance in it, as far as the instance's own flag can tell
5238/// ([`PaintClass::admits_flag`]) — what the marks must agree with, checked in
5239/// debug builds.
5240fn runs_draw(runs: &[ClassRun], instances: &[GpuStrip]) -> bool {
5241    let mut next = 0_u32;
5242    for run in runs {
5243        if run.first != next || run.count == 0 {
5244            return false;
5245        }
5246        let Some(span) = instances.get(run.first as usize..(run.first + run.count) as usize) else {
5247            return false;
5248        };
5249        if !span
5250            .iter()
5251            .all(|instance| run.class.admits_flag(instance.paint_and_rect_flag))
5252        {
5253            return false;
5254        }
5255        next += run.count;
5256    }
5257    next as usize == instances.len()
5258}
5259
5260/// The column of device space one round's target holds: the origin its
5261/// instances are shifted to, and the edges they are clipped against.
5262///
5263/// A page holds its layer at the page's own origin, and its composite samples
5264/// exactly `(0, 0)`-to-its-own-extent back out
5265/// ([`Composite::source`](crate::schedule::Composite::source)). For a layer
5266/// [banded](crate::schedule::pages::page_bands) into column pages that makes
5267/// the band's own rectangle two things at once: the shift, and the *clip*. The
5268/// scheduler replays the layer's whole op list into every band — a band differs
5269/// only in which page it writes and where that page lands — so this is the site
5270/// that decides which part of each replayed strip belongs to the band at hand.
5271/// Emission is where the decision lives because it is the one place that knows
5272/// both the origin and the width; the alternative, clipping the ops
5273/// scheduler-side, would have to re-rasterize geometry the scheduler only holds
5274/// as draw ranges.
5275///
5276/// Two rules, and both halves of each matter:
5277///
5278/// - a span entirely outside the column contributes **no instance** — shifting
5279///   it by a saturating subtraction instead would clamp it onto the page's own
5280///   edge, stretching a span the scene drew elsewhere across content this band
5281///   really holds;
5282/// - a span straddling an edge has its geometry **and** its first alpha column
5283///   ([`GpuStrip::col_idx_or_rect_frac`]) advanced by the same amount, because
5284///   the shader reads that column unshifted and steps one column per pixel of
5285///   the instance (`shaders/strip.wgsl`'s `col_offset`): advancing the x
5286///   without the column would sample another pixel's coverage.
5287///
5288/// The frame's own surface is [`ROOT`](Self::ROOT), a window over the whole
5289/// device grid that shifts nothing and clips nothing. A layer that fits one
5290/// page is one band covering all of it, and a layer's bounds are the
5291/// tile-aligned union of its own draws' bounds, so the clip is a no-op for
5292/// every layer that is not banded.
5293#[derive(Debug, Clone, Copy, PartialEq, Eq)]
5294struct PageWindow {
5295    /// Left edge of the column in device space — what every instance's `x` is
5296    /// shifted by, and what one left of it is culled against.
5297    x0: u16,
5298    /// Right edge of the column in device space, exclusive.
5299    x1: u16,
5300    /// Top edge of the column in device space — what every instance's `y` is
5301    /// shifted by. A band spans its layer's whole height, so the vertical axis
5302    /// is shifted and never clipped.
5303    y0: u16,
5304}
5305
5306impl PageWindow {
5307    /// The whole device grid: the window a surface round carries.
5308    const ROOT: Self = Self {
5309        x0: 0,
5310        x1: u16::MAX,
5311        y0: 0,
5312    };
5313
5314    /// The window `page` renders through — its own tile-aligned bounds, which
5315    /// are one column band's for a banded layer and the whole layer's for
5316    /// every other.
5317    fn of(page: &PageTarget) -> Self {
5318        Self {
5319            x0: page.bounds.x0,
5320            x1: page.bounds.x1,
5321            y0: page.bounds.y0,
5322        }
5323    }
5324
5325    /// The window a retained page's round renders through: the body's own
5326    /// tile-aligned bounds, never banded.
5327    fn of_retained(target: &RetainedTarget) -> Self {
5328        Self {
5329            x0: target.bounds.x0,
5330            x1: target.bounds.x1,
5331            y0: target.bounds.y0,
5332        }
5333    }
5334
5335    /// The origin instances are shifted by.
5336    fn origin(self) -> (u16, u16) {
5337        (self.x0, self.y0)
5338    }
5339
5340    /// Clips `span` — a strip instance still carrying its scene coordinates —
5341    /// to this window and shifts it into the target, answering whether any of
5342    /// it survived.
5343    ///
5344    /// `paint` is the draw's own paint, needed because a left-clipped instance
5345    /// starts at a different scene position than the one it was rasterized at:
5346    /// a position-sampled paint (a gradient, an image) is re-evaluated there,
5347    /// exactly as a gap fill is at its own origin, so the paint keeps landing
5348    /// where the scene put it rather than being squeezed into the clipped span.
5349    fn place(self, span: &mut GpuStrip, paint: PackedPaint) -> bool {
5350        let end = span.x.saturating_add(span.width);
5351        if span.width == 0 || end <= self.x0 || span.x >= self.x1 {
5352            return false;
5353        }
5354
5355        let cut = self.x0.saturating_sub(span.x);
5356        if cut > 0 {
5357            span.x = self.x0;
5358            span.width = span.width.saturating_sub(cut);
5359            // The dense part of an instance starts at its left edge, so the
5360            // columns cut off the geometry are cut off the coverage too.
5361            let dense = cut.min(span.dense_width_or_rect_height);
5362            span.dense_width_or_rect_height = span.dense_width_or_rect_height.saturating_sub(dense);
5363            // A sparse instance names no column at all: the fragment stage
5364            // reads `col + dense_width` as "does this instance sample
5365            // coverage", so leaving a non-zero column on one whose dense part
5366            // is gone would send it to the alpha texture for coverage it never
5367            // wrote.
5368            span.col_idx_or_rect_frac = if span.dense_width_or_rect_height == 0 {
5369                0
5370            } else {
5371                span.col_idx_or_rect_frac.saturating_add(u32::from(dense))
5372            };
5373            span.payload = paint.payload_at(span.x, span.y);
5374        }
5375
5376        // Past the right edge is outside the region the composite samples, so
5377        // it reaches no pixel of the parent either way; clipping it keeps a
5378        // band's instances inside the band's own rectangle rather than relying
5379        // on the page's quantized extent to swallow the overhang.
5380        let over = end.saturating_sub(self.x1);
5381        if over > 0 {
5382            span.width = span.width.saturating_sub(over);
5383            span.dense_width_or_rect_height = span.dense_width_or_rect_height.min(span.width);
5384        }
5385
5386        span.x = span.x.saturating_sub(self.x0);
5387        span.y = span.y.saturating_sub(self.y0);
5388        span.width > 0
5389    }
5390
5391    /// The rect instance drawing `quad` with `paint`, clipped to this window
5392    /// and shifted into its target, or `None` when none of it survives.
5393    ///
5394    /// The same two rules [`Self::place`] keeps, for a whole rectangle: a quad
5395    /// outside the column contributes nothing, and one straddling an edge is
5396    /// trimmed — its atlas texel advanced by exactly the columns cut off its
5397    /// left edge, so every surviving pixel still samples the texel it did. A
5398    /// quad's coverage is its rectangle whole (it lands on whole pixels), so
5399    /// there is no coverage column to advance and the instance carries no edge
5400    /// fractions at all.
5401    fn place_quad(self, quad: &GlyphQuad, paint: PackedPaint) -> Option<GpuStrip> {
5402        let end = quad.x.saturating_add(quad.width);
5403        if quad.width == 0 || quad.height == 0 || end <= self.x0 || quad.x >= self.x1 {
5404            return None;
5405        }
5406        let cut = self.x0.saturating_sub(quad.x);
5407        let x = quad.x.max(self.x0);
5408        let width = end.min(self.x1).saturating_sub(x);
5409        let instance = GpuStrip::from_rect(
5410            x.saturating_sub(self.x0),
5411            quad.y.saturating_sub(self.y0),
5412            width,
5413            quad.height,
5414            0,
5415            StripDraw {
5416                payload: pack_u16_pair(quad.atlas_x.saturating_add(cut), quad.atlas_y),
5417                paint: paint.paint,
5418                depth_index: paint.depth_index,
5419            },
5420        );
5421        (width > 0).then_some(instance)
5422    }
5423}
5424
5425/// The glyph quad the draw at `depth` is, out of a frame's quads in depth
5426/// order — `None` only for a frame whose quads disagree with its draws, which
5427/// then draws nothing for that glyph rather than something wrong.
5428fn glyph_quad(quads: &[GlyphQuad], depth: u32) -> Option<&GlyphQuad> {
5429    let found = quads
5430        .binary_search_by_key(&depth, |quad| quad.depth)
5431        .ok()
5432        .and_then(|index| quads.get(index));
5433    debug_assert!(
5434        found.is_some(),
5435        "a glyph-quad draw with no quad at depth {depth}"
5436    );
5437    found
5438}
5439
5440/// The scheduler roles `plan` gives `frame`'s retained layers, written into
5441/// `roles` (cleared first). A body the store refused gets none and is
5442/// scheduled as the ordinary layer it lowered to.
5443fn retained_roles(frame: &CompiledFrame, plan: &RetainedPlan, roles: &mut RetainedLayers) {
5444    roles.clear();
5445    for (index, (bracket, decision)) in frame.retained.iter().zip(&plan.decisions).enumerate() {
5446        let id = u32::try_from(index).unwrap_or(u32::MAX);
5447        let role = match *decision {
5448            RetainedDecision::Record { .. } => RetainedRole::Record { bracket: id },
5449            RetainedDecision::Draw { dest, origin } => RetainedRole::Draw {
5450                bracket: id,
5451                dest,
5452                origin,
5453            },
5454            RetainedDecision::Hidden => RetainedRole::Hidden,
5455            RetainedDecision::Inline(_) => continue,
5456        };
5457        roles.set(bracket.layer, role);
5458    }
5459}
5460
5461/// Which of `frame`'s retained brackets the schedule renders into a retained
5462/// page — the bodies whose pages the store may commit. A body the store
5463/// admitted can still render nothing, nested inside a layer that composites
5464/// nothing.
5465fn retained_rendered(frame: &CompiledFrame, rounds: &[Round]) -> Vec<bool> {
5466    let mut rendered = vec![false; frame.retained.len()];
5467    for round in rounds {
5468        if let Some(target) = round.retained()
5469            && let Some(slot) = rendered.get_mut(target.bracket as usize)
5470        {
5471            *slot = true;
5472        }
5473    }
5474    rendered
5475}
5476
5477/// The single quad that composites a finished page onto `origin`-shifted
5478/// target.
5479///
5480/// A whole-rectangle instance with no fractional edges: a layer's bounds are
5481/// tile-aligned, so the quad covers whole pixels and the fragment stage leaves
5482/// its coverage at one. The payload is the page texel the quad's top-left
5483/// corner samples — the page holds the layer at its own origin, so that is
5484/// `Composite::source`'s origin and not the layer's device position.
5485fn composite_instance(composite: &Composite, origin: (u16, u16), depth: u32) -> GpuStrip {
5486    let bounds = composite.bounds;
5487    let source = composite.source();
5488
5489    GpuStrip::from_rect(
5490        bounds.x0.saturating_sub(origin.0),
5491        bounds.y0.saturating_sub(origin.1),
5492        bounds.width(),
5493        bounds.height(),
5494        0,
5495        StripDraw {
5496            payload: pack_u16_pair(source.x0, source.y0),
5497            paint: LAYER_PAINT_SOURCE | u32::from(pack_opacity(composite.opacity)),
5498            depth_index: depth,
5499        },
5500    )
5501}
5502
5503/// A layer's constant opacity as the eight bits a composite instance carries.
5504///
5505/// Clamped rather than refused: the scheduler already admits only opacities
5506/// strictly between zero and one, and rounding is what keeps a 0.5 layer at
5507/// exactly the 128 the reference renderer's own packing produces.
5508fn pack_opacity(opacity: f32) -> u8 {
5509    (opacity.clamp(0.0, 1.0) * 255.0).round() as u8
5510}
5511
5512/// Where a strip instance's payload comes from.
5513///
5514/// A solid paint carries its colour there; every other paint reads its colour
5515/// from a record instead, and spends the payload on the scene position the
5516/// paint is sampled at — which is why the payload is per instance rather than
5517/// per draw.
5518#[derive(Debug, Clone, Copy)]
5519enum PaintPayload {
5520    /// A premultiplied RGBA8 colour, the same for every instance of the draw.
5521    Solid(u32),
5522    /// The instance's own scene-space origin, packed as a `u16` pair.
5523    Position,
5524    /// The page texel under the instance's own origin, for a page holding the
5525    /// device region starting at `(x, y)` at its own origin: a backdrop blur's
5526    /// rectangle sampling its blurred page.
5527    Layer(u16, u16),
5528}
5529
5530/// One draw's paint, resolved to the values its instances repeat.
5531#[derive(Debug, Clone, Copy)]
5532struct PackedPaint {
5533    payload: PaintPayload,
5534    /// The packed paint descriptor, without [`RECT_STRIP_FLAG`](crate::gpu::strips::RECT_STRIP_FLAG).
5535    paint: u32,
5536    depth_index: u32,
5537    /// Whether every pixel this paint produces is opaque, and so whether its
5538    /// fully-covered spans may take the depth-writing pass.
5539    opaque: bool,
5540    /// The external-texture slot this paint's instances have to be drawn with
5541    /// bound, or `None` for every paint the frame's atlas binding serves.
5542    ///
5543    /// What breaks a pass's instances into runs: the strip pipelines have one
5544    /// external binding, so consecutive instances may share a segment only
5545    /// while this stays equal (see [`crate::gpu::bindings::ExternalRuns`]).
5546    external: Option<u32>,
5547    /// The strip program class this paint's instances are drawn with — from
5548    /// the resolved record, so an image is told apart by the path it samples
5549    /// through (see [`PaintClass::of_image`]).
5550    class: PaintClass,
5551    /// Whether this is a glyph run's shared record, whose draws are each one
5552    /// glyph quad ([`crate::compile::GlyphQuad`]) rather than a strip run.
5553    glyph_quad: bool,
5554}
5555
5556impl PackedPaint {
5557    /// The payload an instance whose scene origin is `(x, y)` carries.
5558    fn payload_at(self, x: u16, y: u16) -> u32 {
5559        match self.payload {
5560            PaintPayload::Solid(rgba) => rgba,
5561            PaintPayload::Position => pack_u16_pair(x, y),
5562            PaintPayload::Layer(x0, y0) => {
5563                pack_u16_pair(x.saturating_sub(x0), y.saturating_sub(y0))
5564            }
5565        }
5566    }
5567
5568    /// The per-instance values for an instance whose scene origin is `(x, y)`.
5569    fn values_at(self, x: u16, y: u16) -> StripDraw {
5570        StripDraw {
5571            payload: self.payload_at(x, y),
5572            paint: self.paint,
5573            depth_index: self.depth_index,
5574        }
5575    }
5576}
5577
5578/// Two `u16`s in one word, low half first — the packing the shader's
5579/// `unpack_u16_pair` reverses.
5580fn pack_u16_pair(x: u16, y: u16) -> u32 {
5581    u32::from(x) | (u32::from(y) << 16)
5582}
5583
5584/// The shader values for `paint`, resolved against the frame's paint slots.
5585///
5586/// Answers `None` for an indexed paint the frame could not resolve — one whose
5587/// entry is missing from `paint_slots` entirely, or one the lowering refused
5588/// because the engine has nowhere to hold its pixels yet (see the module
5589/// header). The draw is then skipped rather than stamped in a wrong colour.
5590fn pack_paint(
5591    paint: &Paint,
5592    depth: u32,
5593    paint_slots: &[Option<ResolvedPaint>],
5594) -> Option<PackedPaint> {
5595    match paint {
5596        Paint::Solid(color) => Some(PackedPaint {
5597            payload: PaintPayload::Solid(color.as_premul_rgba8().to_u32()),
5598            paint: SOLID_PAINT,
5599            depth_index: depth,
5600            opaque: color.is_opaque(),
5601            external: None,
5602            class: PaintClass::SolidCoverage,
5603            glyph_quad: false,
5604        }),
5605        Paint::Indexed(indexed) => {
5606            let Some(Some(resolved)) = paint_slots.get(indexed.index()) else {
5607                INDEXED_PAINT_WARNING.call_once(|| {
5608                    log::warn!(
5609                        "an indexed paint could not be lowered to a GPU record; those draws are \
5610                         skipped (logged once)"
5611                    );
5612                });
5613                return None;
5614            };
5615            Some(PackedPaint {
5616                payload: PaintPayload::Position,
5617                paint: pack_paint_descriptor(resolved.paint_type, resolved.texel_offset),
5618                depth_index: depth,
5619                opaque: resolved.opaque,
5620                external: resolved.external,
5621                class: resolved.class,
5622                glyph_quad: resolved.paint_type == PaintType::Glyph,
5623            })
5624        }
5625    }
5626}
5627
5628/// One encoded paint after lowering: where its record landed and how a strip
5629/// instance names it.
5630#[derive(Debug, Clone, Copy)]
5631struct ResolvedPaint {
5632    /// How the fragment shader reads the record.
5633    paint_type: PaintType,
5634    /// The texel the record starts at in the encoded-paint texture.
5635    texel_offset: u32,
5636    /// Whether every pixel the paint produces is opaque.
5637    opaque: bool,
5638    /// The external-texture slot this paint samples, for a paint that reads a
5639    /// caller-owned texture rather than the atlas (see
5640    /// [`crate::gpu::bindings`]). `None` — the ordinary case — means the
5641    /// frame's atlas binding serves it.
5642    external: Option<u32>,
5643    /// The strip program class the paint is drawn with ([`class_of_record`]).
5644    class: PaintClass,
5645}
5646
5647/// The strip program class a lowered record is drawn with.
5648///
5649/// The paint type decides it for every record but an image, whose record also
5650/// names the path it samples through: nearest or bilinear from the atlas is
5651/// [`PaintClass::Image`], bicubic or the external texture
5652/// [`PaintClass::ImageFiltered`]. The bits are the ones the shader reads —
5653/// quality in bits 0-1 of `image_params`, the source kind in bit 14.
5654fn class_of_record(record: &GpuEncodedPaint) -> PaintClass {
5655    match record {
5656        GpuEncodedPaint::Image(image) => PaintClass::of_image(
5657            image.image_params & IMAGE_QUALITY_BITS,
5658            (image.image_params >> IMAGE_SOURCE_KIND_SHIFT) & 1 == 1,
5659        ),
5660        _ => PaintClass::of_flag(pack_paint_descriptor(record.paint_type(), 0)),
5661    }
5662}
5663
5664/// Bits 0-1 of an image record's `image_params`: the sampling quality.
5665const IMAGE_QUALITY_BITS: u32 = 0b11;
5666
5667/// The bit of an image record's `image_params` marking an external texture.
5668const IMAGE_SOURCE_KIND_SHIFT: u32 = 14;
5669
5670/// One resource texture and the extent it currently holds.
5671#[derive(Debug)]
5672struct ResourceTexture {
5673    texture: wgpu::Texture,
5674    view: wgpu::TextureView,
5675    width: u32,
5676    height: u32,
5677    /// The rows, counted from the top, that may differ from what a
5678    /// full-footprint upload of the next frame would leave there.
5679    ///
5680    /// A queue upload writes the whole texture every frame: the frame's bytes,
5681    /// then the upload buffer's tail — zeroes for coverage, the reused staging
5682    /// vector's earlier records for encoded paints. A staged upload writes only
5683    /// the rows the frame fills plus every row the last frame left live, and
5684    /// so leaves the texture byte-for-byte as the full upload would: the rows
5685    /// below this mark already hold exactly that tail. Rows at or beyond the
5686    /// current frame's filled rows are never read, so a stale row there (a
5687    /// dropped encoder leaves its rows past `filled` as they were) cannot reach
5688    /// a pixel. A new texture is live throughout, so its first upload is a full
5689    /// one.
5690    live_rows: u32,
5691}
5692
5693impl ResourceTexture {
5694    fn new(device: &wgpu::Device, descriptor: &wgpu::TextureDescriptor<'_>) -> Self {
5695        let texture = device.create_texture(descriptor);
5696        let view = texture.create_view(&wgpu::TextureViewDescriptor::default());
5697        Self {
5698            texture,
5699            view,
5700            width: descriptor.size.width,
5701            height: descriptor.size.height,
5702            live_rows: descriptor.size.height,
5703        }
5704    }
5705
5706    /// The rows a frame whose bytes fill `filled_rows` rows uploads through a
5707    /// stage: those, and every row the last upload left live (see
5708    /// [`Self::live_rows`]), never past the texture's height.
5709    fn staged_rows(&self, filled_rows: u32) -> u32 {
5710        staged_rows(filled_rows, self.live_rows, self.height)
5711    }
5712
5713    /// The extent of an upload of the texture's top `rows` rows.
5714    fn rows_extent(&self, rows: u32) -> wgpu::Extent3d {
5715        wgpu::Extent3d {
5716            width: self.width,
5717            height: rows,
5718            depth_or_array_layers: 1,
5719        }
5720    }
5721
5722    /// The whole texture as a copy destination.
5723    fn copy_target(&self) -> wgpu::TexelCopyTextureInfo<'_> {
5724        wgpu::TexelCopyTextureInfo {
5725            texture: &self.texture,
5726            mip_level: 0,
5727            origin: wgpu::Origin3d::ZERO,
5728            aspect: wgpu::TextureAspect::All,
5729        }
5730    }
5731
5732    /// The extent one full-texture upload covers.
5733    fn extent(&self) -> wgpu::Extent3d {
5734        wgpu::Extent3d {
5735            width: self.width,
5736            height: self.height,
5737            depth_or_array_layers: 1,
5738        }
5739    }
5740}
5741
5742/// The GPU resources one renderer holds across frames.
5743#[derive(Debug)]
5744struct FrameResources {
5745    alphas: ResourceTexture,
5746    paints: ResourceTexture,
5747    gradients: ResourceTexture,
5748    /// Stand-ins for the strip shader's bindings a given pass has nothing real
5749    /// for: the layer input (a real page when a composite is being drawn, this
5750    /// otherwise), the glyph/image atlas array until the first image is
5751    /// resident, and an externally bound texture, which nothing writes yet.
5752    /// Every declared binding has to be bound for a pass to validate, whether
5753    /// or not an instance samples it.
5754    placeholders: Placeholders,
5755    /// The linear, clamp-to-edge sampler the image-sampler program family
5756    /// filters bilinear atlas images through, bound beside the atlas array in
5757    /// every strip group 1; `None` for the manual family, whose layout
5758    /// declares no sampler (see [`EngineRenderer::with_image_sampler`]).
5759    atlas_sampler: Option<wgpu::Sampler>,
5760    config: wgpu::Buffer,
5761    /// One viewport uniform per page round of the busiest frame so far.
5762    ///
5763    /// A round's NDC mapping is against the extent of the attachment it writes,
5764    /// and a page's extent is neither the frame's nor the same from one round
5765    /// to the next, so each needs a buffer of its own — a bind group holds the
5766    /// whole buffer, not an offset into one. Grown only, like every other
5767    /// retained resource here.
5768    page_configs: Vec<wgpu::Buffer>,
5769    instances: Option<wgpu::Buffer>,
5770    instance_capacity: u64,
5771    /// The GPU records this frame's indexed paints resolve against, in
5772    /// serialization order — which is the order the texel offsets in
5773    /// `paint_slots` were taken from.
5774    paints_data: Vec<GpuEncodedPaint>,
5775    /// One slot per encoded paint of the frame, indexed by
5776    /// [`Paint::Indexed`](vello_common::paint::Paint::Indexed): `None` for one
5777    /// the engine could not lower. Kept parallel to the *encoded* paints
5778    /// rather than compacted to the lowered ones, because a draw names its
5779    /// paint by the compiler's index.
5780    paint_slots: Vec<Option<ResolvedPaint>>,
5781    /// Reusable per-paint ramp residency, filled from a frame's LUT requests
5782    /// before its paints are lowered.
5783    paint_ramps: Vec<Option<CachedRamp>>,
5784    /// Reusable staging for the encoded-paint upload, padded to the texture's
5785    /// footprint.
5786    paint_staging: Vec<u8>,
5787    /// The distinct external textures this frame's paints sample, in the order
5788    /// they were first named — the slot numbering the pass segments and the
5789    /// group-1 bind groups both address by.
5790    external_runs: ExternalRuns,
5791    /// One set of bind groups per strip render state, built against any one
5792    /// of that state's class pipelines. Every strip pipeline — each state,
5793    /// each paint class — is built against one explicit layout
5794    /// ([`crate::gpu::bindings::strip_pipeline_layout`]), so a state's set binds
5795    /// under all of its classes; keeping one set per state is what the map
5796    /// already did when each pipeline derived a layout of its own.
5797    bind_groups: HashMap<EnginePipeline, StripBindGroups>,
5798    /// The target format the live bind groups were built against; a change
5799    /// means different pipelines, so the whole map is dropped rather than
5800    /// accumulating a set per format ever rendered to.
5801    bind_group_format: Option<wgpu::TextureFormat>,
5802    /// The real image atlas array, created lazily by the first frame that
5803    /// makes an image resident and grown from then on — `None` is exactly
5804    /// [`Placeholders::atlas_array`]'s domain, a renderer that has never
5805    /// drawn an image.
5806    atlas: Option<AtlasArray>,
5807    /// The atlas rectangle every image the renderer has ever drawn currently
5808    /// holds, keyed by the stable [`ImageId`] the compiler's own residency
5809    /// minted it. See [`FrameResources::resolve_paints`] for how this is
5810    /// kept in step with residency across frames without reaching into the
5811    /// compiler's own cache.
5812    image_registry: HashMap<ImageId, ResidentImage>,
5813    /// Reused zero fill for the atlas eviction clears.
5814    clear_zeros: Vec<u8>,
5815    /// Group-0 and filter-source bind groups naming a page, kept across frames
5816    /// and swept at every frame's end (see [`SourceGroupCache`]). Cleared with
5817    /// the coverage texture they all name.
5818    source_groups: SourceGroupCache,
5819    /// Strip, external and image bind groups built by the group-set paths.
5820    groups_created: u64,
5821    /// How many atlas regions have been declined — a write or clear the array
5822    /// refused, or one the budget says the array could not hold. Counted rather
5823    /// than dropped silently, because each one is a region the frame believed
5824    /// it had filled.
5825    refused_regions: u64,
5826}
5827
5828impl FrameResources {
5829    fn new(device: &wgpu::Device, dim: u32, image_sampler: bool) -> Self {
5830        let min = gpu::MIN_RESOURCE_TEXTURE_HEIGHT;
5831        Self {
5832            alphas: ResourceTexture::new(device, &gpu::alpha_texture_descriptor(dim, min)),
5833            paints: ResourceTexture::new(
5834                device,
5835                &gpu::paint_texture::encoded_paints_texture_descriptor(dim, min),
5836            ),
5837            gradients: ResourceTexture::new(device, &gradient_texture_descriptor(dim, min)),
5838            placeholders: Placeholders::new(device),
5839            atlas_sampler: image_sampler.then(|| atlas_sampler(device)),
5840            config: device.create_buffer(&config_descriptor("frust-engine config uniform")),
5841            page_configs: Vec::new(),
5842            instances: None,
5843            instance_capacity: 0,
5844            paints_data: Vec::new(),
5845            paint_slots: Vec::new(),
5846            paint_ramps: Vec::new(),
5847            paint_staging: Vec::new(),
5848            external_runs: ExternalRuns::new(),
5849            bind_groups: HashMap::new(),
5850            bind_group_format: None,
5851            atlas: None,
5852            image_registry: HashMap::new(),
5853            clear_zeros: Vec::new(),
5854            source_groups: SourceGroupCache::default(),
5855            groups_created: 0,
5856            refused_regions: 0,
5857        }
5858    }
5859
5860    /// Drops the atlas array, everything recorded about what lives in it, and
5861    /// the bind groups naming it.
5862    ///
5863    /// What a re-budget needs: the rectangles the registry holds were allocated
5864    /// in a geometry that no longer exists, so keeping any of them would point
5865    /// a paint at a rectangle of a texture that is gone.
5866    fn reset_atlas(&mut self) {
5867        self.atlas = None;
5868        self.image_registry.clear();
5869        self.bind_groups.clear();
5870    }
5871
5872    /// Count `regions` atlas regions as declined, saying so once.
5873    ///
5874    /// Once, not per region: a budget and an array that disagree disagree about
5875    /// every region, and a per-frame line would bury the fact under itself. The
5876    /// count on [`EngineRenderer::refused_atlas_regions`] is the measure.
5877    fn note_refused_regions(&mut self, regions: u64) {
5878        self.refused_regions = self.refused_regions.saturating_add(regions);
5879        ATLAS_REFUSAL_WARNING.call_once(|| {
5880            log::warn!(
5881                "an atlas region was refused by the image atlas array; those images are skipped \
5882                 and their uploads re-offered on a later frame (logged once — see \
5883                 EngineRenderer::refused_atlas_regions for the count)"
5884            );
5885        });
5886    }
5887
5888    /// Services `frame`'s LUT and image residency, then lowers its encoded
5889    /// paints into the records the shader samples.
5890    ///
5891    /// Leaves `paints_data` holding the lowered records in serialization order
5892    /// and `paint_slots` naming, per *encoded* paint, the texel its record
5893    /// starts at — or `None` where the paint could not be lowered.
5894    ///
5895    /// A solid-only frame leaves both empty and touches neither the cache nor
5896    /// the paint texture, so it costs exactly what it did before paints were
5897    /// wired up. Image residency is still serviced even then, since an image
5898    /// can be evicted on a frame that draws nothing at all (see
5899    /// [`Self::update_image_registry`]).
5900    fn resolve_paints(
5901        &mut self,
5902        frame: &CompiledFrame,
5903        cache: &mut GradientCache,
5904        budget: AtlasBudget,
5905        externals: &ExternalTextures,
5906    ) {
5907        self.paints_data.clear();
5908        self.paint_slots.clear();
5909        self.external_runs.clear();
5910
5911        // Kept in step every frame, not only when this frame's own paints
5912        // need it: an image reaped by the compiler's age-based eviction while
5913        // nothing draws it must still be forgotten here, or a later draw that
5914        // reuses its freed rectangle's `ImageId` would read the stale entry.
5915        self.update_image_registry(frame, budget);
5916
5917        if frame.encoded_paints.is_empty() {
5918            return;
5919        }
5920
5921        // Ramp residency next, for the whole frame: a ramp's offset is only
5922        // meaningful once the cache has finished baking this frame's misses,
5923        // and a record built before that would name a ramp that had not been
5924        // packed yet.
5925        self.paint_ramps.clear();
5926        self.paint_ramps.resize(frame.encoded_paints.len(), None);
5927        for request in &frame.lut_requests {
5928            let ramp = resolve_lut_request(*request, &frame.encoded_paints, cache);
5929            if let Some(slot) = self.paint_ramps.get_mut(request.paint_index) {
5930                *slot = ramp;
5931            }
5932        }
5933
5934        let mut texel_offset = 0;
5935        // Ascending, like the walk below, so one cursor marks them.
5936        let mut glyph_records = frame.glyph_run_paints.iter().copied().peekable();
5937        for (index, paint) in frame.encoded_paints.iter().enumerate() {
5938            // A glyph run's shared record: lowered like any atlas image, but
5939            // drawn by glyph quads (see `crate::compile::GlyphQuad`).
5940            let glyph_record = glyph_records.next_if_eq(&index).is_some();
5941            // An external texture's slot travels with its record: the record
5942            // itself only says "sample the external binding", and which
5943            // texture that binding holds is settled per run when the pass is
5944            // recorded rather than per paint.
5945            let mut external = None;
5946            let lowered = match paint {
5947                EncodedPaint::Image(image) => image_id(image)
5948                    .and_then(|id| self.image_registry.get(&id))
5949                    .and_then(|resident| {
5950                        lower_encoded_image(image, resident)
5951                            .map(|record| fit_minified(record, resident))
5952                    }),
5953                // A paint naming a texture nothing is bound under lowers to
5954                // nothing, so its draws are skipped rather than sampling
5955                // whichever texture the binding happens to hold.
5956                EncodedPaint::ExternalTexture(entry) => externals
5957                    .view(entry.texture_id.0)
5958                    .and_then(|_| self.external_runs.slot_of(entry.texture_id.0))
5959                    .map(|slot| {
5960                        external = Some(slot);
5961                        lower_encoded_external(entry)
5962                    }),
5963                _ => {
5964                    let ramp = self.paint_ramps.get(index).copied().flatten();
5965                    lower_encoded_paint(paint, ramp)
5966                }
5967            };
5968            let slot = lowered.map(|record| {
5969                let resolved = ResolvedPaint {
5970                    paint_type: if glyph_record {
5971                        PaintType::Glyph
5972                    } else {
5973                        record.paint_type()
5974                    },
5975                    texel_offset,
5976                    opaque: !paint.may_have_transparency(),
5977                    external,
5978                    class: if glyph_record {
5979                        PaintClass::Glyph
5980                    } else {
5981                        class_of_record(&record)
5982                    },
5983                };
5984                texel_offset += record.texel_len();
5985                self.paints_data.push(record);
5986                resolved
5987            });
5988            self.paint_slots.push(slot);
5989        }
5990    }
5991
5992    /// Keeps [`Self::image_registry`] in step with the compiler's own image
5993    /// residency, without reaching into it: the residency's rectangles are
5994    /// not reachable from here (see [`crate::gpu::paint_texture::lower_encoded_paint`]'s
5995    /// doc for why), so this reconstructs the same information from what a
5996    /// compiled frame already reports.
5997    ///
5998    /// Two passes, and each reads the frame's plan directly rather than
5999    /// inferring anything from draw order. First, every region this frame's
6000    /// residency reaped is forgotten — `frame.image_forgets` names it by
6001    /// rectangle, and a rectangle uniquely identifies the one image that held it
6002    /// (padding is always zero in this engine, so the reported and the stored
6003    /// rectangle are the same value; see [`crate::cache::images`]'s module doc).
6004    /// Second, every entry of `frame.image_uploads` is registered under the
6005    /// [`ImageId`] it carries.
6006    ///
6007    /// The order matters and the id does. Evictions run first so a same-frame
6008    /// evict-then-reallocate that reuses a rectangle registers the new tenant
6009    /// rather than having it removed again. And the upload naming its own id is
6010    /// what makes this sound under a *re-offered* plan: an upload the previous
6011    /// frame did not service is reported again alongside no new draw of its own,
6012    /// so a walk pairing uploads positionally against this frame's encoded
6013    /// paints would hand a fresh image the stale upload's rectangle.
6014    ///
6015    /// A region the atlas array could not hold at `budget`'s geometry and this
6016    /// frame's depth is counted and left unregistered instead. Its draws are
6017    /// then skipped, which is the whole point: a registered rectangle nothing
6018    /// wrote would be sampled as whatever the texture happened to contain.
6019    fn update_image_registry(&mut self, frame: &CompiledFrame, budget: AtlasBudget) {
6020        if !frame.image_forgets.is_empty() {
6021            // A set rather than `Vec::contains`: both sides of this scan scale
6022            // with content — the registry with how many images and glyph slots
6023            // are resident, the plan with how hard the atlas is churning — and
6024            // their product is the frame path's, not a report's.
6025            let cleared: HashSet<AtlasRegion> = frame.image_forgets.iter().copied().collect();
6026            self.image_registry
6027                .retain(|_, resident| !cleared.contains(&resident.region));
6028        }
6029
6030        let mut refused = 0_u64;
6031        for upload in &frame.image_uploads {
6032            if !budget.contains(upload.region, frame.atlas_layers) {
6033                refused = refused.saturating_add(1);
6034                continue;
6035            }
6036            self.image_registry.insert(
6037                upload.id,
6038                ResidentImage {
6039                    id: upload.id,
6040                    region: upload.region,
6041                    natural: upload.natural,
6042                    padding: u32::from(ATLAS_PADDING),
6043                    may_have_transparency: upload.may_have_transparency,
6044                },
6045            );
6046        }
6047        // The glyph half of the same registry, and the reason it is a second
6048        // loop rather than a branch inside the first: a glyph slot carries no
6049        // pixels and is *not* re-offered across frames, because the pixels are
6050        // produced by the replay pass rather than uploaded from here. Its
6051        // rectangle is reported by every draw that names it (see
6052        // [`crate::compile::GlyphSlot`]), so registering it here is what makes
6053        // a handle `glifo` recycled resolve against its current occupant.
6054        for slot in &frame.glyph_slots {
6055            if !budget.contains(slot.region, frame.atlas_layers) {
6056                refused = refused.saturating_add(1);
6057                continue;
6058            }
6059            self.image_registry.insert(
6060                slot.id,
6061                ResidentImage {
6062                    id: slot.id,
6063                    region: slot.region,
6064                    // Never minified: a glyph is rasterized straight into the
6065                    // rectangle it was allocated, so the natural extent and the
6066                    // resident one are the same value by construction.
6067                    natural: slot.region.size,
6068                    padding: slot.padding,
6069                    may_have_transparency: true,
6070                },
6071            );
6072        }
6073
6074        if refused > 0 {
6075            self.note_refused_regions(refused);
6076        }
6077    }
6078
6079    /// Grows or creates the image atlas array to hold `layers` layers at
6080    /// `budget`'s per-layer extent, clearing the live bind groups when it
6081    /// does — a bind group built against the old (or absent) atlas view would
6082    /// otherwise sample nothing, or a freed texture.
6083    ///
6084    /// A frame that has never made an image resident (`layers == 0`) leaves
6085    /// the atlas unset, so the strip shader's binding stays on
6086    /// [`Placeholders::atlas_array`] until the first one is.
6087    ///
6088    /// Growth submits a maintenance command buffer of its own rather than
6089    /// recording into the frame's encoder — see [`crate::gpu::atlas`]. So this
6090    /// must be called after the frame's last fallible step (a refused frame
6091    /// must submit nothing) and before its atlas writes are issued (the copy has
6092    /// to precede them, and a submit flushes whatever is already queued).
6093    fn ensure_atlas(
6094        &mut self,
6095        device: &wgpu::Device,
6096        queue: &wgpu::Queue,
6097        budget: AtlasBudget,
6098        layers: u32,
6099    ) -> bool {
6100        let grew = match &mut self.atlas {
6101            None if layers == 0 => false,
6102            None => {
6103                self.atlas = Some(AtlasArray::with_layers(
6104                    device,
6105                    budget.atlas_size.0,
6106                    budget.atlas_size.1,
6107                    layers,
6108                ));
6109                true
6110            }
6111            Some(atlas) => atlas.ensure_layers(device, queue, layers),
6112        };
6113        if grew {
6114            self.bind_groups.clear();
6115        }
6116        grew
6117    }
6118
6119    /// Flushes `frame`'s atlas evictions and uploads against the live atlas
6120    /// array, evictions first — a rectangle this frame's residency freed may
6121    /// already hold a fresh upload by the time this runs (see
6122    /// [`crate::cache::images`]'s module doc), so clearing after writing
6123    /// would erase the image that just moved in.
6124    ///
6125    /// Answers whether the whole plan reached the array — which is what tells
6126    /// the compiler it may stop re-offering it. A region the array declined is
6127    /// counted (see [`Self::note_refused_regions`]) and the plan stays pending,
6128    /// so a later frame that has grown the array writes it rather than the
6129    /// image being lost.
6130    ///
6131    /// An absent array with a plan to service is that same disagreement rather
6132    /// than a quiet no-op: both are driven by the same `frame.atlas_layers`, so
6133    /// an empty plan and an absent atlas normally agree.
6134    fn upload_atlas(&mut self, queue: &wgpu::Queue, frame: &CompiledFrame) -> bool {
6135        let refused = match self.atlas.as_ref() {
6136            None => (frame.image_evictions.len() + frame.image_uploads.len()) as u64,
6137            Some(atlas) => {
6138                let mut refused = 0_u64;
6139                for region in &frame.image_evictions {
6140                    if !atlas.clear_region_with(queue, *region, &mut self.clear_zeros) {
6141                        refused = refused.saturating_add(1);
6142                    }
6143                }
6144                for upload in &frame.image_uploads {
6145                    if !atlas.write_region(queue, upload.region, upload.pixels.data_as_u8_slice()) {
6146                        refused = refused.saturating_add(1);
6147                    }
6148                }
6149                refused
6150            }
6151        };
6152
6153        if refused == 0 {
6154            return true;
6155        }
6156        self.note_refused_regions(refused);
6157        false
6158    }
6159
6160    /// Makes sure there is one viewport uniform per page round of this frame.
6161    ///
6162    /// Grown only: the buffers are 32 bytes each and a frame that once needed
6163    /// four keeps them rather than reallocating on the next frame that does.
6164    fn ensure_page_configs(&mut self, device: &wgpu::Device, rounds: usize) {
6165        while self.page_configs.len() < rounds {
6166            self.page_configs
6167                .push(device.create_buffer(&config_descriptor("frust-engine page config uniform")));
6168        }
6169    }
6170
6171    /// The texels this frame's encoded paints occupy.
6172    fn paint_texels(&self) -> u32 {
6173        self.paints_data
6174            .iter()
6175            .map(GpuEncodedPaint::texel_len)
6176            .sum()
6177    }
6178
6179    /// The height the gradient LUT texture must grow to for every ramp the
6180    /// cache has packed, or `None` when it already fits.
6181    ///
6182    /// # Errors
6183    ///
6184    /// [`EngineError::PaintCapacity`]: a LUT set past the resource dimension
6185    /// squared has nowhere to live, and the frame path reports it rather than
6186    /// asserting.
6187    fn grown_gradient_height(&self, cache: &GradientCache) -> Result<Option<u32>, EngineError> {
6188        let width = self.gradients.width;
6189        let texels = u32::try_from(cache.luts_size() / BYTES_PER_TEXEL as usize)
6190            .map_err(|_| EngineError::PaintCapacity)?;
6191        let required = texels.div_ceil(width).max(gpu::MIN_RESOURCE_TEXTURE_HEIGHT);
6192        if required > width {
6193            return Err(EngineError::PaintCapacity);
6194        }
6195        Ok((required > self.gradients.height).then_some(required))
6196    }
6197
6198    fn resize_alphas(&mut self, device: &wgpu::Device, height: Option<u32>) {
6199        if let Some(height) = height {
6200            self.alphas = ResourceTexture::new(
6201                device,
6202                &gpu::alpha_texture_descriptor(self.alphas.width, height),
6203            );
6204            self.bind_groups.clear();
6205            // Every page group names the old coverage view.
6206            self.source_groups.clear();
6207        }
6208    }
6209
6210    fn resize_paints(&mut self, device: &wgpu::Device, height: Option<u32>) {
6211        if let Some(height) = height {
6212            self.paints = ResourceTexture::new(
6213                device,
6214                &gpu::paint_texture::encoded_paints_texture_descriptor(self.paints.width, height),
6215            );
6216            self.bind_groups.clear();
6217        }
6218    }
6219
6220    fn resize_gradients(&mut self, device: &wgpu::Device, height: Option<u32>) {
6221        if let Some(height) = height {
6222            self.gradients = ResourceTexture::new(
6223                device,
6224                &gradient_texture_descriptor(self.gradients.width, height),
6225            );
6226            self.bind_groups.clear();
6227        }
6228    }
6229
6230    /// Uploads the frame's coverage, paint records, colour ramps, atlas
6231    /// evictions/uploads and config, answering whether the atlas plan was
6232    /// serviced in full (see [`Self::upload_atlas`]).
6233    ///
6234    /// The per-frame uploads — coverage, paint records, config — go through
6235    /// `stage` when the frame has one, as copies recorded into `encoder` ahead
6236    /// of every pass; the on-change ones (colour ramps, atlas regions) stay
6237    /// queue writes, which is also what every one of them is without a stage.
6238    #[expect(
6239        clippy::too_many_arguments,
6240        reason = "the upload's two routes (queue, encoder + stage) beside the \
6241                  frame, the ramp cache and the extents it uploads against"
6242    )]
6243    fn upload(
6244        &mut self,
6245        queue: &wgpu::Queue,
6246        encoder: &mut wgpu::CommandEncoder,
6247        mut stage: Option<&mut StagingFrame>,
6248        frame: &mut CompiledFrame,
6249        cache: &mut GradientCache,
6250        size: (u16, u16),
6251        dim: u32,
6252    ) -> bool {
6253        let atlas_serviced = self.upload_atlas(queue, frame);
6254
6255        let alphas = &self.alphas;
6256        let bytes_per_row = gpu::resource_bytes_per_row(alphas.width);
6257        let coverage = &frame.strips.alphas;
6258        // Staged, only the rows this frame's coverage fills and the rows the
6259        // last frame left live are written — the rest of the texture already
6260        // holds the zeroes a full upload would put there.
6261        let filled = rows_filled(coverage.len(), bytes_per_row).min(alphas.height);
6262        let rows = alphas.staged_rows(filled);
6263        let staged = stage.as_deref_mut().is_some_and(|stage| {
6264            rows == 0
6265                || stage.copy_to_texture(
6266                    encoder,
6267                    alphas.copy_target(),
6268                    bytes_per_row,
6269                    rows,
6270                    alphas.rows_extent(rows),
6271                    |dst| gpu::write_zero_padded(dst, coverage),
6272                )
6273        });
6274        if !staged {
6275            gpu::with_padded_alphas(
6276                &mut frame.strips.alphas,
6277                alphas.width,
6278                alphas.height,
6279                |bytes| {
6280                    queue.write_texture(
6281                        alphas.copy_target(),
6282                        bytes,
6283                        resource_layout(bytes_per_row, alphas.height),
6284                        alphas.extent(),
6285                    );
6286                },
6287            );
6288        }
6289        self.alphas.live_rows = filled;
6290
6291        if !self.paints_data.is_empty() {
6292            let footprint = gpu::resource_texture_bytes(self.paints.width, self.paints.height);
6293            self.paint_staging
6294                .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
6295            // The height was checked to fit before any pass was recorded, so a
6296            // buffer too short for the records is unreachable; skipping the
6297            // upload rather than unwrapping keeps the frame path total anyway.
6298            if let Ok(len) =
6299                GpuEncodedPaint::serialize_to_buffer(&self.paints_data, &mut self.paint_staging)
6300            {
6301                let paints = &self.paints;
6302                let staging = &self.paint_staging;
6303                let bytes_per_row = gpu::resource_bytes_per_row(paints.width);
6304                // The rows below the live mark already hold the staging
6305                // vector's own tail, which is what a full upload rewrites.
6306                let filled = rows_filled(len, bytes_per_row).min(paints.height);
6307                let rows = paints.staged_rows(filled);
6308                let staged = stage.as_deref_mut().is_some_and(|stage| {
6309                    rows == 0
6310                        || stage.copy_to_texture(
6311                            encoder,
6312                            paints.copy_target(),
6313                            bytes_per_row,
6314                            rows,
6315                            paints.rows_extent(rows),
6316                            |dst| gpu::write_zero_padded(dst, staging),
6317                        )
6318                });
6319                if !staged {
6320                    queue.write_texture(
6321                        paints.copy_target(),
6322                        staging,
6323                        resource_layout(bytes_per_row, paints.height),
6324                        paints.extent(),
6325                    );
6326                }
6327                self.paints.live_rows = filled;
6328            }
6329        }
6330
6331        if cache.has_changed() {
6332            let layout = GradientTextureLayout {
6333                width: self.gradients.width,
6334                height: self.gradients.height,
6335            };
6336            if let Some(upload) = cache.begin_upload(layout) {
6337                queue.write_texture(
6338                    self.gradients.copy_target(),
6339                    &upload,
6340                    resource_layout(upload.bytes_per_row(), self.gradients.height),
6341                    self.gradients.extent(),
6342                );
6343            }
6344            cache.mark_synced();
6345        }
6346
6347        // The strip program addresses all three resource textures by the
6348        // config's width shifts alone — the LUT by the paint texture's — so
6349        // the config built from `dim` is only sound while each of them really
6350        // is `dim` wide (see `gpu::config`'s module doc). Every resize above
6351        // grows height only, so this holds by construction; the assertion is
6352        // what notices a change that breaks it.
6353        debug_assert!(
6354            resource_widths_agree(
6355                dim,
6356                self.alphas.width,
6357                self.paints.width,
6358                self.gradients.width
6359            ),
6360            "strip resource textures must all be resource_texture_dim ({dim}) wide: \
6361             alphas {}, paints {}, gradients {}",
6362            self.alphas.width,
6363            self.paints.width,
6364            self.gradients.width,
6365        );
6366        let config = GpuConfig::new(u32::from(size.0), u32::from(size.1), dim, dim);
6367        let config = bytemuck::bytes_of(&config);
6368        let staged =
6369            stage.is_some_and(|stage| stage.copy_to_buffer(encoder, config, &self.config, 0));
6370        if !staged {
6371            queue.write_buffer(&self.config, 0, config);
6372        }
6373
6374        atlas_serviced
6375    }
6376
6377    /// Grows the instance buffer if this frame outgrew it, then uploads the
6378    /// opaque and alpha instances back to back — as one staged copy when the
6379    /// frame has a stage, two queue writes otherwise.
6380    fn upload_instances(
6381        &mut self,
6382        device: &wgpu::Device,
6383        queue: &wgpu::Queue,
6384        encoder: &mut wgpu::CommandEncoder,
6385        stage: Option<&mut StagingFrame>,
6386        scratch: &Scratch,
6387    ) {
6388        let (opaque, alpha) = scratch.instance_bytes();
6389        let required = (opaque.len() + alpha.len()) as u64;
6390        if required == 0 {
6391            return;
6392        }
6393
6394        let buffer = match &mut self.instances {
6395            Some(buffer) if self.instance_capacity >= required => buffer,
6396            slot => {
6397                let floor = MIN_INSTANCE_CAPACITY * size_of::<GpuStrip>() as u64;
6398                let capacity = required
6399                    .checked_next_power_of_two()
6400                    .unwrap_or(required)
6401                    .max(floor);
6402                self.instance_capacity = capacity;
6403                slot.insert(device.create_buffer(&wgpu::BufferDescriptor {
6404                    label: Some("frust-engine strip instances"),
6405                    size: capacity,
6406                    usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
6407                    mapped_at_creation: false,
6408                }))
6409            }
6410        };
6411        if let Some(stage) = stage
6412            && stage.copy_parts_to_buffer(encoder, &[opaque, alpha], buffer, 0)
6413        {
6414            return;
6415        }
6416        if !opaque.is_empty() {
6417            queue.write_buffer(buffer, 0, opaque);
6418        }
6419        if !alpha.is_empty() {
6420            queue.write_buffer(buffer, opaque.len() as u64, alpha);
6421        }
6422    }
6423
6424    /// Builds `variant`'s bind groups if it has none, first dropping every
6425    /// set when the target format changed under them.
6426    fn ensure_bind_groups(
6427        &mut self,
6428        device: &wgpu::Device,
6429        variant: EnginePipeline,
6430        pipeline: &wgpu::RenderPipeline,
6431        format: wgpu::TextureFormat,
6432    ) {
6433        if self.bind_group_format != Some(format) {
6434            self.bind_groups.clear();
6435            self.bind_group_format = Some(format);
6436        }
6437        if self.bind_groups.contains_key(&variant) {
6438            return;
6439        }
6440        let atlas_view = self
6441            .atlas
6442            .as_ref()
6443            .map(AtlasArray::view)
6444            .unwrap_or(&self.placeholders.atlas_array);
6445        let groups = StripBindGroups::new(
6446            device,
6447            pipeline,
6448            &self.alphas.view,
6449            &self.config,
6450            &self.placeholders,
6451            atlas_view,
6452            self.atlas_sampler.as_ref(),
6453            &self.paints.view,
6454            &self.gradients.view,
6455        );
6456        self.bind_groups.insert(variant, groups);
6457        self.groups_created = self.groups_created.saturating_add(4);
6458    }
6459
6460    /// Builds `variant`'s group 1 for every external texture this frame draws
6461    /// that it has none for yet.
6462    ///
6463    /// Called after [`Self::ensure_bind_groups`] has built the variant's own
6464    /// set, and only for the variants the frame records with, so a frame that
6465    /// draws no external texture builds nothing. What is built is retained
6466    /// alongside the rest of the variant's groups and dropped with them — on a
6467    /// format change, an atlas growth or a re-budget, all of which change the
6468    /// atlas view this group also holds.
6469    ///
6470    /// A texture with no registered view is skipped rather than substituted:
6471    /// its paint did not resolve either, so nothing in the frame names its
6472    /// slot.
6473    fn ensure_external_groups(
6474        &mut self,
6475        device: &wgpu::Device,
6476        variant: EnginePipeline,
6477        pipeline: &wgpu::RenderPipeline,
6478        externals: &ExternalTextures,
6479    ) {
6480        if self.external_runs.is_empty() {
6481            return;
6482        }
6483        // Destructured rather than reached through `self`: the group map is
6484        // borrowed mutably while the atlas view and the placeholders are read.
6485        let Self {
6486            bind_groups,
6487            external_runs,
6488            atlas,
6489            placeholders,
6490            atlas_sampler,
6491            groups_created,
6492            ..
6493        } = self;
6494        let atlas_view = atlas
6495            .as_ref()
6496            .map(AtlasArray::view)
6497            .unwrap_or(&placeholders.atlas_array);
6498        let Some(groups) = bind_groups.get_mut(&variant) else {
6499            return;
6500        };
6501        for key in external_runs.keys() {
6502            if groups.externals.contains_key(key) {
6503                continue;
6504            }
6505            let Some(view) = externals.view(*key) else {
6506                continue;
6507            };
6508            groups.externals.insert(
6509                *key,
6510                images_bind_group(device, pipeline, atlas_view, view, atlas_sampler.as_ref()),
6511            );
6512            *groups_created = groups_created.saturating_add(1);
6513        }
6514    }
6515
6516    /// Drops every bind group naming the texture registered under `key`.
6517    ///
6518    /// Called when that registration changes, because a group holds its view
6519    /// by value: keeping one past a re-bind would go on sampling the texture
6520    /// the caller replaced, and keeping one past an unbind would hold the
6521    /// caller's texture alive for as long as this renderer lives.
6522    fn forget_external(&mut self, key: u64) {
6523        for groups in self.bind_groups.values_mut() {
6524            groups.externals.remove(&key);
6525        }
6526    }
6527}
6528
6529/// Lower one atlas page's recorded commands into the strips that draw it,
6530/// answering whether the whole page could be expressed.
6531///
6532/// The caller-supplied half of the render-to-atlas seam: `gpu::atlas` owns the
6533/// pass, the orderings and the submit, while turning a command stream into
6534/// strips is compiler work and stays on this side of the edge — `compile`
6535/// already depends on `gpu::atlas`, so taking the reverse dependency would make
6536/// the two mutually recursive.
6537///
6538/// The stream is replayed as a scene in *page* space and compiled by
6539/// `lowering`, which is why that compiler is sized to the page rather than to
6540/// the surface. Only the four commands an outline glyph produces are lowered;
6541/// anything else — a clip path, a blend layer, a gradient paint, which is to
6542/// say every COLR shape — refuses the page whole rather than drawing part of
6543/// it. An indexed paint coming back out of the compile means the same thing:
6544/// the atlas pass binds no paint texture, so a record it would have to sample
6545/// cannot be drawn.
6546///
6547/// Refusing a page is a *last* line rather than the design, because refusal
6548/// cannot be made harmless here: `glifo` clears a recorder's commands whether
6549/// or not this answered `true`, and it offers no way to withdraw the entries
6550/// whose pixels those commands were going to be. Nothing that would reach this
6551/// refusal is therefore admitted to the atlas in the first place — a colour
6552/// face never takes the atlas route at all (`crate::text::atlas_policy`), so
6553/// what arrives here is the solid outline stream this lowers.
6554fn lower_atlas_page(
6555    recorder: &AtlasCommandRecorder,
6556    buffers: &mut AtlasPageBuffers,
6557    lowering: &mut SceneCompiler,
6558    page: (u16, u16),
6559) -> bool {
6560    let mut scene = Scene::new();
6561    {
6562        let mut builder = SceneBuilder::new(&mut scene);
6563        let mut transform = Affine::IDENTITY;
6564        let mut brush = Brush::Solid(Color::BLACK);
6565
6566        for command in &recorder.commands {
6567            match command {
6568                AtlasCommand::SetTransform(next) => transform = *next,
6569                AtlasCommand::SetPaint(AtlasPaint::Solid(color)) => brush = Brush::Solid(*color),
6570                AtlasCommand::FillPath(path) => {
6571                    builder.push_transform(transform);
6572                    builder.fill_path((**path).clone(), brush.clone());
6573                    builder.pop_transform();
6574                }
6575                AtlasCommand::FillRect(rect) => {
6576                    builder.push_transform(transform);
6577                    builder.fill_rect(*rect, brush.clone());
6578                    builder.pop_transform();
6579                }
6580                _ => return false,
6581            }
6582        }
6583    }
6584
6585    let Ok(frame) = lowering.compile(&scene, Affine::IDENTITY, page) else {
6586        return false;
6587    };
6588
6589    let strips = frame.strip_buf();
6590    for draw in frame.draws() {
6591        let Paint::Solid(color) = &draw.paint else {
6592            return false;
6593        };
6594        let Some(run) = strips.get(draw.strip_range.clone()) else {
6595            continue;
6596        };
6597        push_solid_strips(
6598            run,
6599            color.as_premul_rgba8().to_u32(),
6600            draw.depth,
6601            &mut buffers.instances,
6602        );
6603    }
6604    buffers.alphas.extend_from_slice(frame.alphas());
6605    true
6606}
6607
6608/// The [`ImageId`] an encoded image paint names, or `None` for the one
6609/// [`ImageSource`] variant no residency ever mints — the paint carrying its
6610/// pixels inline as a [`vello_common::pixmap::Pixmap`] rather than through a
6611/// handle. The compiler's own image encoding always produces the handle form
6612/// (see [`crate::compile::paint::encode_image`]), so a compiled frame never
6613/// exercises the `None` arm; it exists because the type itself admits both.
6614fn image_id(image: &EncodedImage) -> Option<ImageId> {
6615    match image.source {
6616        ImageSource::OpaqueId { id, .. } => Some(id),
6617        ImageSource::Pixmap(_) => None,
6618    }
6619}
6620
6621/// A lowered image record corrected for an atlas rectangle that holds a
6622/// *minified* copy of the source.
6623///
6624/// The record's transform maps a device position back onto the image's own
6625/// texels, and the compiler composed it against the source's declared extent —
6626/// it had to, since that is all it knows before residency is consulted. Scaling
6627/// its output by [`ResidentImage::minify_scale`] retargets it at the smaller
6628/// rectangle actually uploaded, which is the whole correction a downsampled
6629/// image needs: the record's `image_size` and `image_offset` already describe
6630/// the resident rectangle.
6631///
6632/// A record stored at full size is returned untouched, which is every image but
6633/// one larger than an atlas layer.
6634fn fit_minified(record: GpuEncodedPaint, resident: &ResidentImage) -> GpuEncodedPaint {
6635    let Some((x, y)) = resident.minify_scale() else {
6636        return record;
6637    };
6638    let GpuEncodedPaint::Image(mut image) = record else {
6639        return record;
6640    };
6641
6642    // `[a, b, c, d, tx, ty]`, mapping `(u, v)` to `(a·u + c·v + tx, b·u + d·v +
6643    // ty)`: the x row is scaled by one factor and the y row by the other.
6644    image.transform[0] *= x;
6645    image.transform[2] *= x;
6646    image.transform[4] *= x;
6647    image.transform[1] *= y;
6648    image.transform[3] *= y;
6649    image.transform[5] *= y;
6650
6651    GpuEncodedPaint::Image(image)
6652}
6653
6654/// The descriptor every `Config` uniform buffer is created with.
6655fn config_descriptor(label: &str) -> wgpu::BufferDescriptor<'_> {
6656    wgpu::BufferDescriptor {
6657        label: Some(label),
6658        size: GpuConfig::SIZE,
6659        usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,
6660        mapped_at_creation: false,
6661    }
6662}
6663
6664/// The rows a staged upload writes: the `filled` rows the frame's bytes take
6665/// and every row the last upload left `live`, never past `height` — see
6666/// [`ResourceTexture::live_rows`].
6667fn staged_rows(filled: u32, live: u32, height: u32) -> u32 {
6668    filled.max(live).min(height)
6669}
6670
6671/// The rows of a resource texture `len` bytes fill, the last one partly.
6672fn rows_filled(len: usize, bytes_per_row: u32) -> u32 {
6673    let rows = (len as u64).div_ceil(u64::from(bytes_per_row.max(1)));
6674    u32::try_from(rows).unwrap_or(u32::MAX)
6675}
6676
6677/// The texel copy layout of a full resource-texture upload.
6678fn resource_layout(bytes_per_row: u32, rows: u32) -> wgpu::TexelCopyBufferLayout {
6679    wgpu::TexelCopyBufferLayout {
6680        offset: 0,
6681        bytes_per_row: Some(bytes_per_row),
6682        rows_per_image: Some(rows),
6683    }
6684}
6685
6686/// The descriptor for a gradient LUT texture of `width` x `height` texels.
6687fn gradient_texture_descriptor(width: u32, height: u32) -> wgpu::TextureDescriptor<'static> {
6688    wgpu::TextureDescriptor {
6689        label: Some("frust-engine gradient texture"),
6690        size: wgpu::Extent3d {
6691            width,
6692            height,
6693            depth_or_array_layers: 1,
6694        },
6695        mip_level_count: 1,
6696        sample_count: 1,
6697        dimension: wgpu::TextureDimension::D2,
6698        format: GradientTextureLayout::FORMAT,
6699        usage: gpu::RESOURCE_TEXTURE_USAGES,
6700        view_formats: &[],
6701    }
6702}
6703
6704/// The 1x1 stand-ins for the strip shader's not-yet-written bindings.
6705#[derive(Debug)]
6706struct Placeholders {
6707    layer_input: wgpu::TextureView,
6708    atlas_array: wgpu::TextureView,
6709    external: wgpu::TextureView,
6710}
6711
6712impl Placeholders {
6713    fn new(device: &wgpu::Device) -> Self {
6714        Self {
6715            layer_input: placeholder_view(device, "frust-engine layer input placeholder", false),
6716            atlas_array: placeholder_view(device, "frust-engine atlas placeholder", true),
6717            external: placeholder_view(device, "frust-engine external placeholder", false),
6718        }
6719    }
6720}
6721
6722/// A 1x1 transparent `Rgba8Unorm` texture's view, as a plain 2D texture or as
6723/// a 2D array.
6724///
6725/// The array variant allocates **two** layers, not one, matching
6726/// [`gpu::atlas::atlas_texture_descriptor`]'s own floor: wgpu-hal 30.0.1's
6727/// GLES backend derives the GL target from the descriptor's layer count
6728/// alone, and a one-layer array descriptor binds as `GL_TEXTURE_2D` rather
6729/// than `GL_TEXTURE_2D_ARRAY` (see that function's doc comment for the
6730/// file:line and upstream issue refs). Only layer zero of the two is ever
6731/// sampled here.
6732fn placeholder_view(device: &wgpu::Device, label: &str, array: bool) -> wgpu::TextureView {
6733    let texture = device.create_texture(&wgpu::TextureDescriptor {
6734        label: Some(label),
6735        size: wgpu::Extent3d {
6736            width: 1,
6737            height: 1,
6738            depth_or_array_layers: if array { 2 } else { 1 },
6739        },
6740        mip_level_count: 1,
6741        sample_count: 1,
6742        dimension: wgpu::TextureDimension::D2,
6743        format: wgpu::TextureFormat::Rgba8Unorm,
6744        usage: wgpu::TextureUsages::TEXTURE_BINDING,
6745        view_formats: &[],
6746    });
6747    texture.create_view(&wgpu::TextureViewDescriptor {
6748        label: Some(label),
6749        dimension: Some(if array {
6750            wgpu::TextureViewDimension::D2Array
6751        } else {
6752            wgpu::TextureViewDimension::D2
6753        }),
6754        ..Default::default()
6755    })
6756}
6757
6758/// The four bind groups every strip pass sets.
6759///
6760/// The grouping is the shader's, not this module's: coverage plus config plus
6761/// layer input, atlas array plus external texture, encoded paints, gradient
6762/// ramps — four groups exactly, which is the downlevel ceiling with no
6763/// headroom left.
6764#[derive(Debug)]
6765struct StripBindGroups {
6766    resources: wgpu::BindGroup,
6767    images: wgpu::BindGroup,
6768    paints: wgpu::BindGroup,
6769    gradients: wgpu::BindGroup,
6770    /// Group 1 again, once per externally bound texture this variant has
6771    /// drawn: the same atlas array beside that texture's view instead of the
6772    /// placeholder. Keyed by the id a display list names the texture by.
6773    ///
6774    /// A second group rather than a fifth: the four-group ceiling has no
6775    /// headroom (see [`crate::gpu::pipelines`]), so an external texture is
6776    /// bound by re-setting the group the atlas already occupies — which is why
6777    /// a pass's instances are split into runs at all.
6778    externals: HashMap<u64, wgpu::BindGroup>,
6779}
6780
6781impl StripBindGroups {
6782    /// Takes the eight resources one by one rather than a `&FrameResources`
6783    /// so the caller can build a set while holding the map it lands in
6784    /// mutably.
6785    ///
6786    /// `atlas` is the live [`AtlasArray`] view once the renderer has one, or
6787    /// [`Placeholders::atlas_array`] until then — the caller picks, since
6788    /// only it knows which the frame's own `atlas_layers` calls for. `sampler`
6789    /// is [`FrameResources::atlas_sampler`].
6790    #[expect(
6791        clippy::too_many_arguments,
6792        reason = "one bind-group build's full resource list; a struct would \
6793                  only rename the same borrows the caller already holds \
6794                  mutably in `FrameResources`"
6795    )]
6796    fn new(
6797        device: &wgpu::Device,
6798        pipeline: &wgpu::RenderPipeline,
6799        alphas: &wgpu::TextureView,
6800        config: &wgpu::Buffer,
6801        placeholders: &Placeholders,
6802        atlas: &wgpu::TextureView,
6803        sampler: Option<&wgpu::Sampler>,
6804        paints: &wgpu::TextureView,
6805        gradients: &wgpu::TextureView,
6806    ) -> Self {
6807        let resources_group =
6808            resources_bind_group(device, pipeline, alphas, config, &placeholders.layer_input);
6809        let images = images_bind_group(device, pipeline, atlas, &placeholders.external, sampler);
6810        let paints = device.create_bind_group(&wgpu::BindGroupDescriptor {
6811            label: Some("frust-engine strip paints"),
6812            layout: &pipeline.get_bind_group_layout(2),
6813            entries: &[wgpu::BindGroupEntry {
6814                binding: 0,
6815                resource: wgpu::BindingResource::TextureView(paints),
6816            }],
6817        });
6818        let gradients = device.create_bind_group(&wgpu::BindGroupDescriptor {
6819            label: Some("frust-engine strip gradients"),
6820            layout: &pipeline.get_bind_group_layout(3),
6821            entries: &[wgpu::BindGroupEntry {
6822                binding: 0,
6823                resource: wgpu::BindingResource::TextureView(gradients),
6824            }],
6825        });
6826
6827        Self {
6828            resources: resources_group,
6829            images,
6830            paints,
6831            gradients,
6832            externals: HashMap::new(),
6833        }
6834    }
6835
6836    /// Sets all four groups on `pass`, taking group 0 from `resources` and
6837    /// group 1 from `images` when a segment names one, rather than from this
6838    /// set.
6839    ///
6840    /// Two of the four vary within a pass. Group 0 carries both the pass's
6841    /// viewport uniform and the layer input a composite samples, so a page
6842    /// round and every composite in it substitute their own. Group 1 carries
6843    /// the external texture a run is drawn with, so a segment that samples one
6844    /// substitutes the group holding it; every other segment takes this set's
6845    /// own, which pairs the atlas array with a placeholder nothing reads.
6846    /// Groups 2-3 are frame-wide and belong to the pipeline variant.
6847    fn bind_with(
6848        &self,
6849        pass: &mut wgpu::RenderPass<'_>,
6850        resources: &wgpu::BindGroup,
6851        images: Option<&wgpu::BindGroup>,
6852    ) {
6853        pass.set_bind_group(0, resources, &[]);
6854        pass.set_bind_group(1, images.unwrap_or(&self.images), &[]);
6855        pass.set_bind_group(2, &self.paints, &[]);
6856        pass.set_bind_group(3, &self.gradients, &[]);
6857    }
6858}
6859
6860/// Group 1 of a strip pass: the image atlas array, the caller-owned texture
6861/// an external image paint samples, and — in the image-sampler program family
6862/// only — the sampler the atlas array is filtered through.
6863///
6864/// One function for both shapes the group takes — `external` is
6865/// [`Placeholders::external`] for the frame-wide group, or a registered view
6866/// for the group a run of external instances is drawn with (see
6867/// [`crate::gpu::bindings`]). Built against one pipeline's group layout, as
6868/// [`resources_bind_group`] is; `sampler` is present exactly when that layout
6869/// declares one, which [`FrameResources::atlas_sampler`] already decides.
6870fn images_bind_group(
6871    device: &wgpu::Device,
6872    pipeline: &wgpu::RenderPipeline,
6873    atlas: &wgpu::TextureView,
6874    external: &wgpu::TextureView,
6875    sampler: Option<&wgpu::Sampler>,
6876) -> wgpu::BindGroup {
6877    let textures = [
6878        wgpu::BindGroupEntry {
6879            binding: 0,
6880            resource: wgpu::BindingResource::TextureView(atlas),
6881        },
6882        wgpu::BindGroupEntry {
6883            binding: 1,
6884            resource: wgpu::BindingResource::TextureView(external),
6885        },
6886    ];
6887    let with_sampler;
6888    let entries: &[wgpu::BindGroupEntry<'_>] = match sampler {
6889        Some(sampler) => {
6890            let [atlas, external] = textures;
6891            with_sampler = [
6892                atlas,
6893                external,
6894                wgpu::BindGroupEntry {
6895                    binding: ATLAS_SAMPLER_BINDING,
6896                    resource: wgpu::BindingResource::Sampler(sampler),
6897                },
6898            ];
6899            &with_sampler
6900        }
6901        None => &textures,
6902    };
6903    device.create_bind_group(&wgpu::BindGroupDescriptor {
6904        label: Some("frust-engine strip images"),
6905        layout: &pipeline.get_bind_group_layout(1),
6906        entries,
6907    })
6908}
6909
6910/// The sampler the image-sampler program family filters bilinear atlas
6911/// images through: linear in both directions, clamped to the edge, one mip.
6912///
6913/// Clamp-to-edge is a backstop rather than the mechanism: the shader already
6914/// clamps every sample position to the centres of the image's own edge texels
6915/// (`shaders/helpers.wgsl`'s `sampled_bilinear`), so the filter never reaches
6916/// past the image's rectangle in the first place.
6917fn atlas_sampler(device: &wgpu::Device) -> wgpu::Sampler {
6918    device.create_sampler(&wgpu::SamplerDescriptor {
6919        label: Some("frust-engine atlas sampler"),
6920        address_mode_u: wgpu::AddressMode::ClampToEdge,
6921        address_mode_v: wgpu::AddressMode::ClampToEdge,
6922        address_mode_w: wgpu::AddressMode::ClampToEdge,
6923        mag_filter: wgpu::FilterMode::Linear,
6924        min_filter: wgpu::FilterMode::Linear,
6925        mipmap_filter: wgpu::MipmapFilterMode::Nearest,
6926        ..Default::default()
6927    })
6928}
6929
6930/// The groups a round's `released` list names, as the per-group flags a
6931/// [`RoundPlan`] carries.
6932fn released_groups(released: &[PageParity]) -> [bool; MAX_LIVE_PAGES] {
6933    let mut groups = [false; MAX_LIVE_PAGES];
6934    for parity in released {
6935        if let Some(slot) = groups.get_mut(parity.index()) {
6936            *slot = true;
6937        }
6938    }
6939    groups
6940}
6941
6942/// How a region of a frame's own target is read back into a pooled page —
6943/// the mechanism every effect sampling what the frame has already painted
6944/// shares (a backdrop blur today; see [`crate::filters`]' *The backdrop
6945/// blur*).
6946///
6947/// Either way the region reaches the page through one pass of
6948/// [`EnginePipeline::Copy`] that clears the page and draws the region at its
6949/// origin: a pooled page is a render target and a sampled texture, never a
6950/// copy destination, so a texture copy cannot land in one directly. The modes
6951/// differ only in what that pass reads.
6952///
6953/// Decided per frame from the target texture itself, because the caller
6954/// decides what it renders into: a swapchain texture is `RENDER_ATTACHMENT`
6955/// only on this tier and is never widened, so `frust-render` renders a frame
6956/// carrying a backdrop into an intermediate of its own (in the swapchain's own
6957/// format, sampled and copyable) and presents it with one copy pass.
6958#[derive(Debug, Clone, Copy, PartialEq, Eq)]
6959pub enum RegionCopyMode {
6960    /// The target is `COPY_SRC` in the pages' own [`INTERMEDIATE_FORMAT`]: a
6961    /// `copy_texture_to_texture` lands the region in the renderer's own
6962    /// staging texture, unconverted, and the copy pass reads it from there —
6963    /// the route a target that cannot be sampled (a headless readback target)
6964    /// still has.
6965    Texture,
6966    /// The target is `TEXTURE_BINDING`: the copy pass reads the region
6967    /// straight out of it — any format the target is rendered in, converted on
6968    /// the way into the page's own.
6969    Fragment,
6970    /// The target can be neither copied nor sampled: no region is read back,
6971    /// and a backdrop blur draws nothing.
6972    Unavailable,
6973}
6974
6975impl RegionCopyMode {
6976    /// The mode a frame rendering into `target` reads regions back with.
6977    #[must_use]
6978    pub fn of(target: &wgpu::Texture) -> Self {
6979        Self::for_usage(target.usage(), target.format(), target.sample_count())
6980    }
6981
6982    /// [`Self::of`] over the three facts it reads, so the decision is
6983    /// host-testable without a device.
6984    ///
6985    /// Sampling is preferred whenever the target allows it — it costs the one
6986    /// pass alone, where a texture copy costs a transfer as well.
6987    #[must_use]
6988    pub fn for_usage(
6989        usage: wgpu::TextureUsages,
6990        format: wgpu::TextureFormat,
6991        sample_count: u32,
6992    ) -> Self {
6993        if sample_count != 1 {
6994            Self::Unavailable
6995        } else if usage.contains(wgpu::TextureUsages::TEXTURE_BINDING) {
6996            Self::Fragment
6997        } else if format == INTERMEDIATE_FORMAT && usage.contains(wgpu::TextureUsages::COPY_SRC) {
6998            Self::Texture
6999        } else {
7000            Self::Unavailable
7001        }
7002    }
7003}
7004
7005/// Reports a frame whose backdrop blurs are drawn as nothing because `target`
7006/// can be neither copied nor sampled, once per process.
7007fn note_backdrop_unreadable(target: &wgpu::Texture) {
7008    BACKDROP_UNREADABLE_WARNING.call_once(|| {
7009        log::warn!(
7010            "backdrop blur drawn as nothing: the frame's target ({:?}, {:?}) is neither \
7011             TEXTURE_BINDING nor COPY_SRC in {INTERMEDIATE_FORMAT:?}, so its backdrop cannot be \
7012             read back (logged once)",
7013            target.format(),
7014            target.usage()
7015        );
7016    });
7017}
7018
7019/// What a frame's region copies are recorded with beyond the copy pipeline:
7020/// the instance buffer — one 16-byte slot per copy a frame can run
7021/// ([`crate::filters::MAX_BACKDROPS_PER_FRAME`]), written as each copy is
7022/// recorded — and, for a [`RegionCopyMode::Texture`] frame, the staging
7023/// texture a region is transferred into before the copy pass reads it.
7024#[derive(Debug)]
7025struct RegionCopyResources {
7026    instances: wgpu::Buffer,
7027    /// `COPY_DST | TEXTURE_BINDING` in [`INTERMEDIATE_FORMAT`], grown to the
7028    /// largest region a frame has copied and kept, like the page pool's
7029    /// textures, for the renderer's life. Created by the first texture-mode
7030    /// frame.
7031    staging: Option<ResourceTexture>,
7032    /// Group 0 of the copy pass naming [`Self::staging`]'s view, built on the
7033    /// first texture-mode copy after the staging texture was (re)created.
7034    staging_group: Option<wgpu::BindGroup>,
7035}
7036
7037impl RegionCopyResources {
7038    fn new(device: &wgpu::Device) -> Self {
7039        let slots = crate::filters::MAX_BACKDROPS_PER_FRAME as u64;
7040        Self {
7041            instances: device.create_buffer(&wgpu::BufferDescriptor {
7042                label: Some("frust-engine region copy instances"),
7043                size: slots.saturating_mul(COPY_INSTANCE_BYTES),
7044                usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
7045                mapped_at_creation: false,
7046            }),
7047            staging: None,
7048            staging_group: None,
7049        }
7050    }
7051
7052    /// Grows the staging texture to hold a `width` x `height` region, if it
7053    /// does not already.
7054    fn ensure_staging(&mut self, device: &wgpu::Device, width: u32, height: u32) {
7055        if width == 0 || height == 0 {
7056            return;
7057        }
7058        let (held_width, held_height) = self
7059            .staging
7060            .as_ref()
7061            .map_or((0, 0), |staging| (staging.width, staging.height));
7062        if held_width >= width && held_height >= height {
7063            return;
7064        }
7065        self.staging_group = None;
7066        self.staging = Some(ResourceTexture::new(
7067            device,
7068            &wgpu::TextureDescriptor {
7069                label: Some("frust-engine region copy staging"),
7070                size: wgpu::Extent3d {
7071                    width: width.max(held_width),
7072                    height: height.max(held_height),
7073                    depth_or_array_layers: 1,
7074                },
7075                mip_level_count: 1,
7076                sample_count: 1,
7077                dimension: wgpu::TextureDimension::D2,
7078                format: INTERMEDIATE_FORMAT,
7079                usage: wgpu::TextureUsages::COPY_DST | wgpu::TextureUsages::TEXTURE_BINDING,
7080                view_formats: &[],
7081            },
7082        ));
7083    }
7084}
7085
7086/// One region copy's full recording state: a region of the frame's own
7087/// target, read back into a pooled page at the page's origin.
7088struct RegionCopyPass<'a> {
7089    mode: RegionCopyMode,
7090    /// The frame's own target.
7091    target: &'a wgpu::TextureView,
7092    /// The page the region lands in.
7093    page: &'a PooledTexture,
7094    /// The device region copied — inside the target by construction
7095    /// ([`crate::filters::backdrop_region`] clamps it).
7096    source: RectU16,
7097    /// [`EnginePipeline::Copy`], on every frame that copies.
7098    pipeline: Option<&'a wgpu::RenderPipeline>,
7099    /// The copies' instance buffer and staging texture, on the same frames.
7100    resources: Option<&'a RegionCopyResources>,
7101    /// This copy's instance slot.
7102    slot: u32,
7103}
7104
7105impl RegionCopyPass<'_> {
7106    /// Records the copy into `encoder`: on a texture-mode frame the region is
7107    /// transferred into the staging texture first, then one pass clears the
7108    /// page and draws the region into it at the page's origin.
7109    ///
7110    /// The page holds the region and transparency around it. A copy whose
7111    /// pipeline, instance buffer or staging texture is missing — which cannot
7112    /// happen, `encode` resolves all three for every frame that copies — still
7113    /// clears the page, and the backdrop blurs nothing into nothing: "draw
7114    /// less, never wrong".
7115    fn record(
7116        &self,
7117        device: &wgpu::Device,
7118        queue: &wgpu::Queue,
7119        encoder: &mut wgpu::CommandEncoder,
7120        timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
7121    ) -> u64 {
7122        let width = self.source.width();
7123        let height = self.source.height();
7124        // The texture the pass reads, and where in it the region starts.
7125        let source = match (self.mode, self.resources) {
7126            (RegionCopyMode::Fragment, Some(_)) => {
7127                Some((self.target, (self.source.x0, self.source.y0)))
7128            }
7129            (RegionCopyMode::Texture, Some(resources)) => match resources.staging.as_ref() {
7130                Some(staging)
7131                    if staging.width >= u32::from(width) && staging.height >= u32::from(height) =>
7132                {
7133                    encoder.copy_texture_to_texture(
7134                        wgpu::TexelCopyTextureInfo {
7135                            texture: self.target.texture(),
7136                            mip_level: 0,
7137                            origin: wgpu::Origin3d {
7138                                x: u32::from(self.source.x0),
7139                                y: u32::from(self.source.y0),
7140                                z: 0,
7141                            },
7142                            aspect: wgpu::TextureAspect::All,
7143                        },
7144                        staging.copy_target(),
7145                        wgpu::Extent3d {
7146                            width: u32::from(width),
7147                            height: u32::from(height),
7148                            depth_or_array_layers: 1,
7149                        },
7150                    );
7151                    Some((&staging.view, (0, 0)))
7152                }
7153                _ => None,
7154            },
7155            _ => None,
7156        };
7157
7158        let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
7159            label: Some(FramePass::Copy.label()),
7160            color_attachments: &[Some(color_attachment(
7161                self.page.view(),
7162                wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
7163            ))],
7164            depth_stencil_attachment: None,
7165            timestamp_writes: timestamps,
7166            occlusion_query_set: None,
7167            multiview_mask: None,
7168        });
7169        let (Some((view, origin)), Some(pipeline), Some(resources)) =
7170            (source, self.pipeline, self.resources)
7171        else {
7172            return 0;
7173        };
7174        let offset = u64::from(self.slot).saturating_mul(COPY_INSTANCE_BYTES);
7175        if offset.saturating_add(COPY_INSTANCE_BYTES) > resources.instances.size() {
7176            return 0;
7177        }
7178        let (page_width, page_height) = self.page.size();
7179        let instance = [
7180            pack_u16_pair(0, 0),
7181            pack_u16_pair(origin.0, origin.1),
7182            pack_u16_pair(width, height),
7183            pack_u16_pair(
7184                u16::try_from(page_width).unwrap_or(u16::MAX),
7185                u16::try_from(page_height).unwrap_or(u16::MAX),
7186            ),
7187        ];
7188        queue.write_buffer(
7189            &resources.instances,
7190            offset,
7191            bytemuck::cast_slice(&instance),
7192        );
7193        // The staging texture's group is cached; the frame's own target is
7194        // the caller's and changes with the swapchain image, so a fragment-mode
7195        // copy builds its group per copy.
7196        let built;
7197        let (bind_group, created) = match (self.mode, resources.staging_group.as_ref()) {
7198            (RegionCopyMode::Texture, Some(group)) => (group, 0),
7199            _ => {
7200                built = copy_source_group(device, pipeline, view);
7201                (&built, 1)
7202            }
7203        };
7204        pass.set_pipeline(pipeline);
7205        pass.set_vertex_buffer(0, resources.instances.slice(..));
7206        pass.set_bind_group(0, bind_group, &[]);
7207        pass.draw(
7208            0..COPY_QUAD_VERTICES,
7209            self.slot..self.slot.saturating_add(1),
7210        );
7211        created
7212    }
7213}
7214
7215/// Group 0 of the region copy pass: the texture it reads the region from.
7216fn copy_source_group(
7217    device: &wgpu::Device,
7218    pipeline: &wgpu::RenderPipeline,
7219    view: &wgpu::TextureView,
7220) -> wgpu::BindGroup {
7221    device.create_bind_group(&wgpu::BindGroupDescriptor {
7222        label: Some("frust-engine region copy source"),
7223        layout: &pipeline.get_bind_group_layout(0),
7224        entries: &[wgpu::BindGroupEntry {
7225            binding: 0,
7226            resource: wgpu::BindingResource::TextureView(view),
7227        }],
7228    })
7229}
7230
7231/// Group 0 of a strip pass: the frame's coverage, the pass's own viewport
7232/// uniform, and the texture a composite reads its finished page from.
7233///
7234/// Built against one pipeline's group layout. Every strip pipeline names the
7235/// one explicit strip layout ([`crate::gpu::bindings::strip_pipeline_layout`]),
7236/// so the group binds under every strip state and paint class alike — which is
7237/// what lets one pass switch class pipelines between runs without rebinding.
7238fn resources_bind_group(
7239    device: &wgpu::Device,
7240    pipeline: &wgpu::RenderPipeline,
7241    alphas: &wgpu::TextureView,
7242    config: &wgpu::Buffer,
7243    layer_input: &wgpu::TextureView,
7244) -> wgpu::BindGroup {
7245    device.create_bind_group(&wgpu::BindGroupDescriptor {
7246        label: Some("frust-engine strip resources"),
7247        layout: &pipeline.get_bind_group_layout(0),
7248        entries: &[
7249            wgpu::BindGroupEntry {
7250                binding: 0,
7251                resource: wgpu::BindingResource::TextureView(alphas),
7252            },
7253            wgpu::BindGroupEntry {
7254                binding: 1,
7255                resource: config.as_entire_binding(),
7256            },
7257            wgpu::BindGroupEntry {
7258                binding: 2,
7259                resource: wgpu::BindingResource::TextureView(layer_input),
7260            },
7261        ],
7262    })
7263}
7264
7265// ---------------------------------------------------------------------------
7266// Command ranges (render-model Phase 4 overlay spike)
7267// ---------------------------------------------------------------------------
7268
7269/// How [`project_command_range`] treats the commands it is asked to carry
7270/// into a range — the one thing that differs between the renderer a range is
7271/// projected for.
7272#[derive(Debug, Clone, Copy, PartialEq, Eq)]
7273pub struct RangeOptions {
7274    /// Keep a `DrawRetained` inside the range. A renderer with a retained
7275    /// store behind it keeps them; one without (an overlay renderer built with
7276    /// retention off, whose compiler refuses a frame carrying one) has them
7277    /// dropped and counted in [`RangeCut::dropped_draw_retained`] instead.
7278    pub keep_draw_retained: bool,
7279}
7280
7281/// What cutting a command range out of a scene did — the bracket rule of the
7282/// overlay spike, reported so a caller (and a test) can see what a cut cost.
7283///
7284/// See [`project_command_range`] for the rule itself.
7285#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
7286pub struct RangeCut {
7287    /// The first command index the projection carried, after clamping.
7288    pub start: usize,
7289    /// One past the last command index the projection carried, after
7290    /// clamping.
7291    pub end: usize,
7292    /// Brackets open at `start` that were reopened ahead of the range.
7293    pub reopened: usize,
7294    /// Brackets still open at `end` that were closed after the range.
7295    pub closed: usize,
7296    /// `PushRetained` brackets spanning a cut, painted inline (their
7297    /// presentation scale folded into the body's transforms and their alpha
7298    /// as a plain layer) rather than recorded as a page.
7299    pub demoted_retained: usize,
7300    /// `DrawRetained`s dropped under `keep_draw_retained: false`.
7301    pub dropped_draw_retained: usize,
7302    /// Pops inside the range with no open bracket of their own kind — already
7303    /// ignored by every renderer, so dropped rather than carried.
7304    pub dropped_pops: usize,
7305}
7306
7307/// The bracket families the display list closes with a pop of their own.
7308#[derive(Debug, Clone, Copy, PartialEq, Eq)]
7309enum BracketKind {
7310    Clip,
7311    Layer,
7312    Retained,
7313}
7314
7315/// The bracket kind `command` opens, if it opens one.
7316fn opens(command: &frust_scene::Command) -> Option<BracketKind> {
7317    use frust_scene::Command;
7318    match command {
7319        Command::PushClip { .. }
7320        | Command::PushClipRounded { .. }
7321        | Command::PushClipPath { .. } => Some(BracketKind::Clip),
7322        Command::PushLayer { .. } => Some(BracketKind::Layer),
7323        Command::PushRetained { .. } => Some(BracketKind::Retained),
7324        _ => None,
7325    }
7326}
7327
7328/// The bracket kind `command` closes, if it is a pop.
7329fn closes(command: &frust_scene::Command) -> Option<BracketKind> {
7330    use frust_scene::Command;
7331    match command {
7332        Command::PopClip => Some(BracketKind::Clip),
7333        Command::PopLayer => Some(BracketKind::Layer),
7334        Command::PopRetained => Some(BracketKind::Retained),
7335        _ => None,
7336    }
7337}
7338
7339/// One bracket the projection holds open, and how it will be closed.
7340#[derive(Debug, Clone, Copy)]
7341enum OpenBracket {
7342    /// A clip or layer carried as recorded; closed with its own pop.
7343    Plain(BracketKind),
7344    /// A `PushRetained` carried as recorded; closed with `PopRetained`.
7345    Retained,
7346    /// A `PushRetained` spanning a cut, painted inline. `layer` says whether
7347    /// its alpha opened a layer to close; `prefix` is the correction in force
7348    /// before it, restored when it closes.
7349    Demoted { layer: bool, prefix: Affine },
7350    /// A `PushRetained` nested inside a demoted bracket: its presentation is
7351    /// ignored (only the outermost bracket's is honoured), so it opened
7352    /// nothing and closes nothing.
7353    Absorbed,
7354}
7355
7356impl OpenBracket {
7357    fn kind(self) -> BracketKind {
7358        match self {
7359            OpenBracket::Plain(kind) => kind,
7360            OpenBracket::Retained | OpenBracket::Demoted { .. } | OpenBracket::Absorbed => {
7361                BracketKind::Retained
7362            }
7363        }
7364    }
7365}
7366
7367/// Records `command` into `builder` with `prefix` composed ahead of its own
7368/// transform — the identity outside a demoted retained bracket, its
7369/// presentation correction inside one.
7370///
7371/// The builder's own transform stack is left as found: every arm pushes the
7372/// command's transform, records, and pops, so the recorded transform is
7373/// exactly `prefix * transform`.
7374fn replay(builder: &mut SceneBuilder<'_>, prefix: Affine, command: &frust_scene::Command) {
7375    use frust_scene::{Command, PathStyle};
7376    fn under(
7377        builder: &mut SceneBuilder<'_>,
7378        transform: Affine,
7379        record: impl FnOnce(&mut SceneBuilder<'_>),
7380    ) {
7381        builder.push_transform(transform);
7382        record(builder);
7383        builder.pop_transform();
7384    }
7385    match command {
7386        Command::FillRect {
7387            rect,
7388            brush,
7389            transform,
7390        } => under(builder, prefix * *transform, |b| {
7391            b.fill_rect(*rect, brush.clone())
7392        }),
7393        Command::RoundedRect {
7394            rect,
7395            radii,
7396            brush,
7397            transform,
7398        } => under(builder, prefix * *transform, |b| {
7399            b.fill_rounded_rect_radii(*rect, *radii, brush.clone());
7400        }),
7401        Command::Line {
7402            p0,
7403            p1,
7404            width,
7405            brush,
7406            transform,
7407        } => under(builder, prefix * *transform, |b| {
7408            b.stroke_line(*p0, *p1, *width, brush.clone());
7409        }),
7410        // A glyph run composes the builder's transform ahead of its own.
7411        Command::GlyphRun(run) => under(builder, prefix, |b| b.draw_glyph_run(run.clone())),
7412        Command::PushClip { rect, transform } => {
7413            under(builder, prefix * *transform, |b| b.push_clip(*rect));
7414        }
7415        Command::PushClipRounded {
7416            rect,
7417            radii,
7418            transform,
7419        } => under(builder, prefix * *transform, |b| {
7420            b.push_clip_rounded_radii(*rect, *radii);
7421        }),
7422        Command::PushClipPath { path, transform } => {
7423            under(builder, prefix * *transform, |b| {
7424                b.push_clip_path(path.clone())
7425            });
7426        }
7427        Command::PopClip => builder.pop_clip(),
7428        Command::Image {
7429            data,
7430            dest,
7431            transform,
7432        } => under(builder, prefix * *transform, |b| b.draw_image(data, *dest)),
7433        Command::BlurredRoundedRect {
7434            rect,
7435            radii,
7436            std_dev,
7437            color,
7438            transform,
7439        } => under(builder, prefix * *transform, |b| {
7440            b.draw_blurred_rounded_rect_radii(*rect, *radii, *std_dev, *color);
7441        }),
7442        Command::PushLayer {
7443            rect,
7444            alpha,
7445            transform,
7446        } => under(builder, prefix * *transform, |b| {
7447            b.push_layer(*rect, *alpha)
7448        }),
7449        Command::PopLayer => builder.pop_layer(),
7450        Command::ClearRect { rect, transform } => {
7451            under(builder, prefix * *transform, |b| b.clear_rect(*rect));
7452        }
7453        Command::Path {
7454            path,
7455            style,
7456            brush,
7457            transform,
7458        } => under(builder, prefix * *transform, |b| match style {
7459            PathStyle::Fill => b.fill_path(path.clone(), brush.clone()),
7460            PathStyle::Stroke { width, dash: None } => {
7461                b.stroke_path(path.clone(), *width, brush.clone());
7462            }
7463            PathStyle::Stroke {
7464                width,
7465                dash: Some(dash),
7466            } => b.stroke_path_dashed(path.clone(), *width, *dash, brush.clone()),
7467        }),
7468        Command::ShaderQuad {
7469            program,
7470            dest,
7471            transform,
7472            time,
7473        } => under(builder, prefix * *transform, |b| {
7474            b.draw_shader(program, *dest, *time);
7475        }),
7476        Command::PushRetained {
7477            key,
7478            rect,
7479            alpha,
7480            scale,
7481            transform,
7482        } => under(builder, prefix * *transform, |b| {
7483            b.push_retained(*key, *rect, *alpha, *scale);
7484        }),
7485        Command::PopRetained => builder.pop_retained(),
7486        Command::DrawRetained {
7487            key,
7488            rect,
7489            transform,
7490            alpha,
7491            scale,
7492        } => under(builder, prefix * *transform, |b| {
7493            b.draw_retained(*key, *rect, *alpha, *scale);
7494        }),
7495        Command::SceneTexture {
7496            id,
7497            dest,
7498            transform,
7499        } => {
7500            under(builder, prefix * *transform, |b| {
7501                b.scene_texture(*id, *dest)
7502            });
7503        }
7504        Command::BackdropBlur {
7505            rect,
7506            radius,
7507            transform,
7508        } => under(builder, prefix * *transform, |b| {
7509            b.backdrop_blur(*rect, *radius)
7510        }),
7511    }
7512}
7513
7514/// Writes the commands of `scene` in `range` into `out` (reset first) as a
7515/// self-contained display list, and reports what the cut did — the overlay
7516/// spike's range projection (render-model Phase 4, decision D10).
7517///
7518/// `range` is clamped to the scene: an `end` past the last command stops at
7519/// it, and a `start` past `end` projects nothing. The projection is a scene,
7520/// not a compiled-frame filter: the compiled frame's draws are ordered by
7521/// depth and grouped into rounds, while a command index is the one paint-order
7522/// position every producer of a cut (a platform view's place in the display
7523/// list) can name.
7524///
7525/// # The bracket rule
7526///
7527/// A cut can fall inside a bracket. The rule this spike chose:
7528///
7529/// - **Clips and layers spanning a cut are closed at it and reopened across
7530///   it.** Every clip or layer open at `start` is re-recorded, in its original
7531///   push order and with its original geometry, ahead of the range; every one
7532///   still open at `end` is popped after it, innermost first. Both halves of a
7533///   split clip therefore clip to the same shape, and both halves of a split
7534///   layer composite at the same alpha — which is exact for a clip, and for a
7535///   layer differs from the uncut frame only where content on both sides of
7536///   the cut overlaps inside it (each half is composited on its own).
7537/// - **A retained bracket spanning a cut is demoted to inline paint on both
7538///   sides.** A page recorded from half a body would be a stale page the
7539///   ledger then advertises, so neither side records one: the bracket's
7540///   presentation scale is folded into every body command's transform (the
7541///   same conjugation `crate::compile::layers::retained_correction` composes
7542///   ahead of the frame root) and its alpha below one becomes a plain layer —
7543///   exactly the lowering `frust_scene` prescribes for a renderer that does
7544///   not retain. A retained bracket nested inside a demoted one contributes
7545///   nothing, as a nested bracket's presentation never does. A retained
7546///   bracket wholly inside the range is carried as recorded.
7547/// - **A pop with no open bracket of its kind is dropped** (every renderer
7548///   ignores one anyway), and a `DrawRetained` is dropped when
7549///   `options.keep_draw_retained` is off.
7550///
7551/// Pops match the most recently opened bracket of their own kind, which is
7552/// the stack discipline of every well-nested display list.
7553///
7554/// Every carried command is re-recorded through a [`SceneBuilder`] with its
7555/// transform composed under the identity (or a demoted bracket's correction),
7556/// so geometry and paints are carried verbatim; the cost is one clone of each
7557/// carried command's payload, paid only by a caller that projects.
7558pub fn project_command_range(
7559    scene: &Scene,
7560    range: Range<usize>,
7561    options: RangeOptions,
7562    out: &mut Scene,
7563) -> RangeCut {
7564    project_command_ranges(scene, std::slice::from_ref(&range), options, out)
7565}
7566
7567/// [`project_command_range`] over several ranges at once: writes the
7568/// projection of each range of `ranges`, in the order given, one after the
7569/// other into `out` (reset first), and reports the cuts summed — the segmented
7570/// frame's way of drawing the commands *between* overlay cuts on the main
7571/// surface (render-model Phase 4, decision D10).
7572///
7573/// Each range is projected exactly as [`project_command_range`] projects it
7574/// alone — clamped to the scene, its open clips and layers reopened ahead of
7575/// it and closed after it, a retained bracket spanning either of its ends
7576/// demoted to inline paint — so every range is self-contained and the bracket
7577/// rule holds per range. A clip open across the gap between two ranges is
7578/// therefore closed at the end of the first and reopened at the start of the
7579/// second, never left open over the commands the gap removed.
7580///
7581/// The caller passes the ranges ascending and disjoint (the complement of a
7582/// frame's overlay cuts is); overlapping ranges are projected as given, each
7583/// command carried once per range it falls in. An empty range still reopens
7584/// and closes the brackets open at its start (drawing nothing), so a caller
7585/// that wants no trace of it leaves it out.
7586///
7587/// The returned [`RangeCut`]'s `start` is the first range's clamped start and
7588/// its `end` the last range's clamped end (both `0` for no range at all); its
7589/// counters are the sums over every range.
7590pub fn project_command_ranges(
7591    scene: &Scene,
7592    ranges: &[Range<usize>],
7593    options: RangeOptions,
7594    out: &mut Scene,
7595) -> RangeCut {
7596    let commands = scene.commands();
7597    let len = commands.len();
7598    let clamped: Vec<(usize, usize)> = ranges
7599        .iter()
7600        .map(|range| {
7601            let end = range.end.min(len);
7602            (range.start.min(end), end)
7603        })
7604        .collect();
7605    let mut cut = RangeCut {
7606        start: clamped.first().map_or(0, |&(start, _)| start),
7607        end: clamped.last().map_or(0, |&(_, end)| end),
7608        ..RangeCut::default()
7609    };
7610
7611    // Pass one, over the whole scene: where each bracket closes, and which
7612    // brackets are open at each range's start (in push order).
7613    let mut closed_at: Vec<Option<usize>> = vec![None; len];
7614    let mut stack: Vec<(BracketKind, usize)> = Vec::new();
7615    let mut open_at_start: Vec<Vec<usize>> = vec![Vec::new(); clamped.len()];
7616    let snapshot = |index: usize, stack: &[(BracketKind, usize)], open: &mut [Vec<usize>]| {
7617        for (slot, &(start, _)) in open.iter_mut().zip(&clamped) {
7618            if start == index {
7619                slot.extend(stack.iter().map(|&(_, at)| at));
7620            }
7621        }
7622    };
7623    for (index, command) in commands.iter().enumerate() {
7624        snapshot(index, &stack, &mut open_at_start);
7625        if let Some(kind) = opens(command) {
7626            stack.push((kind, index));
7627        } else if let Some(kind) = closes(command)
7628            && let Some(position) = stack.iter().rposition(|&(open, _)| open == kind)
7629        {
7630            let (_, at) = stack.remove(position);
7631            closed_at[at] = Some(index);
7632        }
7633    }
7634    snapshot(len, &stack, &mut open_at_start);
7635    // A retained bracket spans a cut of the range `start..end` when it opened
7636    // before the range, or is still open where the range ends short of the
7637    // scene's own end.
7638    let spans = |at: usize, start: usize, end: usize| {
7639        at < start || (end < len && closed_at[at].is_none_or(|c| c >= end))
7640    };
7641
7642    out.reset();
7643    let mut builder = SceneBuilder::new(out);
7644
7645    let push = |at: usize,
7646                (start, end): (usize, usize),
7647                builder: &mut SceneBuilder<'_>,
7648                open: &mut Vec<OpenBracket>,
7649                prefix: &mut Affine,
7650                cut: &mut RangeCut| {
7651        let command = &commands[at];
7652        match command {
7653            frust_scene::Command::PushRetained {
7654                rect,
7655                alpha,
7656                scale,
7657                transform,
7658                ..
7659            } => {
7660                if open
7661                    .iter()
7662                    .any(|bracket| matches!(bracket, OpenBracket::Demoted { .. }))
7663                {
7664                    open.push(OpenBracket::Absorbed);
7665                } else if spans(at, start, end) {
7666                    cut.demoted_retained += 1;
7667                    let correction =
7668                        crate::compile::layers::retained_correction(*rect, *scale, *transform);
7669                    let layer = *alpha < 1.0;
7670                    if layer {
7671                        let corrected = *prefix * correction * *transform;
7672                        builder.push_transform(corrected);
7673                        builder.push_layer(*rect, *alpha);
7674                        builder.pop_transform();
7675                    }
7676                    open.push(OpenBracket::Demoted {
7677                        layer,
7678                        prefix: *prefix,
7679                    });
7680                    *prefix *= correction;
7681                } else {
7682                    replay(builder, *prefix, command);
7683                    open.push(OpenBracket::Retained);
7684                }
7685            }
7686            _ => {
7687                if let Some(kind) = opens(command) {
7688                    replay(builder, *prefix, command);
7689                    open.push(OpenBracket::Plain(kind));
7690                }
7691            }
7692        }
7693    };
7694    let close =
7695        |bracket: OpenBracket, builder: &mut SceneBuilder<'_>, prefix: &mut Affine| match bracket {
7696            OpenBracket::Plain(BracketKind::Clip) => builder.pop_clip(),
7697            OpenBracket::Plain(BracketKind::Layer) => builder.pop_layer(),
7698            OpenBracket::Plain(BracketKind::Retained) | OpenBracket::Retained => {
7699                builder.pop_retained();
7700            }
7701            OpenBracket::Demoted {
7702                layer,
7703                prefix: before,
7704            } => {
7705                if layer {
7706                    builder.pop_layer();
7707                }
7708                *prefix = before;
7709            }
7710            OpenBracket::Absorbed => {}
7711        };
7712
7713    let mut open: Vec<OpenBracket> = Vec::new();
7714    for (&bounds, reopen) in clamped.iter().zip(&open_at_start) {
7715        let (start, end) = bounds;
7716        // Every range starts from the identity with nothing open: the one
7717        // before it closed everything it opened.
7718        let mut prefix = Affine::IDENTITY;
7719        for &at in reopen {
7720            cut.reopened += 1;
7721            push(at, bounds, &mut builder, &mut open, &mut prefix, &mut cut);
7722        }
7723        for (index, command) in commands.iter().enumerate().take(end).skip(start) {
7724            if opens(command).is_some() {
7725                push(
7726                    index,
7727                    bounds,
7728                    &mut builder,
7729                    &mut open,
7730                    &mut prefix,
7731                    &mut cut,
7732                );
7733            } else if let Some(kind) = closes(command) {
7734                match open.iter().rposition(|bracket| bracket.kind() == kind) {
7735                    Some(position) => {
7736                        let bracket = open.remove(position);
7737                        close(bracket, &mut builder, &mut prefix);
7738                    }
7739                    None => cut.dropped_pops += 1,
7740                }
7741            } else if matches!(command, frust_scene::Command::DrawRetained { .. })
7742                && !options.keep_draw_retained
7743            {
7744                cut.dropped_draw_retained += 1;
7745            } else {
7746                replay(&mut builder, prefix, command);
7747            }
7748        }
7749        while let Some(bracket) = open.pop() {
7750            cut.closed += 1;
7751            close(bracket, &mut builder, &mut prefix);
7752        }
7753    }
7754    cut
7755}
7756
7757/// The outermost `PushRetained` bracket still open just before command
7758/// `index` of `commands` — its command index — or `None` when no retained
7759/// bracket is open there.
7760///
7761/// Pops are matched the way [`project_command_range`] matches them: a
7762/// `PopRetained` closes the most recently opened retained bracket, whatever
7763/// clips or layers were opened in between, and one with no retained bracket
7764/// open is ignored. An `index` past the end asks about the end of the list,
7765/// so a bracket never closed is open there. A range starting at `index`
7766/// splits the bracket answered here: [`project_command_range`] demotes it to
7767/// inline paint on both sides of the cut. Read-only and one pass over
7768/// `commands[..index]`.
7769pub fn open_retained_bracket(commands: &[frust_scene::Command], index: usize) -> Option<usize> {
7770    let mut open: Vec<usize> = Vec::new();
7771    for (at, command) in commands.iter().enumerate().take(index) {
7772        match command {
7773            frust_scene::Command::PushRetained { .. } => open.push(at),
7774            frust_scene::Command::PopRetained => {
7775                open.pop();
7776            }
7777            _ => {}
7778        }
7779    }
7780    open.first().copied()
7781}
7782
7783#[cfg(test)]
7784mod tests {
7785    use super::*;
7786    use frust_gpu::DownlevelProfile;
7787    use kurbo::Rect;
7788    use peniko::color::palette::css::{BLUE, RED};
7789    use std::collections::BTreeMap;
7790    use vello_common::geometry::RectU16;
7791    use vello_common::paint::IndexedPaint;
7792
7793    /// Polls `future` to completion on the calling thread — device creation
7794    /// is the only future this module's GPU tests await.
7795    fn block_on<F: std::future::Future>(future: F) -> F::Output {
7796        use std::task::{Context, Poll, Waker};
7797
7798        let mut cx = Context::from_waker(Waker::noop());
7799        let mut future = std::pin::pin!(future);
7800        loop {
7801            match future.as_mut().poll(&mut cx) {
7802                Poll::Ready(value) => return value,
7803                Poll::Pending => std::thread::yield_now(),
7804            }
7805        }
7806    }
7807
7808    /// Finishing the warm-up builds the whole (de-duplicated) list exactly
7809    /// once however often it is called, waiting on a finished warm-up returns
7810    /// at once, and neither encodes a frame: the staging ring hands out no
7811    /// slot and creates no buffer.
7812    #[test]
7813    #[ignore = "needs a real GPU adapter; run with `cargo test -p frust-engine --lib -- --ignored \
7814                --test-threads=1` (pin the adapter with WGPU_BACKEND / WGPU_ADAPTER_NAME)"]
7815    fn finishing_the_warm_up_is_idempotent_and_encodes_nothing() {
7816        let mut context = frust_gpu::RenderContext::new();
7817        let handle = block_on(context.device()).expect("device creation");
7818        let caps = handle.caps.clone();
7819        let format = wgpu::TextureFormat::Rgba8Unorm;
7820        let mut renderer = EngineRenderer::new(&handle.device, &caps, format, None)
7821            .expect("the engine builds on this device");
7822        let mut listed = Vec::new();
7823        for desc in warm_up_list(&renderer.shaders, format, renderer.routing.uber) {
7824            if !listed.contains(&desc) {
7825                listed.push(desc);
7826            }
7827        }
7828
7829        renderer.finish_warm_up(&handle.device);
7830        assert!(renderer.warm_up_done());
7831        assert_eq!(
7832            renderer.compiled_pipelines(),
7833            listed.len() as u64,
7834            "the first call builds every listed variant once"
7835        );
7836        for _ in 0..3 {
7837            renderer.finish_warm_up(&handle.device);
7838            assert!(renderer.wait_warm_up(Duration::ZERO));
7839        }
7840        assert_eq!(
7841            renderer.compiled_pipelines(),
7842            listed.len() as u64,
7843            "a finished warm-up compiles nothing more"
7844        );
7845        assert_eq!(
7846            renderer.staging_stats().unwrap_or_default(),
7847            frust_gpu::staging::StagingStats::default(),
7848            "the warm-up records no frame: the staging ring was never used"
7849        );
7850    }
7851
7852    /// The warm-up list is [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for)'s list plus the first-frame
7853    /// openers and the layer-composite tail: a superset of it, opening with
7854    /// the atlas pass and ending with the depth-alpha layer composite, whose
7855    /// distinct entries are exactly the base list and that tail — and under
7856    /// the Uber kill switch it is the base list unchanged. Checked over
7857    /// variant keys, which is all a device-free test can build.
7858    #[test]
7859    fn the_warm_up_list_extends_the_base_list_in_order() {
7860        #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
7861        enum Key {
7862            Atlas,
7863            State(EnginePipeline, PaintClass),
7864        }
7865        let describe = Key::State;
7866        let tail = Key::State(EnginePipeline::StripDepthAlpha, PaintClass::LayerComposite);
7867
7868        for image_sampler in [false, true] {
7869            // The manual family's atlas pass IS the intermediate state's Uber
7870            // variant; the sampler family's is a variant of its own.
7871            let atlas = move || {
7872                if image_sampler {
7873                    Key::Atlas
7874                } else {
7875                    Key::State(
7876                        crate::gpu::pipelines::ATLAS_STRIP_PIPELINE,
7877                        PaintClass::Uber,
7878                    )
7879                }
7880            };
7881            let base = warm_up_plan(image_sampler, false, describe, atlas);
7882            let list = ordered_warm_up(image_sampler, false, describe, atlas);
7883
7884            let listed: std::collections::HashSet<_> = list.iter().copied().collect();
7885            for key in &base {
7886                assert!(listed.contains(key), "{key:?} is missing from the list");
7887            }
7888
7889            // The first-frame openers, in the order an ordinary first frame
7890            // draws them. Pinned exactly: a reorder changes which pipeline
7891            // the frame waits behind.
7892            let openers = [
7893                atlas(),
7894                describe(EnginePipeline::StripDepthAlpha, PaintClass::Glyph),
7895                describe(EnginePipeline::StripDepthAlpha, PaintClass::SolidCoverage),
7896                describe(EnginePipeline::StripOpaque, PaintClass::SolidCoverage),
7897                describe(EnginePipeline::StripDepthAlpha, PaintClass::Uber),
7898            ];
7899            assert_eq!(
7900                &list[..5],
7901                &openers[..],
7902                "opens with the first frame's pipelines"
7903            );
7904            // The base list follows the openers in its own order, unchanged.
7905            assert_eq!(
7906                &list[5..list.len() - 1],
7907                &base[..],
7908                "the base list follows the openers in warm_up_descs_for's order"
7909            );
7910            assert_eq!(list.last(), Some(&tail), "ends with the layer composite");
7911
7912            // De-duplicated by first occurrence, as the queue does: the
7913            // distinct entries are the base list and the tail, nothing else.
7914            let mut seen = std::collections::HashSet::new();
7915            let distinct: Vec<_> = list.iter().filter(|key| seen.insert(**key)).collect();
7916            let mut expected: std::collections::HashSet<_> = base.iter().copied().collect();
7917            expected.insert(tail);
7918            assert_eq!(distinct.len(), expected.len());
7919            assert!(distinct.iter().all(|key| expected.contains(*key)));
7920
7921            let forced = ordered_warm_up(image_sampler, true, describe, atlas);
7922            assert_eq!(forced, warm_up_plan(image_sampler, true, describe, atlas));
7923        }
7924    }
7925
7926    /// The first-frame shader set is exactly the distinct programs of the
7927    /// warm-up list's opening, in the order the opening first names them —
7928    /// four modules in the sampler family (the atlas pass's manual Uber
7929    /// program is one of its own), three in the manual family (where it is the
7930    /// family's Uber program) — and under the Uber kill switch the Uber
7931    /// module(s) the first frame's atlas and surface passes draw with.
7932    #[test]
7933    fn the_first_frame_shaders_are_the_programs_of_the_list_opening() {
7934        #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
7935        enum Key {
7936            Atlas,
7937            State(EnginePipeline, PaintClass),
7938        }
7939        /// A program: its family (sampler or manual) and its class.
7940        type Program = (bool, PaintClass);
7941
7942        for image_sampler in [false, true] {
7943            let atlas = move || {
7944                if image_sampler {
7945                    Key::Atlas
7946                } else {
7947                    Key::State(
7948                        crate::gpu::pipelines::ATLAS_STRIP_PIPELINE,
7949                        PaintClass::Uber,
7950                    )
7951                }
7952            };
7953            // The atlas pass always draws the manual family's Uber program.
7954            let program_of = |key: Key| -> Program {
7955                match key {
7956                    Key::Atlas => (false, PaintClass::Uber),
7957                    Key::State(_, class) => (image_sampler, class),
7958                }
7959            };
7960            let list = ordered_warm_up(image_sampler, false, Key::State, atlas);
7961            let mut opening: Vec<Program> = Vec::new();
7962            for key in &list[..5] {
7963                let program = program_of(*key);
7964                if !opening.contains(&program) {
7965                    opening.push(program);
7966                }
7967            }
7968            let set = first_frame_programs(
7969                false,
7970                |class| (image_sampler, class),
7971                || (false, PaintClass::Uber),
7972            );
7973            assert_eq!(set, opening, "the opening's programs, in its order");
7974            assert_eq!(set.len(), if image_sampler { 4 } else { 3 });
7975
7976            let forced = first_frame_programs(
7977                true,
7978                |class| (image_sampler, class),
7979                || (false, PaintClass::Uber),
7980            );
7981            let mut uber: Vec<Program> = vec![(false, PaintClass::Uber)];
7982            if image_sampler {
7983                uber.push((true, PaintClass::Uber));
7984            }
7985            assert_eq!(forced, uber, "the Uber module(s) alone");
7986        }
7987    }
7988
7989    /// The `frust-perf enc` line is a contract: a capture is graded by
7990    /// grepping its fields, so the field order and the names are pinned here
7991    /// rather than only by the formatter. `perf-trace`-only, with the line —
7992    /// `cargo test -p frust-engine --features perf-trace` is where it runs.
7993    #[cfg(feature = "perf-trace")]
7994    #[test]
7995    fn an_encode_trace_line_reports_every_column_in_order() {
7996        let mut trace = EncodeTrace {
7997            frames: 7,
7998            ..EncodeTrace::default()
7999        };
8000        // One row per column, each carrying a distinct value in its own
8001        // column, so a transposed or dropped field shows up as a wrong number
8002        // rather than only as a wrong name.
8003        let mut row = [0_u32; ENCODE_TRACE_ROW];
8004        for (column, slot) in row.iter_mut().enumerate() {
8005            *slot = (column as u32 + 1) * 1_000;
8006        }
8007        trace.window.push(row);
8008
8009        let line = trace.line();
8010        let phases: Vec<String> = ENCODE_TRACE_COLUMNS
8011            .iter()
8012            .enumerate()
8013            .map(|(column, name)| format!("{name}_us={}.0", column + 1))
8014            .collect();
8015        // The counts carry no unit suffix and are reported as the raw values
8016        // the row holds, not scaled to microseconds like the phases above.
8017        let counts: Vec<String> = ENCODE_TRACE_COUNTS
8018            .iter()
8019            .enumerate()
8020            .map(|(offset, name)| {
8021                format!(
8022                    "{name}={}",
8023                    (ENCODE_TRACE_COLUMNS.len() + offset + 1) * 1_000
8024                )
8025            })
8026            .collect();
8027        assert_eq!(
8028            line,
8029            format!(
8030                "frust-perf enc n=7 w=1 {} total_p95_us={}.0 {}",
8031                phases.join(" "),
8032                ENCODE_TRACE_COLUMNS.len(),
8033                counts.join(" "),
8034            ),
8035        );
8036    }
8037
8038    /// The event counts are sums over the window, the level counts medians,
8039    /// and the pinned order puts them where the contract says.
8040    #[cfg(feature = "perf-trace")]
8041    #[test]
8042    fn an_encode_trace_line_sums_events_and_takes_the_median_of_levels() {
8043        let mut trace = EncodeTrace::default();
8044        let base = ENCODE_TRACE_COLUMNS.len();
8045        for frame in 0..3_u32 {
8046            let mut row = [0_u32; ENCODE_TRACE_ROW];
8047            row[base] = 10 + frame; // draws: median 11
8048            row[base + 5] = 1; // replays: sum 3
8049            row[base + 6] = 2; // dirty_pages: sum 6
8050            row[base + 8] = frame; // glyph_misses: sum 3
8051            row[base + 10] = 4 + frame; // run_keys: median 5
8052            row[base + 11] = 100 + frame; // staged_bytes: median 101
8053            row[base + 12] = 1; // staging_busy: sum 3
8054            row[base + 15] = 2; // bind_groups: sum 6
8055            trace.window.push(row);
8056        }
8057        let line = trace.line();
8058        let tail = line.split(" total_p95_us=").nth(1).unwrap();
8059        let counts = tail.split_once(' ').unwrap().1;
8060        assert_eq!(
8061            counts,
8062            "draws=11 strips=0 alphas=0 glyph_draws=0 atlas_glyphs=0 replays=3 \
8063             dirty_pages=6 atlas_refused=0 glyph_misses=3 glyph_evicted=0 run_keys=5 \
8064             staged_bytes=101 staging_busy=3 staging_too_large=0 staging_buffers=0 \
8065             bind_groups=6"
8066        );
8067    }
8068
8069    /// The staging and bind-group counts are per-frame deltas: each row carries
8070    /// only the change since the previous frame's reading, so a window's sum is
8071    /// the change across the window.
8072    #[cfg(feature = "perf-trace")]
8073    #[test]
8074    fn a_frame_row_carries_only_the_staging_and_bind_group_change_since_the_last() {
8075        let mut trace = EncodeTrace::default();
8076        let base = ENCODE_TRACE_COLUMNS.len();
8077        let readings = [
8078            TraceCounters {
8079                busy: 2,
8080                too_large: 0,
8081                buffers_created: 1,
8082                bind_groups: 4,
8083            },
8084            TraceCounters {
8085                busy: 5,
8086                too_large: 1,
8087                buffers_created: 1,
8088                bind_groups: 4,
8089            },
8090        ];
8091        for now in readings {
8092            trace.record(&EncodeSpans::default(), now);
8093        }
8094        assert_eq!(&trace.window[0][base + 12..], &[2, 0, 1, 4]);
8095        assert_eq!(&trace.window[1][base + 12..], &[3, 1, 0, 0]);
8096    }
8097
8098    /// A window's percentile is nearest-rank over the column it names, so the
8099    /// value reported is one the window really contains.
8100    #[cfg(feature = "perf-trace")]
8101    #[test]
8102    fn an_encode_trace_percentile_is_nearest_rank_per_column() {
8103        let mut trace = EncodeTrace::default();
8104        for value in [4_000_u32, 1_000, 3_000, 2_000] {
8105            let mut row = [0_u32; ENCODE_TRACE_ROW];
8106            row[0] = value;
8107            trace.window.push(row);
8108        }
8109        assert_eq!(trace.percentile(0, 50), 2_000);
8110        assert_eq!(trace.percentile(0, 95), 4_000);
8111        // An empty window reports zero rather than reaching past its end.
8112        trace.window.clear();
8113        assert_eq!(trace.percentile(0, 50), 0);
8114    }
8115
8116    /// Nanoseconds round up to a tenth of a microsecond, so a phase that cost
8117    /// anything at all never reports as free.
8118    #[cfg(feature = "perf-trace")]
8119    #[test]
8120    fn a_span_that_cost_anything_never_formats_as_zero() {
8121        assert_eq!(format_us(0), "0.0");
8122        assert_eq!(format_us(1), "0.1");
8123        assert_eq!(format_us(4_170_000), "4170.0");
8124    }
8125
8126    /// A staged upload covers the rows the frame fills and every row the last
8127    /// upload left live — so a frame whose coverage shrank re-zeroes the rows
8128    /// it gave up exactly once — and nothing past the texture.
8129    #[test]
8130    fn a_staged_upload_covers_filled_and_previously_live_rows() {
8131        assert_eq!(rows_filled(0, 32_768), 0);
8132        assert_eq!(rows_filled(1, 32_768), 1);
8133        assert_eq!(rows_filled(32_768, 32_768), 1);
8134        assert_eq!(rows_filled(32_769, 32_768), 2);
8135        // A new texture is live throughout: its first upload is a full one.
8136        assert_eq!(staged_rows(1, 8, 8), 8);
8137        // Shrinking: the rows the last frame filled are rewritten once...
8138        assert_eq!(staged_rows(2, 5, 8), 5);
8139        // ...and after that only the frame's own.
8140        assert_eq!(staged_rows(2, 2, 8), 2);
8141        // Growing past the live mark, never past the texture.
8142        assert_eq!(staged_rows(6, 2, 8), 6);
8143        assert_eq!(staged_rows(9, 2, 8), 8);
8144        // Nothing filled and nothing live: nothing to write.
8145        assert_eq!(staged_rows(0, 0, 8), 0);
8146    }
8147
8148    /// The compile phases partition the compile they sit inside, so their sum
8149    /// is the whole call minus the overhead the renderer measures around it.
8150    #[test]
8151    fn compile_phases_sum_without_overflowing() {
8152        let spans = CompileSpans {
8153            validate: Duration::from_micros(10),
8154            prepare: Duration::from_micros(20),
8155            classify: Duration::from_micros(30),
8156            admit: Duration::from_micros(40),
8157            walk: Duration::from_micros(50),
8158            // A subset of `walk`, so the sum must not grow by it.
8159            glyphs: Duration::from_micros(45),
8160            finish: Duration::from_micros(60),
8161        };
8162        assert_eq!(spans.total(), Duration::from_micros(210));
8163        assert_eq!(
8164            CompileSpans {
8165                validate: Duration::MAX,
8166                walk: Duration::MAX,
8167                ..CompileSpans::default()
8168            }
8169            .total(),
8170            Duration::MAX,
8171        );
8172    }
8173
8174    #[test]
8175    fn a_target_past_the_u16_grid_is_refused() {
8176        assert_eq!(
8177            grid_size(1920, 1080).expect("a normal surface"),
8178            (1920, 1080)
8179        );
8180        assert_eq!(
8181            grid_size(u32::from(u16::MAX), 16).expect("the grid ceiling itself"),
8182            (u16::MAX, 16)
8183        );
8184        assert!(matches!(
8185            grid_size(u32::from(u16::MAX) + 1, 16),
8186            Err(EngineError::TargetTooLarge)
8187        ));
8188        assert!(matches!(
8189            grid_size(16, u32::from(u16::MAX) + 1),
8190            Err(EngineError::TargetTooLarge)
8191        ));
8192    }
8193
8194    #[test]
8195    fn a_premultiplied_clear_folds_alpha_into_the_colour() {
8196        let half = RED.with_alpha(0.5);
8197        let premultiplied = clear_color(half, OutputAlpha::Premultiplied);
8198        let straight = clear_color(half, OutputAlpha::Straight);
8199
8200        assert!((premultiplied.r - 0.5).abs() < 1e-6);
8201        assert!((premultiplied.a - 0.5).abs() < 1e-6);
8202        assert!((straight.r - 1.0).abs() < 1e-6);
8203        assert!((straight.a - 0.5).abs() < 1e-6);
8204
8205        // An opaque colour is identical under both conventions.
8206        assert_eq!(
8207            clear_color(BLUE, OutputAlpha::Premultiplied),
8208            clear_color(BLUE, OutputAlpha::Straight)
8209        );
8210    }
8211
8212    #[test]
8213    fn a_solid_paint_travels_premultiplied_in_the_instance_payload() {
8214        let paint = pack_paint(&Paint::from(RED), 3, &[]).expect("a solid paint always resolves");
8215        let values = paint.values_at(40, 12);
8216        assert_eq!(values.paint, SOLID_PAINT);
8217        assert_eq!(values.depth_index, 3);
8218        assert_eq!(values.payload, RED.premultiply().to_rgba8().to_u32());
8219        assert!(paint.opaque);
8220        assert_eq!(
8221            paint.values_at(0, 0).payload,
8222            values.payload,
8223            "a solid colour is the same wherever it is stamped"
8224        );
8225
8226        let paint = pack_paint(&Paint::from(RED.with_alpha(0.5)), 0, &[])
8227            .expect("a translucent solid paint still resolves");
8228        assert!(
8229            !paint.opaque,
8230            "a translucent paint never reaches the opaque pass"
8231        );
8232    }
8233
8234    #[test]
8235    fn an_unresolvable_indexed_paint_skips_its_draw() {
8236        let indexed = Paint::Indexed(IndexedPaint::new(0));
8237        assert!(
8238            pack_paint(&indexed, 0, &[]).is_none(),
8239            "an index past the frame's slots resolves to nothing"
8240        );
8241        assert!(
8242            pack_paint(&indexed, 0, &[None]).is_none(),
8243            "so does a slot the lowering refused"
8244        );
8245    }
8246
8247    /// An image record is classed by the path it samples through — the one
8248    /// split an instance's flag cannot make — and every other record by its
8249    /// paint type.
8250    #[test]
8251    fn a_lowered_record_names_its_strip_program_class() {
8252        use crate::gpu::atlas::pack_image_params;
8253        use crate::gpu::paint_texture::GpuEncodedImage;
8254
8255        let image = |quality: u32, external: bool| {
8256            GpuEncodedPaint::Image(GpuEncodedImage {
8257                image_params: pack_image_params(quality, 0, 0, 3, external),
8258                image_size: 0,
8259                image_offset: 0,
8260                transform: [1.0, 0.0, 0.0, 1.0, 0.0, 0.0],
8261                tint: 0,
8262                tint_mode: 0,
8263                image_padding: 0,
8264            })
8265        };
8266        assert_eq!(class_of_record(&image(0, false)), PaintClass::Image);
8267        assert_eq!(class_of_record(&image(1, false)), PaintClass::Image);
8268        assert_eq!(class_of_record(&image(2, false)), PaintClass::ImageFiltered);
8269        assert_eq!(class_of_record(&image(1, true)), PaintClass::ImageFiltered);
8270    }
8271
8272    #[test]
8273    fn a_resolved_indexed_paint_samples_at_each_instances_own_position() {
8274        let slots = [Some(ResolvedPaint {
8275            paint_type: PaintType::LinearGradient,
8276            texel_offset: 6,
8277            opaque: true,
8278            external: None,
8279            class: PaintClass::Gradient,
8280        })];
8281        let paint = pack_paint(&Paint::Indexed(IndexedPaint::new(0)), 2, &slots)
8282            .expect("a lowered paint resolves");
8283
8284        assert_eq!(
8285            paint.paint,
8286            pack_paint_descriptor(PaintType::LinearGradient, 6)
8287        );
8288        assert_eq!(paint.depth_index, 2);
8289        // The payload is the instance's scene origin, not a colour, so two
8290        // instances of the same draw carry different payloads.
8291        assert_eq!(paint.payload_at(8, 4), 8 | (4 << 16));
8292        assert_eq!(paint.payload_at(40, 4), 40 | (4 << 16));
8293    }
8294
8295    #[test]
8296    fn a_u16_pair_packs_low_half_first() {
8297        assert_eq!(pack_u16_pair(0, 0), 0);
8298        assert_eq!(pack_u16_pair(1, 0), 1);
8299        assert_eq!(pack_u16_pair(0, 1), 1 << 16);
8300        assert_eq!(pack_u16_pair(u16::MAX, u16::MAX), u32::MAX);
8301    }
8302
8303    #[test]
8304    fn the_minimum_resource_dimension_keeps_every_upload_row_legal() {
8305        // The gradient LUT is the narrowest resource at four bytes per texel,
8306        // so it is the one that fixes the floor.
8307        assert_eq!(
8308            MIN_RESOURCE_TEXTURE_DIM * BYTES_PER_TEXEL,
8309            wgpu::COPY_BYTES_PER_ROW_ALIGNMENT
8310        );
8311        assert!(MIN_RESOURCE_TEXTURE_DIM.is_power_of_two());
8312    }
8313
8314    // -----------------------------------------------------------------
8315    // Where the hole punch lands in the pass plan
8316    //
8317    // What a frame's pass plan COSTS, and where the erase sits inside it,
8318    // is a decision over the recording's shape — no device, no pixels.
8319    // The pixel half of the same contract is
8320    // `frust-testing`'s `tests/aa_over_punch.rs`, against `vello_cpu`.
8321    // -----------------------------------------------------------------
8322
8323    /// Viewport every plan below is built against.
8324    const PLAN_VIEWPORT: (u16, u16) = (64, 48);
8325
8326    /// The slot, and a chip straddling its right edge — so a chip drawn over
8327    /// the punch is half inside it and half over the backdrop.
8328    const PLAN_SLOT: Rect = Rect::new(4.0, 4.0, 32.0, 44.0);
8329    const PLAN_CHIP: Rect = Rect::new(16.0, 12.0, 56.0, 32.0);
8330
8331    /// The plan `Scratch::build` produces for `scene`, alongside the rounds it
8332    /// was scheduled from, under a target that carries alpha.
8333    fn plan_of(scene: &Scene) -> (Scratch, Vec<Round>) {
8334        let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8335            .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
8336            .expect("an in-range scene compiles");
8337        let rounds = Schedule::build(
8338            &frame.recorder,
8339            &TierCaps::fake(DownlevelProfile::Full),
8340            &PageConfig::default(),
8341        )
8342        .expect("a scene of solid fills schedules");
8343
8344        let mut scratch = Scratch::default();
8345        // Depth on and punching on: the shape a translucent presentation
8346        // carrying a platform-view slot is planned under.
8347        scratch.build(&frame, &rounds, true, !frame.clears.is_empty(), &[], &[]);
8348        (scratch, rounds)
8349    }
8350
8351    /// A scene recorded through the public builder.
8352    fn plan_scene(record: impl FnOnce(&mut SceneBuilder<'_>)) -> Scene {
8353        let mut scene = Scene::new();
8354        let mut builder = SceneBuilder::new(&mut scene);
8355        record(&mut builder);
8356        scene
8357    }
8358
8359    fn plan_backdrop(builder: &mut SceneBuilder<'_>) {
8360        builder.fill_rect(
8361            Rect::new(
8362                0.0,
8363                0.0,
8364                f64::from(PLAN_VIEWPORT.0),
8365                f64::from(PLAN_VIEWPORT.1),
8366            ),
8367            Brush::Solid(RED),
8368        );
8369    }
8370
8371    #[test]
8372    fn a_frame_that_punches_nothing_is_planned_exactly_as_its_rounds() {
8373        let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8374            plan_backdrop(builder);
8375            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8376        }));
8377
8378        assert_eq!(
8379            scratch.rounds.len(),
8380            rounds.len(),
8381            "a frame with no clear is cut nowhere, so it costs exactly its \
8382             scheduled rounds' passes"
8383        );
8384        assert!(!scratch.punches(), "and records no punch pass at all");
8385    }
8386
8387    #[test]
8388    fn a_clear_recorded_last_keeps_its_pass_after_every_round() {
8389        let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8390            plan_backdrop(builder);
8391            builder.clear_rect(PLAN_SLOT);
8392        }));
8393
8394        assert_eq!(
8395            scratch.rounds.len(),
8396            rounds.len(),
8397            "nothing is recorded after the clear, so nothing is cut"
8398        );
8399        assert!(
8400            scratch.rounds.iter().all(|plan| plan.punch.1 == 0),
8401            "no round carries the punch"
8402        );
8403        assert!(
8404            scratch.punch.1 > 0,
8405            "it is issued after the last round instead — the position an \
8406             unconditionally-trailing pass would have put it in, which is why \
8407             a frame shaped like this renders byte-identically"
8408        );
8409    }
8410
8411    #[test]
8412    fn a_draw_over_a_clear_cuts_the_surface_round_and_the_punch_goes_in_the_cut() {
8413        let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8414            plan_backdrop(builder);
8415            builder.clear_rect(PLAN_SLOT);
8416            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8417        }));
8418
8419        assert_eq!(
8420            scratch.rounds.len(),
8421            rounds.len() + 1,
8422            "the surface round is cut in two — one extra pass, and one only"
8423        );
8424        assert_eq!(
8425            scratch.punch,
8426            (0, 0),
8427            "with nothing left over for a trailing pass"
8428        );
8429
8430        let cut = scratch
8431            .rounds
8432            .iter()
8433            .position(|plan| plan.punch.1 > 0)
8434            .expect("the cut plan carries the punch");
8435        assert_eq!(cut, scratch.rounds.len() - 2, "and it is the cut plan");
8436
8437        // The buffer says the same thing the plan does: the backdrop's
8438        // instances precede the punch's, and the chip's follow them.
8439        let before = scratch.rounds[cut].segments.clone();
8440        let after = scratch.rounds[cut + 1].segments.clone();
8441        let end_of = |range: Range<usize>| {
8442            scratch.segments[range]
8443                .iter()
8444                .map(|segment| match *segment {
8445                    Segment::Strips(first, count, _)
8446                    | Segment::External(first, count, _, _)
8447                    | Segment::Backdrop(first, count, _) => first + count,
8448                    Segment::Composite(first, _) => first + 1,
8449                })
8450                .max()
8451                .expect("a plan of this frame draws something")
8452        };
8453        let (punch_first, punch_count) = scratch.rounds[cut].punch;
8454        assert!(
8455            end_of(before) <= punch_first,
8456            "everything recorded before the clear is erased by the punch"
8457        );
8458        assert!(
8459            scratch.segments[after]
8460                .iter()
8461                .all(|segment| match *segment {
8462                    Segment::Strips(first, _, _)
8463                    | Segment::External(first, _, _, _)
8464                    | Segment::Backdrop(first, _, _)
8465                    | Segment::Composite(first, _) => first >= punch_first + punch_count,
8466                }),
8467            "and everything recorded after it lands on top of the erase"
8468        );
8469    }
8470
8471    // -----------------------------------------------------------------
8472    // The pass list: what a frame of each shape records, merged and apart
8473    //
8474    // Host-only: `Scratch::plan_passes` states the list over the plan, and
8475    // a debug build holds every frame `record_frame` really records to it,
8476    // so the device suites (`tests/encode_contract.rs` among them) check the
8477    // same list against a real encoder. The filter shape is pinned only here:
8478    // no `Scene` command records a filter layer, so no `encode` call reaches
8479    // a filter round.
8480    // -----------------------------------------------------------------
8481
8482    /// A rectangle off the pixel grid, so its edges are anti-aliased and the
8483    /// draw always reaches a surface round's blended strips — never only the
8484    /// opaque ones.
8485    const PLAN_SOFT: Rect = Rect::new(8.5, 8.5, 24.5, 40.5);
8486
8487    /// The passes `scratch` records with the opening merged, and apart.
8488    fn pass_lists(scratch: &Scratch) -> (Vec<FramePass>, Vec<FramePass>) {
8489        let shape = |merge| PassShape {
8490            merge,
8491            instances: true,
8492            opaque: !scratch.opaque.is_empty(),
8493            punch: scratch.punches(),
8494        };
8495        let mut merged = Vec::new();
8496        scratch.plan_passes(&shape(true), &mut merged);
8497        let mut apart = Vec::new();
8498        scratch.plan_passes(&shape(false), &mut apart);
8499        assert_opened_once(&merged);
8500        assert!(
8501            !apart.contains(&FramePass::Opening),
8502            "an unmerged frame records no opening: {apart:?}"
8503        );
8504        (merged, apart)
8505    }
8506
8507    /// A merged list opens its surface exactly once, before every other pass
8508    /// over it, and never records the clear or the opaque strips apart.
8509    fn assert_opened_once(passes: &[FramePass]) {
8510        let openings = passes
8511            .iter()
8512            .filter(|pass| **pass == FramePass::Opening)
8513            .count();
8514        assert_eq!(openings, 1, "one opening per frame: {passes:?}");
8515        let first_surface = passes.iter().position(|pass| pass.targets_surface());
8516        assert_eq!(
8517            first_surface,
8518            passes.iter().position(|pass| *pass == FramePass::Opening),
8519            "the opening is the first pass over the surface: {passes:?}"
8520        );
8521        assert!(
8522            !passes.contains(&FramePass::Clear) && !passes.contains(&FramePass::Opaque),
8523            "and the clear and opaque strips live inside it: {passes:?}"
8524        );
8525    }
8526
8527    #[test]
8528    fn an_empty_frame_is_the_opening_alone() {
8529        let (scratch, _) = plan_of(&plan_scene(|_| {}));
8530        let (merged, apart) = pass_lists(&scratch);
8531        assert_eq!(merged, [FramePass::Opening]);
8532        assert_eq!(apart, [FramePass::Clear]);
8533
8534        // And the same before any instance buffer exists at all.
8535        for merge in [true, false] {
8536            let mut passes = Vec::new();
8537            scratch.plan_passes(
8538                &PassShape {
8539                    merge,
8540                    instances: false,
8541                    opaque: false,
8542                    punch: false,
8543                },
8544                &mut passes,
8545            );
8546            assert_eq!(passes.len(), 1, "one clean clear: {passes:?}");
8547        }
8548    }
8549
8550    #[test]
8551    fn a_surface_only_frame_folds_its_clear_and_opaque_strips_into_one_pass() {
8552        let (scratch, _) = plan_of(&plan_scene(|builder| {
8553            plan_backdrop(builder);
8554            builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8555        }));
8556        assert!(!scratch.opaque.is_empty(), "the backdrop is opaque strips");
8557
8558        let (merged, apart) = pass_lists(&scratch);
8559        assert_eq!(merged, [FramePass::Opening]);
8560        assert_eq!(
8561            apart,
8562            [FramePass::Clear, FramePass::Opaque, FramePass::Surface]
8563        );
8564    }
8565
8566    #[test]
8567    fn without_depth_the_opening_is_the_clear_and_the_surface_round() {
8568        let scene = plan_scene(|builder| {
8569            plan_backdrop(builder);
8570            builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8571        });
8572        let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8573            .compile(&scene, Affine::IDENTITY, PLAN_VIEWPORT)
8574            .expect("an in-range scene compiles");
8575        let rounds = Schedule::build(
8576            &frame.recorder,
8577            &TierCaps::fake(DownlevelProfile::Full),
8578            &PageConfig::default(),
8579        )
8580        .expect("a scene of solid fills schedules");
8581        let mut scratch = Scratch::default();
8582        scratch.build(&frame, &rounds, false, false, &[], &[]);
8583        assert!(scratch.opaque.is_empty(), "no depth, no opaque strips");
8584
8585        let (merged, apart) = pass_lists(&scratch);
8586        assert_eq!(merged, [FramePass::Opening]);
8587        assert_eq!(apart, [FramePass::Clear, FramePass::Surface]);
8588    }
8589
8590    #[test]
8591    fn a_page_round_waits_for_the_opening_rather_than_splitting_it() {
8592        let (scratch, _) = plan_of(&plan_scene(|builder| {
8593            plan_backdrop(builder);
8594            builder.push_layer(PLAN_CHIP, 0.5);
8595            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8596            builder.pop_layer();
8597        }));
8598        let (merged, apart) = pass_lists(&scratch);
8599        // The page is rendered first either way — a layer round writes a
8600        // pooled page and reads neither the surface nor its depth — and the
8601        // surface round that composites it carries the opening.
8602        assert_eq!(merged, [FramePass::Page, FramePass::Opening]);
8603        assert_eq!(
8604            apart,
8605            [
8606                FramePass::Clear,
8607                FramePass::Opaque,
8608                FramePass::Page,
8609                FramePass::Surface
8610            ]
8611        );
8612    }
8613
8614    #[test]
8615    fn a_punch_cut_ends_the_opening_where_the_first_surface_pass_ended() {
8616        let (scratch, _) = plan_of(&plan_scene(|builder| {
8617            plan_backdrop(builder);
8618            builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8619            builder.clear_rect(PLAN_SLOT);
8620            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
8621        }));
8622        let (merged, apart) = pass_lists(&scratch);
8623        assert_eq!(
8624            merged,
8625            [FramePass::Opening, FramePass::Punch, FramePass::Surface]
8626        );
8627        assert_eq!(
8628            apart,
8629            [
8630                FramePass::Clear,
8631                FramePass::Opaque,
8632                FramePass::Surface,
8633                FramePass::Punch,
8634                FramePass::Surface
8635            ]
8636        );
8637    }
8638
8639    #[test]
8640    fn a_punch_ahead_of_every_surface_draw_takes_the_opening_alone() {
8641        // Nothing is recorded before the clear, so the plan the punch cuts
8642        // draws nothing: the opening — the backdrop's opaque strips inside it —
8643        // is recorded ahead of the erase on its own, and everything blended
8644        // after the cut loads.
8645        let (scratch, _) = plan_of(&plan_scene(|builder| {
8646            builder.clear_rect(PLAN_SLOT);
8647            plan_backdrop(builder);
8648            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
8649        }));
8650        let (merged, apart) = pass_lists(&scratch);
8651        assert_eq!(
8652            merged,
8653            [FramePass::Opening, FramePass::Punch, FramePass::Surface]
8654        );
8655        assert_eq!(
8656            apart,
8657            [
8658                FramePass::Clear,
8659                FramePass::Opaque,
8660                FramePass::Punch,
8661                FramePass::Surface
8662            ]
8663        );
8664    }
8665
8666    #[test]
8667    fn a_punch_recorded_last_follows_the_opening() {
8668        let (scratch, _) = plan_of(&plan_scene(|builder| {
8669            plan_backdrop(builder);
8670            builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8671            builder.clear_rect(PLAN_SLOT);
8672        }));
8673        let (merged, apart) = pass_lists(&scratch);
8674        assert_eq!(merged, [FramePass::Opening, FramePass::Punch]);
8675        assert_eq!(
8676            apart,
8677            [
8678                FramePass::Clear,
8679                FramePass::Opaque,
8680                FramePass::Surface,
8681                FramePass::Punch
8682            ]
8683        );
8684    }
8685
8686    #[test]
8687    fn a_filter_frame_records_its_filter_passes_before_the_opening() {
8688        use crate::filters::{LayerFilter, push_filter_layer};
8689        use vello_common::peniko::BlendMode;
8690        use vello_common::record::{CommandRecorder, LayerProps};
8691
8692        // Compiled as a backdrop and a chip, then re-recorded with the chip
8693        // inside a blurred layer under the surface: no `Scene` command
8694        // records a filter layer, so this is the only way to plan one.
8695        let mut frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8696            .compile(
8697                &plan_scene(|builder| {
8698                    plan_backdrop(builder);
8699                    builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8700                }),
8701                Affine::IDENTITY,
8702                PLAN_VIEWPORT,
8703            )
8704            .expect("an in-range scene compiles");
8705        let draws = frame.recorder.draws.clone();
8706        assert_eq!(draws.len(), 2, "the backdrop and the chip");
8707        let mut recorder = CommandRecorder::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1);
8708        let strips_of = |draw: &crate::EngineDraw| &frame.strip_buf()[draw.strip_range.clone()];
8709        recorder.push_draw(draws[0].clone(), strips_of(&draws[0]));
8710        push_filter_layer(
8711            &mut recorder,
8712            LayerProps {
8713                blend_mode: BlendMode::default(),
8714                opacity: 1.0,
8715                mask: None,
8716                clip_path: None,
8717            },
8718            LayerFilter::Blur { sigma: 2.0 },
8719            Affine::IDENTITY,
8720        )
8721        .expect("a finite σ under the ceiling records");
8722        recorder.push_draw(draws[1].clone(), strips_of(&draws[1]));
8723        recorder.pop_layer();
8724        frame.recorder = recorder;
8725
8726        let rounds = Schedule::build(
8727            &frame.recorder,
8728            &TierCaps::fake(DownlevelProfile::Full),
8729            &PageConfig::default(),
8730        )
8731        .expect("a blurred layer under the surface schedules");
8732        let mut scratch = Scratch::default();
8733        scratch.build(&frame, &rounds, true, false, &[], &[]);
8734        assert!(scratch.filter_passes > 0, "the layer is filtered");
8735
8736        let (merged, apart) = pass_lists(&scratch);
8737        let filters = |passes: &[FramePass]| {
8738            passes
8739                .iter()
8740                .filter(|pass| **pass == FramePass::Filter)
8741                .count()
8742        };
8743        assert_eq!(filters(&merged), scratch.filter_passes as usize);
8744        assert_eq!(filters(&apart), filters(&merged));
8745        assert_eq!(
8746            merged.len() + 2,
8747            apart.len(),
8748            "the clear and the opaque strips fold into the surface round: \
8749             {merged:?} against {apart:?}"
8750        );
8751        assert_eq!(merged.first(), Some(&FramePass::Page));
8752        assert_eq!(
8753            merged.last(),
8754            Some(&FramePass::Opening),
8755            "every filter pass runs before the surface opens: {merged:?}"
8756        );
8757    }
8758
8759    // -----------------------------------------------------------------
8760    // A banded layer's column pages hold their own column, and only it
8761    //
8762    // The scheduler hands every band of a layer the same op list, so the
8763    // instances a band emits are where a column split is made or lost. Every
8764    // case here is host-only: the plan is built with no device, and the pixel
8765    // half of the same contract is `tests/desktop_stress.rs`'s 5K cases.
8766    // -----------------------------------------------------------------
8767
8768    /// Viewport the band plans below are built against: wide enough that one
8769    /// layer over all of it needs several column pages under [`BAND_PAGES`],
8770    /// and one tile row tall, so a draw costs one strip row per column.
8771    const BAND_VIEWPORT: (u16, u16) = (200, 8);
8772
8773    /// A page ceiling small enough to band a test-sized layer, so no case here
8774    /// has to allocate — or even name — a 5K one.
8775    const BAND_PAGES: PageConfig = PageConfig {
8776        min_page_size: 64,
8777        max_page_size: 64,
8778    };
8779
8780    /// The plan `scene` produces under `config`, alongside the rounds it was
8781    /// scheduled into.
8782    ///
8783    /// Depth is off, so every instance of the frame lands in the one blended
8784    /// buffer in painter order and a case can read the whole plan out of it;
8785    /// the split into the depth-writing pass is a surface-round decision and a
8786    /// page round never takes it.
8787    fn band_plan_of(scene: &Scene, config: &PageConfig) -> (Scratch, Vec<Round>) {
8788        let frame = SceneCompiler::new(BAND_VIEWPORT.0, BAND_VIEWPORT.1)
8789            .compile(scene, Affine::IDENTITY, BAND_VIEWPORT)
8790            .expect("an in-range scene compiles");
8791        let rounds = Schedule::build(
8792            &frame.recorder,
8793            &TierCaps::fake(DownlevelProfile::Full),
8794            config,
8795        )
8796        .expect("a layer wider than the ceiling bands rather than refusing");
8797
8798        let mut scratch = Scratch::default();
8799        scratch.build(&frame, &rounds, false, false, &[], &[]);
8800        assert_eq!(
8801            scratch.rounds.len(),
8802            rounds.len(),
8803            "a frame with no clear is cut nowhere, so the plans and the rounds \
8804             line up one for one"
8805        );
8806        (scratch, rounds)
8807    }
8808
8809    /// A layer at half opacity — so it cannot be inlined and has to take a page
8810    /// — holding one solid rectangle per entry of `rects`.
8811    fn band_scene(rects: &[(Rect, Color)]) -> Scene {
8812        let mut scene = Scene::new();
8813        let mut builder = SceneBuilder::new(&mut scene);
8814        builder.push_layer(
8815            Rect::new(
8816                0.0,
8817                0.0,
8818                f64::from(BAND_VIEWPORT.0),
8819                f64::from(BAND_VIEWPORT.1),
8820            ),
8821            0.5,
8822        );
8823        for (rect, color) in rects {
8824            builder.fill_rect(*rect, Brush::Solid(*color));
8825        }
8826        builder.pop_layer();
8827        scene
8828    }
8829
8830    /// The bounds of every page round, in order — one band's column each.
8831    fn band_bounds(rounds: &[Round]) -> Vec<RectU16> {
8832        rounds
8833            .iter()
8834            .filter_map(|round| round.page().map(|page| page.bounds))
8835            .collect()
8836    }
8837
8838    /// The strip instances one round plan issues, in execution order.
8839    fn plan_strips(scratch: &Scratch, plan: &RoundPlan) -> Vec<GpuStrip> {
8840        scratch.segments[plan.segments.clone()]
8841            .iter()
8842            .filter_map(|segment| match *segment {
8843                Segment::Strips(first, count, _)
8844                | Segment::External(first, count, _, _)
8845                | Segment::Backdrop(first, count, _) => Some((first as usize, count as usize)),
8846                Segment::Composite(..) => None,
8847            })
8848            .flat_map(|(first, count)| scratch.alpha[first..first + count].iter().copied())
8849            .collect()
8850    }
8851
8852    /// Every pixel column the frame's *page* rounds paint, mapped back out of
8853    /// the pages they were shifted into, and valued by what the shader reads
8854    /// there: the instance's paint payload, plus the exact alpha column that
8855    /// pixel samples (`None` for a sparse instance, which samples none).
8856    ///
8857    /// Keyed by the draw's own depth as well as the position, so a page painted
8858    /// by two different draws is not conflated — and so a *second* instance of
8859    /// one draw covering a pixel it already covered, which is precisely what a
8860    /// clamped band replay produces, is caught here rather than silently
8861    /// overwriting the first.
8862    fn painted_pixels(
8863        scratch: &Scratch,
8864        rounds: &[Round],
8865    ) -> BTreeMap<(u32, u16, u16), (u32, Option<u32>)> {
8866        let mut painted = BTreeMap::new();
8867
8868        for (plan, round) in scratch.rounds.iter().zip(rounds) {
8869            let Some(page) = round.page() else {
8870                continue;
8871            };
8872            for span in plan_strips(scratch, plan) {
8873                for offset in 0..span.width {
8874                    let x = page.bounds.x0 + span.x + offset;
8875                    let y = page.bounds.y0 + span.y;
8876                    let column = (offset < span.dense_width_or_rect_height)
8877                        .then(|| span.col_idx_or_rect_frac + u32::from(offset));
8878                    assert!(
8879                        painted
8880                            .insert((span.depth_index, y, x), (span.payload, column))
8881                            .is_none(),
8882                        "one draw covers a device pixel at most once, however the \
8883                         layer holding it was split"
8884                    );
8885                }
8886            }
8887        }
8888
8889        painted
8890    }
8891
8892    #[test]
8893    fn a_banded_layer_paints_exactly_what_one_page_would_have() {
8894        // The whole point of the split, asserted as an equality rather than as
8895        // a rectangle property: the same scene planned onto one page and onto
8896        // column bands has to paint the same device pixels, from the same
8897        // paints, sampling the same alpha columns. A band replay that clamped
8898        // a strip left of its own column onto the page's edge fails here twice
8899        // over — once on the ghost pixel, once on the column it would sample.
8900        let scene = band_scene(&[
8901            (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8902            // Straddles a band edge on a tile that is only partly covered, so
8903            // the case exercises an alpha-sampled instance cut in two, not
8904            // just a solid one.
8905            (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
8906            (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
8907        ]);
8908
8909        let (one_page, one_page_rounds) = band_plan_of(&scene, &PageConfig::default());
8910        let (banded, banded_rounds) = band_plan_of(&scene, &BAND_PAGES);
8911
8912        assert_eq!(
8913            band_bounds(&one_page_rounds).len(),
8914            1,
8915            "the reference plan really does hold the layer on one page"
8916        );
8917        assert!(
8918            band_bounds(&banded_rounds).len() > 1,
8919            "and the case only means anything while the other one is banded"
8920        );
8921
8922        assert_eq!(
8923            painted_pixels(&banded, &banded_rounds),
8924            painted_pixels(&one_page, &one_page_rounds),
8925            "a banded layer paints what one whole-layer page would have"
8926        );
8927    }
8928
8929    #[test]
8930    fn a_band_holds_no_instance_of_a_strip_outside_its_own_column() {
8931        // The counterexample the equality above generalizes: content confined
8932        // to the outer columns, and nothing at all in the middle. A band whose
8933        // column the scene never drew in must render nothing — under a
8934        // saturating shift it would render the leftmost content clamped onto
8935        // its own edge instead.
8936        let scene = band_scene(&[
8937            (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8938            (Rect::new(160.0, 0.0, 200.0, 8.0), BLUE),
8939        ]);
8940        let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
8941        let bands = band_bounds(&rounds);
8942        assert!(bands.len() > 2, "the layer bands: {bands:?}");
8943
8944        let mut empty = 0_usize;
8945        for (plan, band) in scratch
8946            .rounds
8947            .iter()
8948            .zip(&rounds)
8949            .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
8950        {
8951            let strips = plan_strips(&scratch, plan);
8952            if strips.is_empty() {
8953                empty += 1;
8954            }
8955            for span in strips {
8956                let end = span.x + span.width;
8957                assert!(
8958                    end <= band.width(),
8959                    "an instance of {span:?} reaches past the {band:?} band's own \
8960                     column, which its composite never samples"
8961                );
8962                // Mapped back to the scene, every instance lands where one of
8963                // the two rectangles was actually drawn.
8964                let x = band.x0 + span.x;
8965                assert!(
8966                    x < 44 || band.x0 + end > 160,
8967                    "an instance covers device x {x}..{}, which neither \
8968                     rectangle reaches",
8969                    band.x0 + end
8970                );
8971            }
8972        }
8973
8974        assert!(
8975            empty > 0,
8976            "a band whose column holds nothing renders nothing: {bands:?}"
8977        );
8978    }
8979
8980    #[test]
8981    fn a_strip_straddling_a_band_edge_advances_its_alpha_column_with_its_geometry() {
8982        // A pixel-aligned rectangle's left edge lands on a tile of its own, so
8983        // a rectangle starting at 49 puts an alpha-sampled instance across
8984        // 48..52 — and a band edge falls inside it. The two pieces together
8985        // have to read the same coverage the whole instance would: neighbouring
8986        // pixels of one strip row sampling neighbouring alpha columns, with no
8987        // column repeated. The rectangles either side of it are what carries
8988        // the layer past the ceiling, so the middle one is banded at all.
8989        let scene = band_scene(&[
8990            (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8991            (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
8992            (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
8993        ]);
8994        let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
8995        let edges: Vec<u16> = band_bounds(&rounds)
8996            .iter()
8997            .map(|bounds| bounds.x0)
8998            .collect();
8999
9000        // Every alpha-sampled pixel of every band, as (strip row, device x,
9001        // the alpha column it samples) — a row at a time, because two rows of
9002        // one draw sample different columns at the same x by construction.
9003        let mut dense: Vec<(u16, u16, u32)> = Vec::new();
9004        for (plan, band) in scratch
9005            .rounds
9006            .iter()
9007            .zip(&rounds)
9008            .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
9009        {
9010            for span in plan_strips(&scratch, plan) {
9011                for offset in 0..span.dense_width_or_rect_height {
9012                    dense.push((
9013                        band.y0 + span.y,
9014                        band.x0 + span.x + offset,
9015                        span.col_idx_or_rect_frac + u32::from(offset),
9016                    ));
9017                }
9018            }
9019        }
9020        dense.sort_unstable();
9021
9022        let neighbours = || {
9023            dense
9024                .windows(2)
9025                .map(|pair| (pair[0], pair[1]))
9026                .filter(|((row, x, _), (next_row, next_x, _))| row == next_row && x + 1 == *next_x)
9027        };
9028        assert!(
9029            neighbours().any(|(_, (_, x, _))| edges.contains(&x)),
9030            "the case only means anything while an alpha-sampled instance \
9031             really is cut by a band edge {edges:?}: {dense:?}"
9032        );
9033
9034        for ((_, _, column), (_, x, next_column)) in neighbours() {
9035            assert_eq!(
9036                next_column,
9037                column + 1,
9038                "neighbouring pixels of one strip row sample neighbouring alpha \
9039                 columns, band edge at {x} or not: {dense:?}"
9040            );
9041        }
9042    }
9043
9044    #[test]
9045    fn a_translucent_layer_over_a_clear_is_composited_after_the_punch() {
9046        let (scratch, rounds) = plan_of(&plan_scene(|builder| {
9047            plan_backdrop(builder);
9048            builder.clear_rect(PLAN_SLOT);
9049            builder.push_layer(PLAN_CHIP, 0.5);
9050            builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
9051            builder.pop_layer();
9052        }));
9053
9054        assert_eq!(
9055            scratch.rounds.len(),
9056            rounds.len() + 1,
9057            "the layer's page round, then the surface round cut in two"
9058        );
9059        let cut = scratch
9060            .rounds
9061            .iter()
9062            .position(|plan| plan.punch.1 > 0)
9063            .expect("the cut plan carries the punch");
9064        // The composite is what the cut has to fall before: a layer recorded
9065        // after the clear carries a deeper index than the punch, and the
9066        // composite writes no depth for the test to save it by.
9067        let composited = scratch.segments[scratch.rounds[cut + 1].segments.clone()]
9068            .iter()
9069            .any(|segment| matches!(segment, Segment::Composite(..)));
9070        assert!(
9071            composited,
9072            "the layer composites in the plan AFTER the punch, not before it"
9073        );
9074        assert!(
9075            !scratch.segments[scratch.rounds[cut].segments.clone()]
9076                .iter()
9077                .any(|segment| matches!(segment, Segment::Composite(..))),
9078            "and nothing composites into the plan the punch closes"
9079        );
9080    }
9081
9082    // -----------------------------------------------------------------
9083    // Backdrop blurs: the copy round, the sampled page, the opaque limit
9084    //
9085    // Host-only, over the same plan the frame path records from: a
9086    // backdrop's region copy is the one round besides a punch that must
9087    // follow the opening, and its rectangle the one strip run that samples
9088    // a page rather than a paint.
9089    // -----------------------------------------------------------------
9090
9091    /// The rectangle every backdrop case below blurs behind.
9092    const PLAN_PANEL: Rect = Rect::new(8.0, 8.0, 40.0, 36.0);
9093
9094    /// [`plan_of`] for a frame whose backdrops are served — scheduled through
9095    /// [`Schedule::build_frame`] and planned with the frame's own backdrop
9096    /// records, as `encode` does on a readable target.
9097    fn backdrop_plan_of(scene: &Scene, punches: bool) -> (Scratch, Vec<Round>, CompiledFrame) {
9098        let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
9099            .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
9100            .expect("an in-range scene compiles");
9101        let rounds = Schedule::build_frame(
9102            &frame.recorder,
9103            &TierCaps::fake(DownlevelProfile::Full),
9104            &PageConfig::default(),
9105            &RetainedLayers::new(),
9106            &frame.backdrops,
9107        )
9108        .expect("a backdrop under the surface schedules");
9109        let mut scratch = Scratch::default();
9110        scratch.build(&frame, &rounds, true, punches, &[], &frame.backdrops);
9111        (scratch, rounds, frame)
9112    }
9113
9114    /// An opaque field, the backdrop, and a chip painted over the panel.
9115    fn backdrop_scene() -> Scene {
9116        plan_scene(|builder| {
9117            builder.fill_rect(
9118                Rect::new(
9119                    0.0,
9120                    0.0,
9121                    f64::from(PLAN_VIEWPORT.0),
9122                    f64::from(PLAN_VIEWPORT.1),
9123                ),
9124                Brush::Solid(BLUE),
9125            );
9126            builder.backdrop_blur(PLAN_PANEL, 2.0);
9127            builder.fill_rect(Rect::new(12.0, 12.0, 20.0, 20.0), Brush::Solid(RED));
9128        })
9129    }
9130
9131    #[test]
9132    fn a_backdrop_ends_the_opening_before_its_copy_and_draws_after_its_passes() {
9133        let (scratch, _, frame) = backdrop_plan_of(&backdrop_scene(), false);
9134        assert_eq!(frame.backdrops.len(), 1);
9135        let (merged, apart) = pass_lists(&scratch);
9136        // σ 2 convolves directly: one horizontal and one vertical pass.
9137        assert_eq!(
9138            merged,
9139            [
9140                FramePass::Opening,
9141                FramePass::Copy,
9142                FramePass::Filter,
9143                FramePass::Filter,
9144                FramePass::Surface,
9145            ],
9146            "the opening — clear, opaque strips and the field — lands before the copy reads \
9147             the surface back, and the panel and chip draw after the blur"
9148        );
9149        assert_eq!(
9150            apart,
9151            [
9152                FramePass::Clear,
9153                FramePass::Opaque,
9154                FramePass::Surface,
9155                FramePass::Copy,
9156                FramePass::Filter,
9157                FramePass::Filter,
9158                FramePass::Surface,
9159            ],
9160            "unmerged, the same order with the clear and the opaque strips apart"
9161        );
9162    }
9163
9164    #[test]
9165    fn a_backdrops_rectangle_samples_the_page_its_round_releases() {
9166        let (scratch, rounds, frame) = backdrop_plan_of(&backdrop_scene(), false);
9167        let backdrop = frame.backdrops[0];
9168        let sampling = rounds
9169            .iter()
9170            .position(|round| !round.samples.is_empty())
9171            .expect("one surface round samples the blurred page");
9172        let sample = rounds[sampling].samples[0];
9173        assert_eq!(sample.layer, backdrop.layer);
9174        assert!(
9175            rounds[sampling].released.contains(&sample.parity),
9176            "the round that samples the page hands it back"
9177        );
9178
9179        let plan = &scratch.rounds[sampling];
9180        let segments = &scratch.segments[plan.segments.clone()];
9181        let (first, count, parity) = segments
9182            .iter()
9183            .find_map(|segment| match *segment {
9184                Segment::Backdrop(first, count, parity) => Some((first, count, parity)),
9185                _ => None,
9186            })
9187            .expect("the panel is drawn as a backdrop segment");
9188        assert_eq!(parity, sample.parity);
9189        assert!(count > 0);
9190        for instance in &scratch.alpha[first as usize..(first + count) as usize] {
9191            assert_eq!(
9192                instance.paint_and_rect_flag & !crate::gpu::strips::RECT_STRIP_FLAG,
9193                LAYER_PAINT_SOURCE | u32::from(u8::MAX),
9194                "every panel instance reads the layer input at full opacity"
9195            );
9196            let (page_x, page_y) = (instance.payload & 0xffff, instance.payload >> 16);
9197            assert_eq!(
9198                (page_x, page_y),
9199                (
9200                    u32::from(instance.x - backdrop.region.x0),
9201                    u32::from(instance.y - backdrop.region.y0)
9202                ),
9203                "and samples the page texel under its own device origin"
9204            );
9205        }
9206    }
9207
9208    #[test]
9209    fn draws_after_a_backdrop_never_take_the_depth_writing_pass() {
9210        // The field recorded *after* the backdrop: its opaque interior must
9211        // not be drawn in the opening, ahead of the copy that reads the
9212        // surface back.
9213        let after = plan_scene(|builder| {
9214            builder.backdrop_blur(PLAN_PANEL, 2.0);
9215            builder.fill_rect(
9216                Rect::new(
9217                    0.0,
9218                    0.0,
9219                    f64::from(PLAN_VIEWPORT.0),
9220                    f64::from(PLAN_VIEWPORT.1),
9221                ),
9222                Brush::Solid(BLUE),
9223            );
9224        });
9225        let (scratch, _, _) = backdrop_plan_of(&after, false);
9226        assert!(
9227            scratch.opaque.is_empty(),
9228            "nothing at or past the backdrop's depth is routed opaque"
9229        );
9230
9231        // The same field before the backdrop keeps its opaque spans.
9232        let (before, _, _) = backdrop_plan_of(&backdrop_scene(), false);
9233        assert!(!before.opaque.is_empty());
9234    }
9235
9236    #[test]
9237    fn a_punch_ahead_of_a_backdrop_is_erased_before_the_copy() {
9238        let scene = plan_scene(|builder| {
9239            builder.fill_rect(PLAN_SLOT, Brush::Solid(BLUE));
9240            builder.clear_rect(PLAN_SLOT);
9241            builder.backdrop_blur(PLAN_PANEL, 2.0);
9242        });
9243        let (scratch, _, _) = backdrop_plan_of(&scene, true);
9244        let (merged, _) = pass_lists(&scratch);
9245        let punch = merged
9246            .iter()
9247            .position(|pass| *pass == FramePass::Punch)
9248            .expect("the punch is recorded");
9249        let copy = merged
9250            .iter()
9251            .position(|pass| *pass == FramePass::Copy)
9252            .expect("the copy is recorded");
9253        assert!(
9254            punch < copy,
9255            "the backdrop reads the erased surface, not the field the punch erases: {merged:?}"
9256        );
9257    }
9258
9259    #[test]
9260    fn a_target_that_cannot_be_read_back_serves_no_backdrop() {
9261        // `encode` hands the plan no backdrop records on such a target: the
9262        // panel is the transparent draw it was recorded as, scheduled as the
9263        // ordinary layer it is — no copy, no filter pass, no sampled page.
9264        let (scratch, rounds) = plan_of(&backdrop_scene());
9265        assert!(
9266            rounds
9267                .iter()
9268                .all(|round| round.copy.is_none() && round.samples.is_empty())
9269        );
9270        let (merged, _) = pass_lists(&scratch);
9271        assert_eq!(merged, [FramePass::Opening]);
9272        assert_eq!(scratch.copies, 0);
9273    }
9274
9275    #[test]
9276    fn the_region_copy_mode_prefers_sampling_and_needs_a_single_sample() {
9277        use wgpu::TextureFormat::{Bgra8Unorm, Rgba8Unorm, Rgba8UnormSrgb};
9278        use wgpu::TextureUsages as U;
9279        let attach = U::RENDER_ATTACHMENT;
9280        assert_eq!(
9281            RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8Unorm, 1),
9282            RegionCopyMode::Texture,
9283            "a headless readback target in the pages' own format is copied"
9284        );
9285        assert_eq!(
9286            RegionCopyMode::for_usage(attach | U::COPY_SRC | U::TEXTURE_BINDING, Rgba8Unorm, 1),
9287            RegionCopyMode::Fragment,
9288            "sampling costs the one pass alone, so it wins whenever it is allowed"
9289        );
9290        assert_eq!(
9291            RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING, Bgra8Unorm, 1),
9292            RegionCopyMode::Fragment,
9293            "any format can be sampled across into a page"
9294        );
9295        assert_eq!(
9296            RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8UnormSrgb, 1),
9297            RegionCopyMode::Unavailable,
9298            "a copy would move encoded bytes into a linear page"
9299        );
9300        assert_eq!(
9301            RegionCopyMode::for_usage(attach, Bgra8Unorm, 1),
9302            RegionCopyMode::Unavailable,
9303            "a swapchain is RENDER_ATTACHMENT only"
9304        );
9305        assert_eq!(
9306            RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING | U::COPY_SRC, Rgba8Unorm, 4),
9307            RegionCopyMode::Unavailable,
9308            "a multisampled target is neither"
9309        );
9310    }
9311}