frust_engine/renderer.rs
1//! [`EngineRenderer`]: the public seam a host drives one surface's 2D frames
2//! through.
3//!
4//! # The encode contract
5//!
6//! [`EngineRenderer::encode`] **records into a `wgpu::CommandEncoder` the
7//! caller owns and never submits it.** That is the whole point of the seam: a
8//! host compositing 2D over its own 3D content records its passes before and
9//! after the engine's into one encoder and submits once, and a submit hidden
10//! inside the engine would split that into two command buffers with a pipeline
11//! flush between them. Two obligations follow, and both are contract rather
12//! than preference:
13//!
14//! - every pass the engine begins is ended before `encode` returns, so the
15//! caller's next `begin_render_pass` on the same encoder is legal; and
16//! - the engine issues no `queue.submit` for scene work. It does issue
17//! `queue.write_texture`/`write_buffer` uploads, which are ordered ahead of
18//! the command buffers submitted after them and so land before the passes
19//! that read them.
20//!
21//! One thing the engine *does* submit, and it is worth naming precisely because
22//! the rule above is otherwise absolute: growing the image atlas array submits a
23//! command buffer of its own, holding one texture copy and nothing else, on the
24//! rare frame that grows it. That is **maintenance, not scene work** — it never
25//! touches the caller's encoder, records no pass and no draw, and exists because
26//! the queued writes it has to precede would otherwise be flushed ahead of it
27//! (see [`crate::gpu::atlas`]'s *Why growth submits a command buffer of its
28//! own*). The caller's encoder is still never submitted by the engine, and the
29//! frame's own passes still reach the queue only when the caller submits it.
30//!
31//! # Sharing the depth attachment
32//!
33//! A caller recording its own depth-writing passes into that encoder hands the
34//! same attachment in as `EngineTarget::depth`, and the two renderers then
35//! occlude each other correctly in either order. Three rules make that work,
36//! and [`crate::gpu::depth`] is where they are stated in full: the shared
37//! comparison and which end of the range is near
38//! ([`DEPTH_COMPARE`](crate::gpu::depth::DEPTH_COMPARE) over a buffer whose far
39//! plane is [`DEPTH_CLEAR`](crate::gpu::depth::DEPTH_CLEAR)); the depth
40//! attachment's extent matching the colour target's; and who owns the clear —
41//! whichever pass runs first in the encoder, which the caller states through
42//! [`EngineRenderer::set_depth_pre_cleared`].
43//!
44//! What that contract does *not* extend to is colour. The frame's opening
45//! below clears the frame's colour target unconditionally, so content painted
46//! into that target before `encode` keeps its depth and loses its pixels: a
47//! host compositing over its own content records that content after the frame,
48//! or into a target of its own the frame composites.
49//!
50//! # The frame's passes
51//!
52//! A frame records into that encoder, in this order.
53//!
54//! 1. **The opening** ([`FramePass::Opening`]), the first pass over the
55//! surface, which does three things in one render pass:
56//! - **clears** the colour target to the frame's base colour, and the depth
57//! attachment to the far plane unless the caller stated it is already
58//! populated (see [`crate::gpu::depth`]) — the pass's own load
59//! operations, so a frame with no draws at all still resolves to a clean
60//! surface;
61//! - draws the **opaque strips**, depth-tested and depth-writing,
62//! unblended: only the fully-covered interior spans of opaque draws
63//! *targeting the surface* — an anti-aliased edge is by definition not
64//! opaque, and a page carries no depth attachment for the split to be
65//! sound against. The depth they establish is what lets every blended
66//! strip after them reject fragments an opaque draw in front of it
67//! already covered;
68//! - then draws the **first surface round's own strips** (pass 2 below),
69//! in order, the pipeline switching between the two inside the pass.
70//!
71//! It ends exactly where that round's own pass would have ended — at a
72//! punch cut, at a page or filter round, or at the end of the frame. Page
73//! and filter rounds scheduled ahead of the first surface round run ahead
74//! of the opening: they write pooled pages and read neither the surface nor
75//! its depth, so the opening waits past them. A punch issued before any
76//! surface round draws takes the opening as a pass of its own, ahead of the
77//! erase, and so does the end of a frame whose surface rounds drew nothing
78//! blended.
79//!
80//! The opaque strips are drawn once, not once per surface round, and that
81//! is a correctness requirement: the surface can take several rounds (a
82//! [cut](crate::schedule::cut_at) round is how a wide sibling fan is
83//! served), and re-drawing them ahead of each would re-draw opaque coverage
84//! at equal stored depth over composites the round before it had already
85//! blended.
86//!
87//! One pass rather than three is what a tile-based GPU is paid for: every
88//! pass over the surface is a whole-framebuffer load into tile memory and a
89//! store back out, and the clear and the opaque strips were two such passes
90//! that drew little or nothing of their own. The pixels are identical —
91//! the same draws in the same order against the same attachments, only the
92//! pass boundaries gone. `FRUST_ENGINE_NO_PASS_MERGE`
93//! ([`EngineRenderer::set_pass_merge`]) records them apart again: a
94//! [`FramePass::Clear`] pass drawing nothing and a [`FramePass::Opaque`]
95//! pass loading it, ahead of every round, and then every surface round in
96//! a pass of its own.
97//! 2. **One pass per [round](crate::schedule)**, the first surface round that
98//! draws excepted — it is drawn inside the opening. A page round
99//! ([`FramePass::Page`]) renders one isolated layer into a pooled
100//! intermediate [page](crate::schedule::pages) it clears to transparent, at
101//! the page's own origin — so every instance is shifted by the page's
102//! tile-aligned bounds, and clipped to them, which is what makes a
103//! [banded](crate::schedule::pages::page_bands) layer's column pages tile
104//! their layer instead of each holding a clamped copy of it (see
105//! [`PageWindow`]) — and through the page's own viewport uniform; a round
106//! continuing a page an earlier round of the same layer opened loads it
107//! instead. A surface round ([`FramePass::Surface`]) draws **alpha
108//! strips**, premultiplied-blended, in painter order, loading the surface:
109//! depth-tested but not depth-writing when a depth attachment is in play, a
110//! plain painter's-algorithm pass when it is not. A surface round with
111//! nothing blended to draw records no pass. A round's ops run in the order
112//! [`Schedule::build`] listed them, so a finished child page composites into
113//! its parent exactly where the recording entered it.
114//!
115//! A **filter round** is the one round that draws no strip at all: it runs
116//! one pass of a [filter](crate::filters)'s sequence, one instanced quad
117//! through [`EnginePipeline::Filter`], reading the layer's other pooled page
118//! through the engine's only sampler and clearing the page it writes (see
119//! [`FilterResources`]). It is an ordinary round of this walk in every other
120//! respect — recorded into the caller's own encoder, in the order the
121//! scheduler listed it, with its pages handed back the moment its pass ends.
122//! It is *not* an own-encoder exception; the atlas replay remains the only
123//! one of those.
124//!
125//! A **region-copy round** ([`FramePass::Copy`]) is a backdrop blur's
126//! first: it reads a region of the frame's own target back into a pooled
127//! page ([`RegionCopyMode`]) for the blur's filter rounds to run over. It is
128//! the one page round that *follows* the opening rather than waiting past
129//! it — it reads the surface, so the clear, the opaque strips and every
130//! surface round ahead of it in painter order must have landed (the
131//! scheduler cuts the surface round there; a punch recorded ahead of it is
132//! erased first) — and, for the same reason, no draw at or past the first
133//! backdrop's depth is routed into the opaque strips. The backdrop's own
134//! rectangle is then drawn in the next surface round as strip instances
135//! reading the blurred page through the composite's layer input, so a clip
136//! open around it masks it like any fill.
137//! 3. **The hole punch** ([`FramePass::Punch`]), destination-out, when the
138//! frame recorded a `ClearRect` — see [`crate::compile::clear`] for the
139//! whole contract this pass implements. It is issued at the punch's own painter-order position,
140//! not at the end of the frame: a surface round is *cut* where the punch
141//! was recorded, the punch pass goes into that cut, and the round's
142//! remaining ops resume in a pass of their own after it. Everything drawn
143//! over the slot is therefore recorded after the erase and survives it,
144//! whether or not it wrote depth. A punch past every op of the frame — the
145//! ordinary case, a `ClearRect` recorded last — cuts nothing and lands after
146//! the last round exactly as it always did.
147//!
148//! With depth unavailable — no attachment, or `FRUST_ENGINE_NO_DEPTH` set —
149//! the opening draws no opaque strips (and, unmerged, the opaque pass
150//! disappears) and every instance travels through the surface rounds, blended
151//! in painter order. That is a correctness requirement rather than a
152//! fallback detail: routing the opaque spans into a separate, earlier pass is
153//! only sound because the depth buffer re-establishes their ordering against the
154//! blended ones.
155//!
156//! # Compositing a layer
157//!
158//! A finished page reaches its parent as ONE instanced quad through the strip
159//! program every draw goes through (its [`PaintClass::LayerComposite`]
160//! specialisation — see *Paint-class runs*), flagged as a whole rectangle and
161//! naming the layer colour source: the fragment stage then reads the page
162//! bound as `layer_input_texture` at the quad's own texel and scales it by the
163//! opacity packed into the instance's low byte. The page is bound through a
164//! bind group of its own rather than the frame's shared one, because group 0
165//! carries both the pass's viewport uniform and that layer input, and a page
166//! round's viewport is its own.
167//!
168//! A retained page ([`crate::cache::retained`]) is composited by exactly the
169//! same quad, bound the same way; only the texture differs — the store's own,
170//! named by the retained bracket rather than by a page group. A round
171//! rendering a body into one is a page round in every respect above (cleared on
172//! its first round, loaded on a continued one, through its own viewport
173//! uniform, never taking the surface opening) and is listed as a
174//! [`FramePass::Page`]. A `DrawRetained`'s composite carries the painter-order
175//! index the command itself was recorded at.
176//!
177//! A composite carries the deepest painter's-order index of everything inside
178//! the layer it composites, nested layers included. That is what keeps a
179//! translucent layer correctly ordered against the root round's own draws
180//! without giving the scheduler a depth model: every draw recorded before the
181//! layer sits behind that index and every draw recorded after it sits in front.
182//!
183//! # Paint-class runs
184//!
185//! Every strip pass is drawn in **runs**, each through the pipeline of one
186//! [`PaintClass`] — the strip program specialised to the paints the run holds
187//! (see [`crate::gpu::pipelines`]'s *The paint-class axis*). The runs are cut
188//! when a segment is planned (`Scratch::push_run`, by
189//! [`merge_class_runs`]): contiguous, in painter order, covering the segment
190//! exactly, so a pass draws the same instances in the same order it always did
191//! and only the program shading each run changes. A run shorter than the
192//! renderer's threshold ([`EngineRenderer::run_split_min`],
193//! [`crate::gpu::strips::RUN_SPLIT_MIN`] by default) is merged and drawn with
194//! [`PaintClass::Uber`] rather than paying a pipeline switch of its own; a
195//! composite always takes [`PaintClass::LayerComposite`], the opaque pass its
196//! own runs, and a punch [`PaintClass::SolidCoverage`]. A pass sets a pipeline
197//! only where the class changes between runs.
198//!
199//! `FRUST_ENGINE_UBER_PROGRAM` ([`EngineRenderer::set_uber_program`]) turns all
200//! of it off: no segment is split, every pass draws through its state's one
201//! Uber pipeline, and no class program is warmed or built.
202//!
203//! `FRUST_ENGINE_IMAGE_SAMPLER` ([`EngineRenderer::with_image_sampler`]) picks
204//! the program *family* the classes are compiled from, not a class: in the
205//! sampler family — the default — every program that reads atlas images (the
206//! image classes and Uber) filters a bilinear image through one hardware
207//! sampler, which every strip bind group then carries beside the atlas array;
208//! its kill switch (`0`) compiles the manual four-tap family instead. The
209//! runs, and the pipelines a frame resolves, are the same in both families.
210//!
211//! # Paint resolution
212//!
213//! A solid colour travels inside the strip instance itself. Anything else the
214//! compiler encoded — a gradient, an image, a blurred rounded rectangle — is
215//! resolved once per frame before a single instance is built, in three steps
216//! that have to happen in this order:
217//!
218//! 1. residency is settled for the whole frame: the frame's LUT requests are
219//! serviced through the [`GradientCache`] so every gradient's colour ramp
220//! has an offset into the packed LUT buffer, and every image paint's atlas
221//! rectangle is looked up (or learned, the first time it is drawn) in the
222//! renderer's own image registry — see [`FrameResources::resolve_paints`];
223//! 2. each encoded paint is lowered into the [`GpuEncodedPaint`] record the
224//! fragment shader samples, carrying that residency; and
225//! 3. the records are serialized back to back, which fixes the texel each one
226//! starts at — the index a strip instance names its paint by.
227//!
228//! Ramp offsets are only valid within the frame that took them: the cache
229//! compacts and rewrites them in [`EngineRenderer::end_frame`], which is why
230//! residency is decided here rather than at compile time. An image's atlas
231//! rectangle, by contrast, is stable for as long as the image stays resident
232//! (the compiler's [`crate::cache::images::ImageResidency`] does not move a
233//! live image), so the renderer's own registry only ever forgets an entry
234//! when the frame that compiled it reports the entry's region evicted.
235//!
236//! A paint that still cannot be resolved — an image the atlas has no room
237//! for, a gradient whose ramp could not be baked, an external texture (nothing
238//! binds one yet) — leaves its draw skipped rather than stamped in a wrong
239//! colour: the same "a frame draws less, never wrong" rule the compiler
240//! follows for the commands it does not lower.
241//!
242//! An image the atlas holds a *minified* copy of is the one paint whose lowered
243//! record needs a correction here. The compiler composed its natural-to-device
244//! transform against the source's declared extent, before residency was
245//! consulted and so before the fit was known; the shader samples the resident
246//! rectangle. [`ResidentImage::minify_scale`] is the ratio between the two, and
247//! folding it into the lowered record's transform is what keeps a downsampled
248//! image landing on the destination rectangle the display list asked for.
249
250use core::ops::Range;
251use std::collections::{HashMap, HashSet};
252use std::sync::{Arc, Once};
253// Durations and the compile's own phase record are read by the `perf-trace`
254// encode window and by the tests that pin its arithmetic, and by nothing else.
255#[cfg(any(test, feature = "perf-trace"))]
256use std::time::Duration;
257
258use frust_gpu::{PipelineCache, PooledTexture, SceneTextureId, ShaderLibrary, TierCaps};
259use frust_scene::{Scene, SceneBuilder};
260use glifo::{AtlasCommand, AtlasCommandRecorder, AtlasPaint};
261use kurbo::Affine;
262use peniko::{Brush, Color};
263use vello_common::encode::{EncodedImage, EncodedPaint};
264use vello_common::fearless_simd::Level;
265use vello_common::paint::{ImageId, ImageSource, Paint};
266use vello_common::strip::Strip;
267
268use crate::cache::images::{ATLAS_PADDING, AtlasRegion};
269use crate::cache::retained::{
270 RetainedBudget, RetainedDecision, RetainedEntry, RetainedPages, RetainedPlan,
271};
272use crate::cache::{
273 AtlasBudget, BYTES_PER_TEXEL, CachedRamp, GradientCache, GradientTextureLayout, ResidentImage,
274};
275#[cfg(any(test, feature = "perf-trace"))]
276use crate::compile::CompileSpans;
277use crate::compile::paint::resolve_lut_request;
278use crate::compile::{
279 ClearPunch, CompiledFrame, GlyphQuad, PhaseClock, RetainedKind, SceneCompiler,
280};
281use crate::config;
282use crate::diag::{EngineSpan, FrameTimestamps};
283use crate::error::EngineError;
284use crate::filters::blur::{FilterInstanceData, GpuFilterData, GpuGaussianBlur};
285use crate::filters::drop_shadow::GpuDropShadow;
286use crate::filters::{Backdrop, FilterStep, ServedFilter, served_filter};
287use crate::gpu::atlas::{
288 AtlasPageBuffers, AtlasRenderReport, AtlasRenderer, lower_encoded_image, push_solid_strips,
289};
290use crate::gpu::bindings::{
291 ATLAS_SAMPLER_BINDING, ExternalRuns, ExternalTextures, lower_encoded_external,
292};
293use crate::gpu::config::resource_widths_agree;
294use crate::gpu::depth::DepthAttachment;
295use crate::gpu::paint_texture::lower_encoded_paint;
296use crate::gpu::pipelines::{
297 EnginePipeline, EngineShaders, INTERMEDIATE_FORMAT, atlas_strip_desc, warm_up_descs_for,
298};
299use crate::gpu::strips::{
300 ClassRun, ClassSet, PaintClass, PaintType, merge_class_runs, pack_paint_descriptor,
301};
302use crate::gpu::targets::{
303 IntermediateTargets, IntermediateTexture, filter_data_texture_descriptor,
304 filter_data_texture_height, filter_sampler,
305};
306use crate::gpu::{self, AtlasArray, GpuConfig, GpuEncodedPaint, GpuStrip, StripDraw};
307use crate::schedule::pages::{PageConfig, PageSize};
308use crate::schedule::{
309 Composite, MAX_LIVE_PAGES, PageParity, PageTarget, RetainedLayers, RetainedRole,
310 RetainedTarget, Round, RoundOp, Schedule,
311};
312use crate::{EngineTarget, OutputAlpha};
313use vello_common::geometry::RectU16;
314use vello_common::geometry::SizeU16;
315use vello_common::record::RecordedLayerKind;
316
317/// The packed paint descriptor of an inline premultiplied solid colour.
318///
319/// A solid paint indexes no encoded-paint record, so its descriptor carries
320/// only the colour source and paint type — both of which are zero, which is
321/// why this is named rather than written as a bare `0` at its call site: the
322/// zero is a coincidence of the layout, not an absence of information.
323const SOLID_PAINT: u32 = pack_paint_descriptor(PaintType::Solid, 0);
324
325/// The smallest resource-texture dimension the engine can address.
326///
327/// A resource texture's row stride is its width times its texel size, and
328/// `wgpu::COPY_BYTES_PER_ROW_ALIGNMENT` is 256; the narrowest of the three
329/// resources is the gradient LUT at 4 bytes per texel, so 64 texels is the
330/// point below which an upload's `bytes_per_row` stops being legal.
331const MIN_RESOURCE_TEXTURE_DIM: u32 = 64;
332
333/// The smallest strip instance buffer the engine allocates, in instances.
334///
335/// A first frame with a handful of strips should not force a second allocation
336/// on the second frame.
337const MIN_INSTANCE_CAPACITY: u64 = 4096;
338
339/// The colour source a composite instance names its finished page by:
340/// `COLOR_SOURCE_LAYER` in bits 29-30 of the packed paint descriptor.
341///
342/// Deliberately not built through
343/// [`pack_paint_descriptor`](crate::gpu::strips::pack_paint_descriptor): that
344/// helper packs a paint type and a paint-record index into the low bits, and a
345/// composite spends the same bits on a constant opacity instead. Two readings
346/// of one word, so each is written where its own reading is obvious.
347const LAYER_PAINT_SOURCE: u32 = 1 << 29;
348
349/// The premultiplied source colour a hole-punch instance erases with.
350///
351/// Opaque white. Destination-out weights the erase by the source's own *alpha*
352/// and multiplies its colour by zero, so the colour channels never reach the
353/// target and only full alpha matters — it is what makes a fully covered pixel
354/// read exactly `(0, 0, 0, 0)`.
355const PUNCH_SOURCE: u32 = u32::MAX;
356
357/// The label every intermediate page is acquired from the pool under.
358const PAGE_LABEL: &str = "frust-engine layer page";
359
360/// The label every filter round's pass is recorded under.
361const FILTER_LABEL: &str = "frust-engine filter pass";
362
363/// Vertices one filter pass's quad is built from, the same four-vertex
364/// triangle strip every engine program expands an instance into.
365const FILTER_QUAD_VERTICES: u32 = 4;
366
367/// The smallest filter instance buffer the engine allocates, in instances.
368///
369/// A σ-32 blur is ten passes, so this is roughly "one deep blur costs no second
370/// allocation"; the buffer is 32 bytes an instance and grows from here.
371const MIN_FILTER_INSTANCE_CAPACITY: u64 = 16;
372
373static INDEXED_PAINT_WARNING: Once = Once::new();
374
375/// Raised the first time a frame's backdrop blurs are drawn as nothing because
376/// its target can be neither copied nor sampled (see [`RegionCopyMode`]), on
377/// the same once-per-process terms as the compiler's skip warnings.
378static BACKDROP_UNREADABLE_WARNING: Once = Once::new();
379
380/// Vertices the region-copy program's one quad draws (`copy.wgsl`'s
381/// `quad_corner`, a triangle strip).
382const COPY_QUAD_VERTICES: u32 = 4;
383
384/// The bytes of one region-copy instance: four `u16` pairs
385/// ([`crate::gpu::pipelines::copy_vertex_layout`]).
386const COPY_INSTANCE_BYTES: u64 = 16;
387
388/// Raised the first time an atlas region is declined, so a renderer whose
389/// budget and array have gone out of agreement says so once rather than every
390/// frame.
391static ATLAS_REFUSAL_WARNING: Once = Once::new();
392
393/// The CPU phases one [`EngineRenderer::encode_traced`] call splits into.
394///
395/// The GPU half of a frame is [`EngineSpan`]'s, measured by the timestamp ring
396/// at pass boundaries; this is the CPU half, measured by lapping a
397/// [`PhaseClock`] between the encode's own steps. The two answer different
398/// questions and neither substitutes for the other — a text-heavy scene can
399/// cost milliseconds here while its passes cost a fraction of one there.
400///
401/// The phases partition the call in the order they run, so [`Self::total`] is
402/// the whole encode. [`counts`](Self::counts) trails them and is not one of
403/// them.
404///
405/// - [`compile`](Self::compile) — [`SceneCompiler::compile`] end to end,
406/// itself split six ways by [`CompileSpans`].
407/// - [`schedule`](Self::schedule) — building the frame's pass plan and
408/// settling its depth attachment.
409/// - [`paints`](Self::paints) — resolving every draw's paint to a texel of the
410/// encoded-paint texture.
411/// - [`instances`](Self::instances) — building the per-round GPU instance
412/// arrays from the recording.
413/// - [`resize`](Self::resize) — the fallible capacity checks, the image
414/// atlas's own growth, the glyph-rectangle clears an earlier frame's
415/// eviction owes, and the three resource-texture resizes.
416/// - [`upload`](Self::upload) — the frame's queue writes: alpha coverage,
417/// encoded paints, gradient ramps, image texels and the instance arrays.
418/// - [`replay`](Self::replay) — the render-to-atlas pass that draws newly
419/// cached glyphs into their array layers (zero on a steady frame, which
420/// caches none).
421/// - [`pipelines`](Self::pipelines) — taking (or building) this frame's
422/// pipelines and bind groups, and preparing the filter blocks.
423/// - [`record`](Self::record) — recording the frame's passes into the caller's
424/// encoder.
425///
426/// Compiled only under `perf-trace`, with everything that reads it: the whole
427/// `frust-perf enc` route is a `#[cfg]` island rather than a runtime branch the
428/// optimizer is trusted to fold, so a release-lean binary carries none of its
429/// field names, format strings or window (see [`EncodeTrace`]).
430#[cfg(feature = "perf-trace")]
431#[derive(Debug, Clone, Copy, Default)]
432struct EncodeSpans {
433 /// Scene compilation, split further by its own six phases.
434 compile: CompileSpans,
435 /// Whole-call cost of [`SceneCompiler::compile`], call overhead included.
436 ///
437 /// Deliberately measured from outside rather than summed from
438 /// [`Self::compile`]: the difference between the two is the part of the
439 /// call the six inner phases do not cover, and a breakdown that could only
440 /// report its own sum could never show that gap.
441 compile_total: Duration,
442 /// Pass planning and depth settlement.
443 schedule: Duration,
444 /// Paint resolution.
445 paints: Duration,
446 /// Instance-array building.
447 instances: Duration,
448 /// Capacity checks, atlas growth, glyph-rectangle clears, resource resizes.
449 resize: Duration,
450 /// The frame's queue writes.
451 upload: Duration,
452 /// The render-to-atlas pass for newly cached glyphs.
453 replay: Duration,
454 /// Pipeline/bind-group acquisition and filter preparation.
455 pipelines: Duration,
456 /// Recording the frame's passes into the caller's encoder.
457 record: Duration,
458 /// This frame's own draw, strip, alpha and glyph counts.
459 ///
460 /// Last because it is not a phase: it is what the phases above were spent
461 /// on. Carried here because a phase's cost is only readable against the
462 /// work it did — "the walk cost two milliseconds" says nothing on its own,
463 /// while "two milliseconds over three thousand strips and three hundred
464 /// glyphs" says where a lever would have to bite.
465 counts: EncodeCounts,
466}
467
468/// The columns one `frust-perf enc` line reports, in row order.
469///
470/// The first seven are [`CompileSpans`]' own — six phases plus the `glyphs`
471/// subset of the walk — and `compile` after them is the whole compile they sit
472/// inside (so the six sum to at most it, never past it). The next eight are
473/// [`EncodeSpans`]' remaining phases, and `total` closes the line. Pinned as
474/// one list because the line's field order is what a capture is graded by —
475/// the same contract the `frust-perf img` line keeps.
476#[cfg(feature = "perf-trace")]
477const ENCODE_TRACE_COLUMNS: [&str; 17] = [
478 "validate",
479 "prepare",
480 "classify",
481 "admit",
482 "walk",
483 "glyphs",
484 "finish",
485 "compile",
486 "schedule",
487 "paints",
488 "instances",
489 "resize",
490 "upload",
491 "replay",
492 "pipelines",
493 "record",
494 "total",
495];
496
497/// The unitless per-frame counts the line reports after its phases, in row
498/// order — what the phases above were spent on.
499///
500/// The field order is a capture contract: append only. The first five are
501/// per-frame medians over the window. The next five are **sums over the
502/// window** (frames' worth of events, not a typical frame):
503///
504/// - `replays` — frames in which the render-to-atlas replay pass ran.
505/// - `dirty_pages` — atlas pages those replays rendered.
506/// - `atlas_refused` — pages, clears and uploads the replays refused.
507/// - `glyph_misses` — glyphs that took the cache-miss path.
508/// - `glyph_evicted` — atlas rectangles the eviction pass freed.
509///
510/// The last, `run_keys`, is a per-frame median again: the distinct run
511/// identities (font, size, hinting) the policy classified in a frame. The
512/// replay figures read zero on a frame whose replay did not run.
513#[cfg(feature = "perf-trace")]
514const ENCODE_TRACE_COUNTS: [&str; 11] = [
515 "draws",
516 "strips",
517 "alphas",
518 "glyph_draws",
519 "atlas_glyphs",
520 "replays",
521 "dirty_pages",
522 "atlas_refused",
523 "glyph_misses",
524 "glyph_evicted",
525 "run_keys",
526];
527
528/// Index into [`ENCODE_TRACE_COUNTS`] of the first count reported as a sum over
529/// the window rather than its median, up to and excluding
530/// [`ENCODE_TRACE_MEDIAN_AGAIN`]: `replays`, `dirty_pages`, `atlas_refused`,
531/// `glyph_misses` and `glyph_evicted` are events, mostly zero on a frame, so a
532/// median would hide exactly the frames the line exists to count.
533#[cfg(feature = "perf-trace")]
534const ENCODE_TRACE_FIRST_SUMMED: usize = 5;
535
536/// Index into [`ENCODE_TRACE_COUNTS`] of the first count after the summed run
537/// (`run_keys`, a per-frame level, reported as its median).
538#[cfg(feature = "perf-trace")]
539const ENCODE_TRACE_MEDIAN_AGAIN: usize = 10;
540
541/// Width of one window row: every phase column followed by every count.
542#[cfg(feature = "perf-trace")]
543const ENCODE_TRACE_ROW: usize = ENCODE_TRACE_COLUMNS.len() + ENCODE_TRACE_COUNTS.len();
544
545/// What one frame drew, carried beside its phases.
546///
547/// Counts rather than durations, and so reported without a unit suffix.
548/// `perf-trace`-only, with [`EncodeSpans`], which is all that carries it.
549#[cfg(feature = "perf-trace")]
550#[derive(Debug, Clone, Copy, Default)]
551struct EncodeCounts {
552 /// Recorded draws.
553 draws: u32,
554 /// Strips generated.
555 strips: u32,
556 /// Bytes of alpha coverage those strips index — the frame's largest
557 /// single queue write.
558 alphas: u32,
559 /// Draws that painted one glyph outline.
560 glyph_draws: u32,
561 /// How many of those sampled the glyph atlas rather than rasterizing.
562 atlas_glyphs: u32,
563 /// 1 when the render-to-atlas replay ran this frame, else 0.
564 replays: u32,
565 /// Atlas pages the replay rendered this frame.
566 dirty_pages: u32,
567 /// Atlas pages, clears and uploads the replay refused this frame.
568 atlas_refused: u32,
569 /// Glyphs that took the cache-miss path while compiling this frame.
570 glyph_misses: u32,
571 /// Atlas rectangles this frame's eviction freed.
572 glyph_evicted: u32,
573 /// Distinct run identities classified this frame.
574 run_keys: u32,
575}
576
577#[cfg(feature = "perf-trace")]
578impl EncodeCounts {
579 /// What `frame` drew, with the atlas figures the frame's replay and policy
580 /// reported.
581 fn of(
582 frame: &CompiledFrame,
583 replayed: bool,
584 report: &AtlasRenderReport,
585 policy: &crate::text::atlas_policy::AtlasTrace,
586 ) -> Self {
587 fn count(len: usize) -> u32 {
588 u32::try_from(len).unwrap_or(u32::MAX)
589 }
590 Self {
591 draws: count(frame.draws().len()),
592 strips: count(frame.strip_buf().len()),
593 alphas: count(frame.alphas().len()),
594 glyph_draws: frame.glyph_draws,
595 atlas_glyphs: frame.atlas_glyph_draws,
596 replays: u32::from(replayed),
597 dirty_pages: report.pages,
598 atlas_refused: report.refused,
599 glyph_misses: policy.glyph_misses,
600 glyph_evicted: policy.glyph_evicted,
601 run_keys: policy.run_keys(),
602 }
603 }
604}
605
606/// Frames one `frust-perf enc` line summarises — one line a second at 60 Hz.
607///
608/// A window rather than a line per frame: the encode is the very thing being
609/// measured, so formatting and logging inside it once per frame would charge
610/// the measurement to its own subject. One line per sixty frames keeps that
611/// charge under a microsecond a frame while still resolving a scenario's
612/// phases (a thirty-second run reports thirty times), and it matches the
613/// cadence `frust-shell-common`'s own rate-limited frame summary already
614/// emits at.
615#[cfg(feature = "perf-trace")]
616const ENCODE_TRACE_WINDOW: usize = 60;
617
618#[cfg(feature = "perf-trace")]
619impl EncodeSpans {
620 /// The whole encode.
621 ///
622 /// Saturating throughout: a diagnostic sum must not take the frame with it
623 /// on overflow (E17).
624 fn total(&self) -> Duration {
625 // `compile.glyphs` is deliberately absent: it is a subset of the walk
626 // inside `compile_total`, and adding it would count text twice.
627 [
628 self.schedule,
629 self.paints,
630 self.instances,
631 self.resize,
632 self.upload,
633 self.replay,
634 self.pipelines,
635 self.record,
636 ]
637 .iter()
638 .fold(self.compile_total, |acc, span| acc.saturating_add(*span))
639 }
640
641 /// This frame's row, in [`ENCODE_TRACE_COLUMNS`] order, in nanoseconds.
642 ///
643 /// Nanoseconds in a `u32` rather than a `Duration` per column: a window of
644 /// sixty rows is then three kilobytes of plain integers to sort, and no
645 /// single phase of one frame reaches the four-second ceiling that would
646 /// saturate one.
647 fn row(&self) -> [u32; ENCODE_TRACE_ROW] {
648 fn ns(span: Duration) -> u32 {
649 u32::try_from(span.as_nanos()).unwrap_or(u32::MAX)
650 }
651 [
652 ns(self.compile.validate),
653 ns(self.compile.prepare),
654 ns(self.compile.classify),
655 ns(self.compile.admit),
656 ns(self.compile.walk),
657 ns(self.compile.glyphs),
658 ns(self.compile.finish),
659 ns(self.compile_total),
660 ns(self.schedule),
661 ns(self.paints),
662 ns(self.instances),
663 ns(self.resize),
664 ns(self.upload),
665 ns(self.replay),
666 ns(self.pipelines),
667 ns(self.record),
668 ns(self.total()),
669 self.counts.draws,
670 self.counts.strips,
671 self.counts.alphas,
672 self.counts.glyph_draws,
673 self.counts.atlas_glyphs,
674 self.counts.replays,
675 self.counts.dirty_pages,
676 self.counts.atlas_refused,
677 self.counts.glyph_misses,
678 self.counts.glyph_evicted,
679 self.counts.run_keys,
680 ]
681 }
682}
683
684/// The rolling window of per-frame [`EncodeSpans`] one `frust-perf enc` line
685/// is computed over.
686///
687/// Absent altogether from a build without `perf-trace`, rather than inert in
688/// one. The type, its window, its column names and the `frust-perf enc` literal
689/// are all inside a `#[cfg]` island — including the renderer's own field — so
690/// "no frame pays a push, a sort or a format" is what the compiler emitted and
691/// not what the optimizer was expected to prove about a constant branch. That
692/// distinction is the one the release-lean gate measures: it greps a shipping
693/// binary for `frust-perf` and expects to find nothing.
694#[cfg(feature = "perf-trace")]
695#[derive(Debug, Default)]
696struct EncodeTrace {
697 /// One row per frame in the window, in [`ENCODE_TRACE_COLUMNS`] order.
698 window: Vec<[u32; ENCODE_TRACE_ROW]>,
699 /// The column being ranked, kept across windows so a steady stream of
700 /// them allocates once.
701 ranked: Vec<u32>,
702 /// Frames recorded since this renderer was built — the line's own `n`,
703 /// which is the engine's count and deliberately not the host's raw-line
704 /// `n` (a different emitter counting different frames).
705 frames: u64,
706}
707
708#[cfg(feature = "perf-trace")]
709impl EncodeTrace {
710 /// Adds one frame's phases to the window, emitting the window's line when
711 /// it fills.
712 ///
713 /// Called at the very end of the encode, so the formatting a full window
714 /// costs falls outside every phase the line reports — the numbers describe
715 /// the encode, not the encode plus its own accounting.
716 fn record(&mut self, spans: &EncodeSpans) {
717 // Saturating like every other counter on this path: a renderer that
718 // outlived `u64::MAX` frames would report a wrapped `n`, and a wrong
719 // number in a capture is worse than a stuck one.
720 self.frames = self.frames.saturating_add(1);
721 self.window.push(spans.row());
722 if self.window.len() < ENCODE_TRACE_WINDOW {
723 return;
724 }
725 let line = self.line();
726 self.window.clear();
727 log::info!("{line}");
728 }
729
730 /// The `frust-perf enc` line the current window reports.
731 ///
732 /// The counts `replays` through `glyph_evicted` are window sums; every
733 /// other column (including `run_keys`) is its own median over the window.
734 /// The trailing field is the whole encode's 95th percentile, so a column
735 /// answers "what does this phase usually cost" while the tail answers "how
736 /// bad is a bad frame". Medians are taken per column and so do not sum to
737 /// the `total_us`
738 /// median exactly — they are close on a steady scene, and the gap is
739 /// itself the signal that the phases are not moving together.
740 fn line(&mut self) -> String {
741 // Sized for the longest line the loops below write: a
742 // `<name>_us=<7 digits>.<1>` field per phase, a `<name>=<10 digits>`
743 // field per count, the two `n`/`w` fields and the p95 tail, with room
744 // to spare rather than a byte-exact fit.
745 let mut line = String::with_capacity(640);
746 line.push_str("frust-perf enc n=");
747 line.push_str(&self.frames.to_string());
748 line.push_str(" w=");
749 line.push_str(&self.window.len().to_string());
750 for (column, name) in ENCODE_TRACE_COLUMNS.iter().enumerate() {
751 let p50 = self.percentile(column, 50);
752 line.push(' ');
753 line.push_str(name);
754 line.push_str("_us=");
755 line.push_str(&format_us(p50));
756 }
757 let tail = self.percentile(ENCODE_TRACE_COLUMNS.len() - 1, 95);
758 line.push_str(" total_p95_us=");
759 line.push_str(&format_us(tail));
760 for (offset, name) in ENCODE_TRACE_COUNTS.iter().enumerate() {
761 let column = ENCODE_TRACE_COLUMNS.len() + offset;
762 let value = if (ENCODE_TRACE_FIRST_SUMMED..ENCODE_TRACE_MEDIAN_AGAIN).contains(&offset)
763 {
764 self.window
765 .iter()
766 .filter_map(|row| row.get(column))
767 .fold(0_u64, |acc, v| acc.saturating_add(u64::from(*v)))
768 } else {
769 u64::from(self.percentile(column, 50))
770 };
771 line.push(' ');
772 line.push_str(name);
773 line.push('=');
774 line.push_str(&value.to_string());
775 }
776 line
777 }
778
779 /// The `pct`-th percentile of `column` over the window, in nanoseconds.
780 ///
781 /// Nearest-rank on the sorted column, which needs no interpolation and so
782 /// reports a value the window really contains — the same choice
783 /// `frust-shell-common`'s frame summary makes, and the one that keeps a
784 /// sixty-sample window honest.
785 fn percentile(&mut self, column: usize, pct: usize) -> u32 {
786 self.ranked.clear();
787 self.ranked
788 .extend(self.window.iter().filter_map(|row| row.get(column)));
789 self.ranked.sort_unstable();
790 let len = self.ranked.len();
791 if len == 0 {
792 return 0;
793 }
794 let rank = (len * pct).div_ceil(100).clamp(1, len);
795 self.ranked.get(rank - 1).copied().unwrap_or(0)
796 }
797}
798
799/// Nanoseconds as microseconds with one decimal — the unit every
800/// `frust-perf enc` field is reported in.
801///
802/// Fixed-point rather than a float format: a phase can be tens of nanoseconds
803/// (an empty punch pass) or milliseconds (a ten-thousand-row table's walk), and
804/// one decimal microsecond reads the same either way without a float's
805/// locale-dependent formatting reaching a capture the harness greps.
806#[cfg(feature = "perf-trace")]
807fn format_us(nanos: u32) -> String {
808 let tenths = u64::from(nanos).div_ceil(100);
809 format!("{}.{}", tenths / 10, tenths % 10)
810}
811
812/// One surface's 2D render engine: a scene in, recorded passes out.
813///
814/// Create one per surface and keep it across frames — the retained scene
815/// compiler, gradient cache, pipeline cache, resource textures and intermediate
816/// pool are the reason a steady-state frame allocates nothing.
817#[derive(Debug)]
818pub struct EngineRenderer {
819 caps: TierCaps,
820 format: wgpu::TextureFormat,
821 shaders: EngineShaders,
822 pipelines: PipelineCache,
823 compiler: SceneCompiler,
824 gradients: GradientCache,
825 depth: DepthAttachment,
826 targets: IntermediateTargets,
827 /// The bounds an intermediate page is sized between — the policy half of
828 /// page sizing, kept beside the pool the extents are requested from.
829 pages: PageConfig,
830 /// The caller-owned textures a `Command::SceneTexture` resolves against
831 /// (see [`crate::gpu::bindings`]). Its extent half lives on the compiler,
832 /// written by the same two calls that write this.
833 textures: ExternalTextures,
834 resources: FrameResources,
835 scratch: Scratch,
836 /// How each pass's instances are split into paint-class runs — the kill
837 /// switch and the run threshold, settled at construction from
838 /// [`crate::config`] and overridable per renderer.
839 routing: ClassRouting,
840 /// Whether the surface opening is one pass — settled at construction from
841 /// `FRUST_ENGINE_NO_PASS_MERGE` and overridable per renderer (see
842 /// [`Self::set_pass_merge`]).
843 pass_merge: bool,
844 /// The passes the last encoded frame recorded into the caller's encoder,
845 /// in order; refilled every frame, so its allocation is reused.
846 frame_passes: Vec<FramePass>,
847 /// The strip pipelines resolved so far, by state and class, so a frame
848 /// takes each one it draws with by a two-enum lookup rather than by
849 /// rebuilding and re-keying its description through the cache.
850 resolved: ResolvedPipelines,
851 /// The render-to-atlas pass, created by the first frame that caches a
852 /// glyph.
853 ///
854 /// Lazy because it owns a coverage texture, an instance buffer and five
855 /// stand-in bindings of its own, and a renderer that never draws text —
856 /// or one running with `FRUST_ENGINE_NO_ATLAS` — should pay for none of
857 /// them.
858 atlas_glyphs: Option<AtlasRenderer>,
859 /// The compiler the atlas replay lowers a page's recorded commands
860 /// through, sized to the atlas page rather than to the surface.
861 ///
862 /// A second compiler rather than this renderer's own: the frame's compiler
863 /// is mid-frame (its glyph entry map is exactly what the replay is
864 /// draining) and its viewport is the surface's, while a page's commands are
865 /// in page space. Created on the first replay and kept, so a steady stream
866 /// of first-seen glyphs allocates a strip generator once.
867 atlas_lowering: Option<SceneCompiler>,
868 /// What the last frame's replay serviced, for
869 /// [`Self::atlas_render_report`].
870 atlas_report: AtlasRenderReport,
871 /// The filter-data texture, sampler and instance buffer a filter round is
872 /// executed with.
873 ///
874 /// Lazy for the same reason [`Self::atlas_glyphs`] is: a renderer that
875 /// never blurs should own neither a sampler nor a resource texture it will
876 /// not read. Created by the first frame that schedules a filter round.
877 filters: Option<FilterResources>,
878 /// The instance buffer a frame's fragment region copies are drawn from
879 /// ([`RegionCopyMode::Fragment`]). Lazy for [`Self::filters`]' reason: a
880 /// renderer whose frames never copy a region owns none.
881 region_copies: Option<RegionCopyResources>,
882 /// The retained pages `PushRetained` bodies render into and
883 /// `DrawRetained`s composite from, kept across frames (see
884 /// [`crate::cache::retained`]).
885 retained: RetainedPages,
886 /// The roles the store gave this frame's retained layers, handed to the
887 /// scheduler — retained across frames for its allocation alone.
888 retained_layers: RetainedLayers,
889 /// The scene [`Self::record_range`] projects a range into, kept for its
890 /// allocation alone; empty until the first range is recorded.
891 range_scene: Scene,
892 /// The rolling CPU-phase window `frust-perf enc` lines are emitted from.
893 ///
894 /// Absent entirely from a build without `perf-trace`, along with the type
895 /// itself — see [`EncodeTrace`].
896 #[cfg(feature = "perf-trace")]
897 encode_trace: EncodeTrace,
898}
899
900impl EngineRenderer {
901 /// Builds a renderer for `format` targets on `caps`' adapter, warming
902 /// every engine pipeline in the background.
903 ///
904 /// `pipeline_cache` is the host's persisted driver cache when it has one —
905 /// `None` on every backend but Vulkan.
906 ///
907 /// # Errors
908 ///
909 /// [`EngineError::AtlasError`] when the adapter's `resource_texture_dim`
910 /// is not a power of two of at least [`MIN_RESOURCE_TEXTURE_DIM`]. The
911 /// strip shader reconstructs a resource texture's width as `1 << bits` and
912 /// an upload's row stride must satisfy wgpu's copy alignment, so neither
913 /// the shader's addressing nor the uploads would be valid otherwise.
914 pub fn new(
915 device: &wgpu::Device,
916 caps: &TierCaps,
917 format: wgpu::TextureFormat,
918 pipeline_cache: Option<&wgpu::PipelineCache>,
919 ) -> Result<Self, EngineError> {
920 Self::with_image_sampler(
921 device,
922 caps,
923 format,
924 pipeline_cache,
925 config::image_sampler_enabled(),
926 )
927 }
928
929 /// [`Self::new`] with the strip program family stated rather than read
930 /// from `FRUST_ENGINE_IMAGE_SAMPLER` — the programmatic counterpart of
931 /// that switch, which only sets the family [`Self::new`] picks.
932 ///
933 /// With `image_sampler` set, every strip program that reads atlas images
934 /// filters a bilinear one through a hardware sampler instead of four
935 /// `textureLoad` taps (see [`crate::gpu::shader_src`]'s *The two program
936 /// families*), and every strip bind group carries the one linear,
937 /// clamp-to-edge sampler this renderer creates for it. A family is a
938 /// construction-time choice rather than a per-frame setter because it
939 /// decides which programs are compiled and warmed: a caller comparing the
940 /// two builds one renderer of each.
941 ///
942 /// # Errors
943 ///
944 /// As [`Self::new`].
945 pub fn with_image_sampler(
946 device: &wgpu::Device,
947 caps: &TierCaps,
948 format: wgpu::TextureFormat,
949 pipeline_cache: Option<&wgpu::PipelineCache>,
950 image_sampler: bool,
951 ) -> Result<Self, EngineError> {
952 let dim = caps.resource_texture_dim;
953 if !dim.is_power_of_two() || dim < MIN_RESOURCE_TEXTURE_DIM {
954 return Err(EngineError::AtlasError);
955 }
956
957 let routing = ClassRouting::from_config();
958 let mut library = ShaderLibrary::new();
959 let shaders = EngineShaders::register_family(&mut library, device, image_sampler);
960 let mut pipelines = PipelineCache::new(Arc::new(library), pipeline_cache.cloned());
961 pipelines.warm_up(device, &warm_up_descs_for(&shaders, format, routing.uber));
962
963 let level = Level::try_detect().unwrap_or(Level::baseline());
964 let gradients = GradientCache::for_texture(GradientTextureLayout::square(dim), level);
965
966 Ok(Self {
967 caps: caps.clone(),
968 format,
969 shaders,
970 pipelines,
971 // The viewport is re-asserted on every compile, so the extent is
972 // only an initial allocation hint. `caps` is not: it is what fixes
973 // the image atlas budget for this renderer's whole life (mobile or
974 // desktop tier, the adapter's own ceilings, and any
975 // `FRUST_ENGINE_ATLAS_SIZE` override), and the adapter is known
976 // exactly here — a compiler built without it would silently keep
977 // the mobile budget on every device.
978 compiler: {
979 let mut compiler = SceneCompiler::for_caps(1, 1, caps);
980 // A renderer has a store behind its compiler, so retained
981 // brackets are recorded for it — unless the engine-side kill
982 // switch says otherwise.
983 compiler.set_retained_layers(!config::layers_disabled());
984 compiler
985 },
986 gradients,
987 depth: DepthAttachment::new(),
988 targets: IntermediateTargets::new(caps),
989 pages: PageConfig::default(),
990 textures: ExternalTextures::new(),
991 resources: FrameResources::new(device, dim, image_sampler),
992 scratch: Scratch::default(),
993 routing,
994 pass_merge: !config::pass_merge_disabled(),
995 frame_passes: Vec::new(),
996 resolved: ResolvedPipelines::default(),
997 atlas_glyphs: None,
998 atlas_lowering: None,
999 atlas_report: AtlasRenderReport::default(),
1000 filters: None,
1001 region_copies: None,
1002 retained: RetainedPages::for_caps(caps),
1003 retained_layers: RetainedLayers::new(),
1004 range_scene: Scene::new(),
1005 #[cfg(feature = "perf-trace")]
1006 encode_trace: EncodeTrace::default(),
1007 })
1008 }
1009
1010 /// The target format this renderer warmed its pipelines for.
1011 #[must_use]
1012 pub fn format(&self) -> wgpu::TextureFormat {
1013 self.format
1014 }
1015
1016 /// Whether this renderer's strip programs filter bilinear atlas images
1017 /// through the hardware sampler (see [`Self::with_image_sampler`]).
1018 #[must_use]
1019 pub fn image_sampler(&self) -> bool {
1020 self.shaders.image_sampler
1021 }
1022
1023 /// The adapter capabilities every sizing decision is made against.
1024 #[must_use]
1025 pub fn caps(&self) -> &TierCaps {
1026 &self.caps
1027 }
1028
1029 /// The pool the engine's off-screen intermediates come from.
1030 #[must_use]
1031 pub fn targets(&self) -> &IntermediateTargets {
1032 &self.targets
1033 }
1034
1035 /// The bounds this renderer sizes intermediate layer pages between.
1036 #[must_use]
1037 pub fn page_config(&self) -> PageConfig {
1038 self.pages
1039 }
1040
1041 /// The atlas geometry image residency allocates within.
1042 ///
1043 /// Derived from the adapter in [`Self::new`], so this is the tier's budget
1044 /// narrowed to what the adapter can create — not a constant.
1045 #[must_use]
1046 pub fn atlas_budget(&self) -> AtlasBudget {
1047 self.compiler.images().budget()
1048 }
1049
1050 /// Re-budget image residency, dropping every image currently resident and
1051 /// the atlas array holding them.
1052 ///
1053 /// An atlas rectangle only means anything against the geometry it was
1054 /// allocated in, so a new budget invalidates every one already handed out —
1055 /// which is why the array, the registry of where each image lives and the
1056 /// bind groups naming that array all go in the same step, and each image
1057 /// re-uploads on the next frame that draws it. A start-up or adapter-change
1058 /// operation, never a per-frame one.
1059 pub fn set_atlas_budget(&mut self, budget: AtlasBudget) {
1060 self.compiler.set_atlas_budget(budget);
1061 self.resources.reset_atlas();
1062 }
1063
1064 /// Replace image residency wholesale, on the same invalidation terms as
1065 /// [`Self::set_atlas_budget`].
1066 ///
1067 /// The programmatic counterpart to `FRUST_ENGINE_NO_ATLAS`: an
1068 /// [`ImageResidency::disabled`] residency takes both atlas classes out of
1069 /// the frame — images are skipped and every glyph is drawn as outline
1070 /// strips — without a process-global environment variable, which is what a
1071 /// caller comparing the two paths on one device needs.
1072 pub fn set_image_residency(&mut self, images: crate::cache::images::ImageResidency) {
1073 self.compiler.set_image_residency(images);
1074 self.resources.reset_atlas();
1075 }
1076
1077 /// How many atlas regions this renderer has declined to write or clear.
1078 ///
1079 /// Zero on every sound frame: residency allocates inside the budget the
1080 /// array is created at, and the array is grown to the depth the frame
1081 /// reports before its regions are written, so a refusal means those two
1082 /// went out of agreement. The count exists so that disagreement is
1083 /// measurable rather than silent — a refused write is a region the frame
1084 /// believed it had filled.
1085 #[must_use]
1086 pub fn refused_atlas_regions(&self) -> u64 {
1087 self.resources.refused_regions
1088 }
1089
1090 /// Finishes pipeline warm-up on the calling thread, returning only once
1091 /// every engine pipeline exists.
1092 ///
1093 /// Warm-up is started in the background by [`Self::new`] and normally
1094 /// needs no attention. Two callers want it forced: a host that must not
1095 /// let the *first* frame pay for a compile, and anything about to drop the
1096 /// `wgpu::Device` shortly after building a renderer — the warm-up worker
1097 /// holds its own handle on that device, and tearing it down while the
1098 /// worker is mid-compile is a driver-level hazard rather than a clean
1099 /// cancellation.
1100 ///
1101 /// Each variant is requested through the cache, which builds a queued one
1102 /// inline and waits for one the worker has already started, so nothing is
1103 /// compiled twice and nothing is left for the worker to claim afterwards.
1104 ///
1105 /// "Every" is the warm-up list: the nine Uber states, plus — unless the
1106 /// Uber program is forced — the warmed paint classes. A class variant
1107 /// outside that list is created on the first frame that draws a run of it.
1108 pub fn finish_warm_up(&mut self, device: &wgpu::Device) {
1109 for desc in warm_up_descs_for(&self.shaders, self.format, self.routing.uber) {
1110 let _ = self.pipelines.get_or_create(device, &desc);
1111 }
1112 }
1113
1114 /// Forces every strip run through the whole (Uber) strip program, or
1115 /// lets passes split into paint-class runs again — the programmatic
1116 /// counterpart of `FRUST_ENGINE_UBER_PROGRAM`, which only sets the value a
1117 /// renderer starts with.
1118 ///
1119 /// Takes effect from the next frame. With it set a frame draws exactly
1120 /// what it drew before paint classes existed — one pipeline per pass, the
1121 /// whole `strip.wgsl` program of this renderer's family — which is what a
1122 /// caller comparing the two needs without a process-global environment
1123 /// variable.
1124 pub fn set_uber_program(&mut self, uber: bool) {
1125 self.routing.uber = uber;
1126 }
1127
1128 /// Whether every strip run is drawn with the Uber program.
1129 #[must_use]
1130 pub fn uber_program(&self) -> bool {
1131 self.routing.uber
1132 }
1133
1134 /// Draws a glyph run resolved entirely out of the glyph atlas as glyph
1135 /// quads, or the general way — the programmatic counterpart of
1136 /// `FRUST_ENGINE_NO_GLYPH_QUADS`, which only sets the value a renderer
1137 /// starts with (see [`SceneCompiler::set_glyph_quads`]).
1138 ///
1139 /// Takes effect from the next frame. Both arms draw the same pixels, which
1140 /// is what a caller comparing them in one process checks.
1141 pub fn set_glyph_quads(&mut self, enabled: bool) {
1142 self.compiler.set_glyph_quads(enabled);
1143 }
1144
1145 /// Whether resident glyph runs are drawn as glyph quads.
1146 #[must_use]
1147 pub fn glyph_quads(&self) -> bool {
1148 self.compiler.glyph_quads_enabled()
1149 }
1150
1151 /// Sets the shortest paint-class run worth a pipeline switch of its own
1152 /// (see [`merge_class_runs`]); shorter runs are merged and drawn with the
1153 /// Uber program. `FRUST_ENGINE_RUN_SPLIT_MIN` only sets the value a
1154 /// renderer starts with.
1155 pub fn set_run_split_min(&mut self, min_run: u32) {
1156 self.routing.min_run = min_run;
1157 }
1158
1159 /// The run threshold this renderer splits passes at.
1160 #[must_use]
1161 pub fn run_split_min(&self) -> u32 {
1162 self.routing.min_run
1163 }
1164
1165 /// Records the surface opening as one pass, or as the clear and the
1166 /// opaque strips apart from the first surface round — the programmatic
1167 /// counterpart of `FRUST_ENGINE_NO_PASS_MERGE`, which only sets the value a
1168 /// renderer starts with.
1169 ///
1170 /// Takes effect from the next frame. Unmerged, a frame records the clear
1171 /// and the opaque strips as passes of their own ahead of every round (see
1172 /// the module doc's *The frame's passes*) and draws the same pixels, which
1173 /// is what a caller comparing the two arms in one process needs.
1174 pub fn set_pass_merge(&mut self, merge: bool) {
1175 self.pass_merge = merge;
1176 }
1177
1178 /// Whether the surface opening is recorded as one pass.
1179 #[must_use]
1180 pub fn pass_merge(&self) -> bool {
1181 self.pass_merge
1182 }
1183
1184 /// The passes the last [`Self::encode`] recorded into the caller's
1185 /// encoder, in recording order.
1186 ///
1187 /// Empty after a refused frame, which records no pass at all. A
1188 /// diagnostic: how many passes a frame costs is the number a tile-based
1189 /// GPU pays a whole-framebuffer load and store for, so it is what a test
1190 /// pins and a capture is read against.
1191 #[must_use]
1192 pub fn last_frame_passes(&self) -> &[FramePass] {
1193 &self.frame_passes
1194 }
1195
1196 /// How many render pipelines this renderer has compiled so far.
1197 ///
1198 /// Counts *distinct* pipelines, which is why it is a diagnostic rather
1199 /// than something to wait on: two entries of [`EnginePipeline::ALL`] that
1200 /// differ only in their colour format describe the same pipeline whenever
1201 /// the frame's target format happens to equal
1202 /// [`crate::gpu::pipelines::INTERMEDIATE_FORMAT`], and the cache compiles
1203 /// that one variant once. Use [`Self::finish_warm_up`] to wait.
1204 #[must_use]
1205 pub fn compiled_pipelines(&self) -> u64 {
1206 self.pipelines.compiled_variants()
1207 }
1208
1209 /// States whether a caller-supplied depth attachment already holds the
1210 /// depth this frame should test against — see
1211 /// [`DepthAttachment::set_pre_cleared`].
1212 pub fn set_depth_pre_cleared(&mut self, pre_cleared: bool) {
1213 self.depth.set_pre_cleared(pre_cleared);
1214 }
1215
1216 /// Whether a caller-supplied depth attachment is treated as already
1217 /// populated.
1218 ///
1219 /// The statement is sticky and set once, so a host driving several
1220 /// surfaces (or re-establishing one after a device loss) can read back
1221 /// what this renderer is on rather than tracking it a second time.
1222 #[must_use]
1223 pub fn depth_pre_cleared(&self) -> bool {
1224 self.depth.is_pre_cleared()
1225 }
1226
1227 /// Registers a caller-owned texture so a
1228 /// [`Command::SceneTexture`](frust_scene::Command::SceneTexture) naming
1229 /// `id` draws it, returning whatever was registered under `id` before.
1230 ///
1231 /// `id` is the [`SceneTextureId`] the texture minted for itself
1232 /// (`frust_gpu::Texture::as_scene_texture`), and `size` is its extent in
1233 /// texels — the rectangle a display list's destination is mapped onto.
1234 /// `view` must be a non-array 2D view of a float-sampleable texture
1235 /// carrying `wgpu::TextureUsages::TEXTURE_BINDING`; `wgpu` rejects
1236 /// anything else when the frame's bind group is built.
1237 ///
1238 /// Both halves of the registration land here: the view a pass samples and
1239 /// the extent the compiler composes a paint transform against. Registering
1240 /// is idempotent — re-registering the same id replaces the view and drops
1241 /// the bind groups naming the old one.
1242 ///
1243 /// An extent past `u16::MAX` on either axis, or a zero one, registers
1244 /// nothing and answers `None`: the record the shader reads packs the
1245 /// source region into `u16` halves, so there is no honest rectangle to
1246 /// name. Scenes drawing that id go on drawing nothing.
1247 pub fn bind_texture(
1248 &mut self,
1249 id: SceneTextureId,
1250 size: (u32, u32),
1251 view: wgpu::TextureView,
1252 ) -> Option<wgpu::TextureView> {
1253 self.resources.forget_external(id.get());
1254 if !self.compiler.bind_external_texture(id.get(), size) {
1255 return self.textures.unbind(id);
1256 }
1257 self.textures.bind(id, view)
1258 }
1259
1260 /// Removes the texture registered under `id`, returning its view.
1261 pub fn unbind_texture(&mut self, id: SceneTextureId) -> Option<wgpu::TextureView> {
1262 self.compiler.unbind_external_texture(id.get());
1263 self.resources.forget_external(id.get());
1264 self.textures.unbind(id)
1265 }
1266
1267 /// The view registered under `id`, if any.
1268 #[must_use]
1269 pub fn bound_texture(&self, id: SceneTextureId) -> Option<&wgpu::TextureView> {
1270 self.textures.get(id)
1271 }
1272
1273 /// How many external textures are currently bound.
1274 #[must_use]
1275 pub fn bound_texture_count(&self) -> usize {
1276 self.textures.len()
1277 }
1278
1279 /// Releases everything sized against the old surface extent and
1280 /// re-establishes what the next frame needs at the new one.
1281 ///
1282 /// Pipelines, shader modules, the gradient cache and the resource textures
1283 /// are all extent-independent and deliberately survive: a resize must not
1284 /// cost a pipeline rebuild or a ramp re-bake. What goes is the intermediate
1285 /// pool's parked entries — every one keyed on an extent nothing will ask
1286 /// for again — and the engine-owned depth attachment, which has to match
1287 /// its colour attachment exactly. Reallocating the depth buffer here rather
1288 /// than on the next frame keeps it off the frame path.
1289 ///
1290 /// Every retained page goes too: a page holds its body as it was laid out
1291 /// for the old extent, and the resident-key ledger published after the
1292 /// next frame reports the empty store, so every body is painted (and
1293 /// re-recorded) at the new one.
1294 pub fn resize(&mut self, device: &wgpu::Device, width: u32, height: u32) {
1295 self.targets.drop_parked();
1296 self.depth.resize(device, width, height);
1297 self.retained.clear();
1298 }
1299
1300 /// The retained pages this renderer holds resident after its last frame —
1301 /// the renderer's half of the resident-key ledger (see
1302 /// [`crate::cache::retained`]), in no particular order.
1303 pub fn retained_ledger(&self) -> impl Iterator<Item = RetainedEntry> + '_ {
1304 self.retained.entries()
1305 }
1306
1307 /// The retained store's frame clock — how many frames it has planned —
1308 /// for a ledger's diagnostic frame stamp.
1309 #[must_use]
1310 pub fn retained_frame(&self) -> u64 {
1311 self.retained.frame()
1312 }
1313
1314 /// A counter advanced whenever what [`Self::retained_ledger`] reports
1315 /// changes; a host publishing the ledger only when it moved publishes
1316 /// every change and nothing else.
1317 #[must_use]
1318 pub fn retained_generation(&self) -> u64 {
1319 self.retained.generation()
1320 }
1321
1322 /// Whether a frame was refused with [`EngineError::RetainedKeyMissing`]
1323 /// (and the store cleared) since the last call, clearing the flag. A host
1324 /// that sees `true` owes the widget tree one more paint against the
1325 /// now-empty ledger: the refused frame presented nothing.
1326 pub fn take_retained_refusal(&mut self) -> bool {
1327 self.retained.take_refusal()
1328 }
1329
1330 /// Drop every retained page — what a surface recreate or a device-scale
1331 /// change calls for. The next ledger read reports the empty store.
1332 pub fn clear_retained(&mut self) {
1333 self.retained.clear();
1334 }
1335
1336 /// The byte budget the retained store is held to.
1337 #[must_use]
1338 pub fn retained_budget(&self) -> RetainedBudget {
1339 self.retained.budget()
1340 }
1341
1342 /// Re-budget the retained store, evicting least recently used pages until
1343 /// what it holds fits.
1344 pub fn set_retained_budget(&mut self, budget: RetainedBudget) {
1345 self.retained.set_budget(budget);
1346 }
1347
1348 /// Set whether this renderer records retained brackets at all — the
1349 /// programmatic counterpart of `FRUST_ENGINE_NO_LAYERS` (see
1350 /// [`SceneCompiler::set_retained_layers`]). Turning it off clears the
1351 /// store, since nothing would ever draw from it again.
1352 pub fn set_retained_layers(&mut self, enabled: bool) {
1353 self.compiler.set_retained_layers(enabled);
1354 if !enabled {
1355 self.retained.clear();
1356 }
1357 }
1358
1359 /// Whether this renderer records retained brackets.
1360 #[must_use]
1361 pub fn retained_layers(&self) -> bool {
1362 self.compiler.retained_layers_enabled()
1363 }
1364
1365 /// Takes the time this renderer's pipeline cache has spent resolving
1366 /// variants it did not already hold (waiting on the warm-up worker's
1367 /// in-flight build, or building one inline) since the last call.
1368 ///
1369 /// `perf-trace`-only: it feeds the first frame's
1370 /// `first_submit_pipeline_wait` startup span.
1371 #[cfg(feature = "perf-trace")]
1372 pub fn take_pipeline_wait(&mut self) -> Duration {
1373 self.pipelines.take_resolve_wait()
1374 }
1375
1376 /// How many pipelines the warm-up worker has finished building.
1377 #[must_use]
1378 pub fn warm_up_finished(&self) -> u64 {
1379 self.pipelines.warm_up_finished()
1380 }
1381
1382 /// Closes the frame out: ages the intermediate pool by one frame and
1383 /// evicts the gradient cache down to its capacity.
1384 ///
1385 /// Call once per frame, after the frame's commands have been submitted and
1386 /// including frames that drew nothing — those are the frames a parked
1387 /// intermediate ages on. Gradient eviction compacts the packed LUT buffer
1388 /// and rewrites the offsets of the survivors, which is why it belongs at
1389 /// the frame boundary rather than mid-frame, where it would invalidate
1390 /// offsets the frame's own encoded paints already carry.
1391 ///
1392 /// `_queue` is part of the signature because the end-of-frame maintenance
1393 /// this method owns grows queue writes as the engine does (an atlas region
1394 /// cleared after the frame that consumed it, in the reference renderer);
1395 /// it has none of them yet.
1396 pub fn end_frame(&mut self, _queue: &wgpu::Queue) {
1397 self.targets.end_frame();
1398 self.gradients.maintain();
1399 }
1400
1401 /// Compiles `scene` and records the frame's passes into `encoder`.
1402 ///
1403 /// `root` is applied ahead of every command's own transform and
1404 /// `base_color` is what the target is cleared to before anything is drawn.
1405 /// Neither the encoder nor the queue is submitted — see the module header.
1406 ///
1407 /// # Errors
1408 ///
1409 /// [`EngineError::TargetTooLarge`] for a target outside the `u16` device
1410 /// grid the strip pipeline addresses, [`EngineError::InvalidTransform`] for
1411 /// a non-finite transform, [`EngineError::InvalidGeometry`] for non-finite
1412 /// command geometry (rect extents, radii, path points, stroke or dash
1413 /// values), [`EngineError::SchedulerEscalation`] for a layer shape the
1414 /// engine's scheduler does not serve,
1415 /// [`EngineError::IntermediateTextureTooLarge`] for a layer no page can be
1416 /// sized to, [`EngineError::AlphaCapacity`] when a frame's coverage
1417 /// outgrows the alpha texture, and [`EngineError::PaintCapacity`] when its
1418 /// encoded paints or colour ramps outgrow theirs. Every one of them is
1419 /// returned before anything is recorded, uploaded, allocated or submitted,
1420 /// so a refused frame leaves `encoder` exactly as it was found and the
1421 /// renderer's own resources — the atlas array included — exactly as they
1422 /// were. That is what lets the caller skip the frame cleanly (nothing is
1423 /// presented and the previously presented content persists) rather than
1424 /// present it half-drawn, and what keeps a refused frame's image uploads
1425 /// alive for the next frame that is not refused.
1426 #[expect(
1427 clippy::too_many_arguments,
1428 reason = "the seam frust-render drives: device, queue, encoder, scene, \
1429 target, base colour and root transform are each supplied by a \
1430 different owner, so bundling them would only move the \
1431 assembly to every call site"
1432 )]
1433 pub fn encode(
1434 &mut self,
1435 device: &wgpu::Device,
1436 queue: &wgpu::Queue,
1437 encoder: &mut wgpu::CommandEncoder,
1438 scene: &Scene,
1439 target: EngineTarget<'_>,
1440 base_color: Color,
1441 root: Affine,
1442 ) -> Result<(), EngineError> {
1443 self.encode_traced(
1444 device,
1445 queue,
1446 encoder,
1447 scene,
1448 target,
1449 base_color,
1450 root,
1451 FrameTimestamps::inert(),
1452 )
1453 }
1454
1455 /// [`Self::encode`], with each pass's GPU time stamped into `timestamps`.
1456 ///
1457 /// The one difference is the sink: every pass this records asks
1458 /// `timestamps` for its own `timestamp_writes` and takes `None` for an
1459 /// answer, so a frame encoded with [`FrameTimestamps::inert`] — which is
1460 /// exactly what [`Self::encode`] passes — records byte-identical work.
1461 /// Which pass is charged to which span is [`EngineSpan`]'s own
1462 /// documentation; the host owns the ring behind the sink and reads the
1463 /// frame's spans back out of it some frames later (see
1464 /// [`frust_gpu::diag::TimestampRing`]).
1465 ///
1466 /// # Errors
1467 ///
1468 /// Exactly [`Self::encode`]'s, on exactly its terms — a refused frame has
1469 /// recorded no pass, so it has taken no timestamp either and the host
1470 /// abandons the ring's slot rather than mapping it.
1471 #[expect(
1472 clippy::too_many_arguments,
1473 reason = "[`Self::encode`]'s argument list plus the timestamp sink, \
1474 each still supplied by a different owner"
1475 )]
1476 pub fn encode_traced(
1477 &mut self,
1478 device: &wgpu::Device,
1479 queue: &wgpu::Queue,
1480 encoder: &mut wgpu::CommandEncoder,
1481 scene: &Scene,
1482 target: EngineTarget<'_>,
1483 base_color: Color,
1484 root: Affine,
1485 timestamps: FrameTimestamps<'_>,
1486 ) -> Result<(), EngineError> {
1487 // The CPU half of the frame's accounting, lapped phase by phase
1488 // through to the end of the call (see [`EncodeSpans`]). Free without
1489 // `perf-trace` — no clock is read at all.
1490 let mut clock = PhaseClock::start();
1491 // Emptied first, so a frame refused below reports the nothing it
1492 // recorded rather than the last frame's passes.
1493 self.frame_passes.clear();
1494 let size = grid_size(target.width, target.height)?;
1495 let mut frame = self.compiler.compile(scene, root, size)?;
1496 let compile_total = clock.lap();
1497
1498 // The frame's pass plan, settled before anything is allocated or
1499 // recorded: a layer shape this scheduler does not serve, or one larger
1500 // than a page can be sized to, refuses the whole frame here so the
1501 // caller can skip it cleanly rather than present it half-drawn.
1502 //
1503 // The retained store decides first, because its answers are roles the
1504 // scheduler serves: which bodies render into a kept page, where each
1505 // `DrawRetained` composites — or that one cannot be served, which
1506 // refuses the frame here and empties the store (see
1507 // `crate::cache::retained`). Nothing it decides is applied until the
1508 // frame is past its last fallible step.
1509 let retained_plan = self
1510 .retained
1511 .plan(&frame.retained, &mut frame.recorder.layers)?;
1512 retained_roles(&frame, &retained_plan, &mut self.retained_layers);
1513 // Whether the frame's backdrop blurs can be served at all: each one
1514 // reads its region of this very target back (see `RegionCopyMode`).
1515 // A target that can be neither copied nor sampled serves none — every
1516 // backdrop's rectangle is then the transparent draw the compiler
1517 // recorded, drawing nothing, and is scheduled as the ordinary layer it
1518 // is — rather than refusing a frame it can otherwise draw whole.
1519 let copy_mode = RegionCopyMode::of(target.view.texture());
1520 let serve_backdrops = copy_mode != RegionCopyMode::Unavailable;
1521 if !frame.backdrops.is_empty() && !serve_backdrops {
1522 note_backdrop_unreadable(target.view.texture());
1523 }
1524 let backdrops: &[Backdrop] = if serve_backdrops {
1525 &frame.backdrops
1526 } else {
1527 &[]
1528 };
1529 // The largest region a texture-mode frame transfers, which the
1530 // staging texture is grown to before the first pass is recorded.
1531 let staging_extent = backdrops.iter().fold((0_u32, 0_u32), |(w, h), backdrop| {
1532 (
1533 w.max(u32::from(backdrop.region.width())),
1534 h.max(u32::from(backdrop.region.height())),
1535 )
1536 });
1537 let rounds = Schedule::build_frame(
1538 &frame.recorder,
1539 &self.caps,
1540 &self.pages,
1541 &self.retained_layers,
1542 backdrops,
1543 )?;
1544 let ceiling = self.targets.max_texture_size();
1545 for round in &rounds {
1546 if let Some(page) = round.page()
1547 && (page.size.width > ceiling || page.size.height > ceiling)
1548 {
1549 return Err(EngineError::IntermediateTextureTooLarge);
1550 }
1551 }
1552
1553 // Depth is available only when there is an attachment to use and the
1554 // kill switch is off. Settling that before a single instance is built
1555 // is what keeps the opaque/alpha split and the pass shape in agreement.
1556 let depth_enabled = !config::depth_disabled();
1557 if depth_enabled && target.depth.is_none() {
1558 self.depth.ensure(device, target.width, target.height);
1559 }
1560 // Cloned out of the renderer rather than borrowed from it: an engine-
1561 // owned attachment lives in `self.depth`, and recording the frame needs
1562 // `self` mutably (the page pool, the pipeline cache). A `wgpu`
1563 // texture view is a reference-counted handle, so the clone is a
1564 // refcount bump once per frame rather than an allocation.
1565 let depth_view = depth_enabled
1566 .then(|| target.depth.or_else(|| self.depth.owned_view()))
1567 .flatten()
1568 .cloned();
1569 let depth_view = depth_view.as_ref();
1570
1571 // Destination-out erases colour as well as alpha, so a target whose
1572 // alpha is disregarded would take a black rectangle where the display
1573 // list says nothing changes. A frame cleared to an opaque base colour
1574 // is exactly that target — every pixel of it presents opaquely — and is
1575 // the only such statement `encode` is handed, so it is what the skip
1576 // `compile::clear`'s contract calls for is decided on.
1577 let punches = !frame.clears.is_empty() && !is_opaque(base_color);
1578 let schedule_span = clock.lap();
1579
1580 // Paints are resolved before instances are built: an instance names
1581 // its paint by the texel its record starts at, which only exists once
1582 // the frame's ramps are resident and its records are laid out.
1583 let atlas_budget = self.compiler.images().budget();
1584 self.resources
1585 .resolve_paints(&frame, &mut self.gradients, atlas_budget, &self.textures);
1586 let paints_span = clock.lap();
1587 self.scratch.routing = self.routing;
1588 self.scratch.build(
1589 &frame,
1590 &rounds,
1591 depth_view.is_some(),
1592 punches,
1593 &self.resources.paint_slots,
1594 if serve_backdrops {
1595 &frame.backdrops
1596 } else {
1597 &[]
1598 },
1599 );
1600 let instances_span = clock.lap();
1601
1602 // Everything that can fail does so here, ahead of the first
1603 // `begin_render_pass` AND ahead of the first thing this frame changes
1604 // about the renderer's frame-visible state: apart from the engine's
1605 // own depth attachment (re-sized above, an internal resource no pass
1606 // has read yet), a frame refused below has allocated nothing, replaced
1607 // no texture and submitted nothing, which is what lets the caller
1608 // skip it cleanly.
1609 let dim = self.caps.resource_texture_dim;
1610 let alphas_grown = gpu::grow_alpha_texture_height(
1611 self.resources.alphas.height,
1612 frame.alphas().len(),
1613 dim,
1614 )?;
1615 let paints_grown = gpu::paint_texture::grow_encoded_paints_texture_height(
1616 self.resources.paints.height,
1617 self.resources.paint_texels(),
1618 dim,
1619 )?;
1620 let gradients_grown = self.resources.grown_gradient_height(&self.gradients)?;
1621
1622 // Past the last fallible step. The retained store applies its plan
1623 // first — evictions, then a page for every body a round really renders
1624 // into — so every retained page the frame's rounds name exists before
1625 // they are recorded. The evictions include every key the frame
1626 // re-recorded whose page will not hold that recording (refused,
1627 // nested or unrendered), so the ledger published after this frame
1628 // never names a stale page.
1629 let rendered = retained_rendered(&frame, &rounds);
1630 self.retained.commit(
1631 device,
1632 &frame.retained,
1633 &frame.retained_superseded,
1634 &retained_plan,
1635 &rendered,
1636 );
1637
1638 // The atlas is created or grown first, so
1639 // the growth copy's own submit precedes the frame's atlas writes below
1640 // (see `gpu::atlas`) and the array is deep enough for every region they
1641 // name.
1642 self.resources
1643 .ensure_atlas(device, queue, atlas_budget, frame.atlas_layers);
1644
1645 // The glyph atlas, in the order [`crate::gpu::atlas`] documents: the
1646 // rectangles last frame's eviction freed are zeroed first, ahead of
1647 // every write this frame issues, so a rectangle handed straight back
1648 // out cannot be erased after its new occupant landed in it.
1649 //
1650 // Acknowledged only once the writes were really issued — the same
1651 // re-offer contract the image plan keeps below. A frame refused before
1652 // this point leaves every rectangle pending, so the next frame that
1653 // gets here still zeroes it.
1654 if self.clear_glyph_rects(device, queue, &frame) {
1655 self.compiler.acknowledge_glyph_clears();
1656 }
1657
1658 self.resources.resize_alphas(device, alphas_grown);
1659 self.resources.resize_paints(device, paints_grown);
1660 self.resources.resize_gradients(device, gradients_grown);
1661 let resize_span = clock.lap();
1662 let atlas_serviced =
1663 self.resources
1664 .upload(queue, &mut frame, &mut self.gradients, size, dim);
1665 self.resources
1666 .upload_instances(device, queue, &self.scratch);
1667
1668 // Residency is committed exactly here: the frame passed every fallible
1669 // step and its evictions and uploads have reached the array, so the
1670 // compiler may stop re-offering them. A frame that returned early above
1671 // never gets here, and its plan is re-offered on the next frame that
1672 // does (see `cache::images`).
1673 if atlas_serviced {
1674 self.compiler.acknowledge_image_plan();
1675 }
1676 let upload_span = clock.lap();
1677
1678 // The glyph pixels themselves, last of the atlas work and strictly
1679 // before the scene pass: every page `glifo` dirtied this frame is
1680 // lowered to strips and drawn into its own array layer, on an encoder
1681 // this call owns and submits (the sanctioned exception to the encode
1682 // contract — see this module's header and `gpu::atlas`). The queue
1683 // writes issued above are flushed ahead of that submit, so the pass
1684 // composites onto a layer whose clears and image uploads have landed.
1685 //
1686 // Driven by the atlas's own pending work, never by this frame's
1687 // surviving draws: `glifo` dirties a page when it *inserts* an entry,
1688 // so a run culled away behind a clip records fills while drawing
1689 // nothing, and a draw-gated replay would leave those commands recorded
1690 // until some later frame happened to run one — by which time eviction
1691 // may have re-let the rectangles they name.
1692 //
1693 // Acknowledged only when the pass really ran, the same way the clears
1694 // above are: acknowledging is what lifts the eviction deferral, so an
1695 // acknowledgement for a replay that returned early would let `glifo`
1696 // free and re-let the very rectangles those commands still name.
1697 // The report describes only a frame whose replay ran: a frame that
1698 // skipped it must not carry the last replay's numbers forward.
1699 self.atlas_report = AtlasRenderReport::default();
1700 #[cfg(feature = "perf-trace")]
1701 let mut replayed = false;
1702 if self.compiler.glyph_replay_pending()
1703 && self.replay_glyph_pages(device, queue, timestamps)
1704 {
1705 self.compiler.acknowledge_glyph_replay();
1706 #[cfg(feature = "perf-trace")]
1707 {
1708 replayed = true;
1709 }
1710 }
1711
1712 let replay_span = clock.lap();
1713
1714 let format = target.format;
1715 let pipelines = self.frame_pipelines(device, format, depth_view.is_some(), copy_mode);
1716 if pipelines.copy.is_some() {
1717 let copies = self
1718 .region_copies
1719 .get_or_insert_with(|| RegionCopyResources::new(device));
1720 if copy_mode == RegionCopyMode::Texture {
1721 copies.ensure_staging(device, staging_extent.0, staging_extent.1);
1722 }
1723 }
1724
1725 // The filter rounds' own resources, created by the first frame that
1726 // schedules one. Only the parameter blocks are uploaded here: a pass's
1727 // instance names the extent the *pool* quantized its destination page
1728 // up to, which only the round that acquires it knows, so the instances
1729 // are written round by round in `record_frame`.
1730 if let Some(pipeline) = pipelines.filter.as_ref() {
1731 let scratch = &self.scratch;
1732 self.filters
1733 .get_or_insert_with(|| FilterResources::new(device))
1734 .prepare(
1735 device,
1736 queue,
1737 pipeline,
1738 &scratch.filter_blocks,
1739 scratch.filter_passes,
1740 );
1741 }
1742
1743 let pipelines_span = clock.lap();
1744
1745 self.record_frame(
1746 device, queue, encoder, &target, depth_view, base_color, &pipelines, copy_mode,
1747 timestamps,
1748 );
1749
1750 // The laps above still run without `perf-trace` — `PhaseClock` reads
1751 // no clock there, so each one is `Duration::ZERO` — but nothing records
1752 // them, and the trace they would have fed is not compiled at all.
1753 #[cfg(not(feature = "perf-trace"))]
1754 let _ = (
1755 compile_total,
1756 schedule_span,
1757 paints_span,
1758 instances_span,
1759 resize_span,
1760 upload_span,
1761 replay_span,
1762 pipelines_span,
1763 clock.lap(),
1764 );
1765
1766 // Last, so the window's own formatting is charged to no phase it
1767 // reports. A frame refused above records nothing: its phases are a
1768 // partial encode and would drag every percentile toward a frame that
1769 // was never presented.
1770 #[cfg(feature = "perf-trace")]
1771 self.encode_trace.record(&EncodeSpans {
1772 compile: frame.compile_spans,
1773 compile_total,
1774 schedule: schedule_span,
1775 paints: paints_span,
1776 instances: instances_span,
1777 resize: resize_span,
1778 upload: upload_span,
1779 replay: replay_span,
1780 pipelines: pipelines_span,
1781 record: clock.lap(),
1782 counts: EncodeCounts::of(
1783 &frame,
1784 replayed,
1785 &self.atlas_report,
1786 self.compiler.glyph_atlas_trace(),
1787 ),
1788 });
1789
1790 Ok(())
1791 }
1792
1793 /// [`Self::encode_traced`] over the commands of `scene` in `range` alone,
1794 /// cut out by [`project_command_range`] — the overlay spike's way of
1795 /// drawing one segment of a frame into a target of its own (render-model
1796 /// Phase 4, decision D10).
1797 ///
1798 /// The range is projected into a scene this renderer keeps for the
1799 /// purpose (its allocation reused from call to call), carrying
1800 /// `DrawRetained`s only when this renderer retains
1801 /// ([`Self::retained_layers`]) — a renderer with retention off refuses a
1802 /// frame carrying one. The bracket rule is [`project_command_range`]'s.
1803 /// `target`, `base_color` (an overlay passes transparent), `root` and
1804 /// `timestamps` mean exactly what they mean to [`Self::encode_traced`].
1805 ///
1806 /// A range covering the whole scene is still projected (and re-recorded),
1807 /// so a caller that draws the whole frame calls [`Self::encode_traced`]
1808 /// instead; this method is never on a path that does.
1809 ///
1810 /// # Errors
1811 ///
1812 /// Exactly [`Self::encode_traced`]'s, over the projected scene.
1813 #[expect(
1814 clippy::too_many_arguments,
1815 reason = "[`Self::encode_traced`]'s argument list plus the range, each \
1816 still supplied by a different owner"
1817 )]
1818 pub fn record_range(
1819 &mut self,
1820 device: &wgpu::Device,
1821 queue: &wgpu::Queue,
1822 encoder: &mut wgpu::CommandEncoder,
1823 scene: &Scene,
1824 range: Range<usize>,
1825 target: EngineTarget<'_>,
1826 base_color: Color,
1827 root: Affine,
1828 timestamps: FrameTimestamps<'_>,
1829 ) -> Result<RangeCut, EngineError> {
1830 // Taken out for the call (and always put back): `encode_traced`
1831 // borrows the renderer mutably while it reads the scene.
1832 let mut projected = std::mem::take(&mut self.range_scene);
1833 let options = RangeOptions {
1834 keep_draw_retained: self.retained_layers(),
1835 };
1836 let cut = project_command_range(scene, range, options, &mut projected);
1837 let result = self.encode_traced(
1838 device, queue, encoder, &projected, target, base_color, root, timestamps,
1839 );
1840 self.range_scene = projected;
1841 result.map(|()| cut)
1842 }
1843
1844 /// Creates the render-to-atlas pass on first use.
1845 fn ensure_atlas_renderer(&mut self, device: &wgpu::Device) {
1846 if self.atlas_glyphs.is_none() {
1847 self.atlas_glyphs = Some(AtlasRenderer::new(device, &self.caps));
1848 }
1849 }
1850
1851 /// Zero every atlas rectangle an earlier frame's glyph eviction freed,
1852 /// answering whether they were serviced.
1853 ///
1854 /// Queue writes, issued before this frame's image uploads and before the
1855 /// replay pass's submit — the first of the three orderings
1856 /// [`crate::gpu::atlas`] states. A rectangle the array will not take is
1857 /// counted rather than dropped silently, on the same terms an image region
1858 /// it refuses is.
1859 ///
1860 /// `true` means every rectangle was *offered* to the array — including one
1861 /// it refused, which no later frame could place either — so the caller may
1862 /// stop re-offering them. `false` means there was no array to write to at
1863 /// all, which is the one case where trying again later can succeed. An
1864 /// empty list is serviced trivially.
1865 fn clear_glyph_rects(
1866 &mut self,
1867 device: &wgpu::Device,
1868 queue: &wgpu::Queue,
1869 frame: &CompiledFrame,
1870 ) -> bool {
1871 if frame.glyph_clears.is_empty() {
1872 return true;
1873 }
1874 if self.resources.atlas.is_none() {
1875 return false;
1876 }
1877 self.ensure_atlas_renderer(device);
1878
1879 let refused = {
1880 let Self {
1881 atlas_glyphs,
1882 resources,
1883 ..
1884 } = self;
1885 let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_ref(), resources.atlas.as_ref())
1886 else {
1887 return false;
1888 };
1889 frame
1890 .glyph_clears
1891 .iter()
1892 .filter(|rect| !glyphs.clear_rect(queue, atlas, **rect))
1893 .count()
1894 };
1895
1896 if refused > 0 {
1897 self.resources.note_refused_regions(refused as u64);
1898 }
1899 true
1900 }
1901
1902 /// Draw every atlas page `glifo` dirtied this frame into its own array
1903 /// layer.
1904 ///
1905 /// The pixels of a newly cached glyph, and the last atlas work before the
1906 /// scene pass. Each page's recorded commands are lowered to strips by
1907 /// [`lower_atlas_page`] and drawn through the pipeline
1908 /// [`atlas_strip_desc`] describes; a page the lowering declines is left
1909 /// undrawn and counted, so a glyph whose shape this tier cannot express
1910 /// goes *missing* rather than landing half-painted.
1911 ///
1912 /// Answers whether the pass really ran, on the same terms
1913 /// [`clear_glyph_rects`](Self::clear_glyph_rects) does and for the same
1914 /// reason: `false` means there was no atlas array to draw into at all, so
1915 /// the recorded commands are still recorded and the caller must go on
1916 /// offering them. A page the lowering *declined* is not a `false` — it was
1917 /// offered to the array and counted refused, and no later frame could lower
1918 /// it either.
1919 ///
1920 /// `timestamps` is passed straight through to
1921 /// [`gpu::atlas::AtlasRenderer::render_pending`], which charges each dirty
1922 /// page's own pass to [`EngineSpan::Prepass`] — this is the frame's own
1923 /// Prepass recording site.
1924 #[must_use]
1925 fn replay_glyph_pages(
1926 &mut self,
1927 device: &wgpu::Device,
1928 queue: &wgpu::Queue,
1929 timestamps: FrameTimestamps<'_>,
1930 ) -> bool {
1931 if self.resources.atlas.is_none() {
1932 return false;
1933 }
1934 self.ensure_atlas_renderer(device);
1935 let pipeline = self
1936 .pipelines
1937 .get_or_create(device, &atlas_strip_desc(&self.shaders))
1938 .clone();
1939
1940 let report = {
1941 let Self {
1942 atlas_glyphs,
1943 atlas_lowering,
1944 resources,
1945 compiler,
1946 ..
1947 } = self;
1948 let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_mut(), resources.atlas.as_ref())
1949 else {
1950 return false;
1951 };
1952
1953 let (width, height) = atlas.size();
1954 let page = (
1955 u16::try_from(width).unwrap_or(u16::MAX),
1956 u16::try_from(height).unwrap_or(u16::MAX),
1957 );
1958 let lowering = atlas_lowering.get_or_insert_with(|| SceneCompiler::new(page.0, page.1));
1959
1960 glyphs.render_pending(
1961 device,
1962 queue,
1963 &pipeline,
1964 atlas,
1965 compiler.glyph_atlas_mut(),
1966 timestamps,
1967 |recorder, buffers| lower_atlas_page(recorder, buffers, lowering, page),
1968 )
1969 };
1970
1971 if report.refused > 0 {
1972 self.resources
1973 .note_refused_regions(u64::from(report.refused));
1974 }
1975 self.atlas_report = report;
1976 true
1977 }
1978
1979 /// What the last frame's render-to-atlas pass serviced.
1980 ///
1981 /// Zero across the board on a steady-state frame: text that hit the cache
1982 /// on every glyph frees no rectangle, queues no pixmap and dirties no page.
1983 #[must_use]
1984 pub fn atlas_render_report(&self) -> AtlasRenderReport {
1985 self.atlas_report
1986 }
1987
1988 /// Builds (or takes from the cache) every pipeline this frame's passes
1989 /// need, and the bind groups each of them will be bound through.
1990 ///
1991 /// All of it happens before the first `begin_render_pass`: a pipeline
1992 /// compiled mid-recording would be the very stall the warm-up exists to
1993 /// avoid. Each pass resolves exactly the paint classes its runs name
1994 /// ([`Scratch`]'s class sets), folded to what its state serves; the bind
1995 /// groups are built once per state, against any one of its class
1996 /// pipelines — every strip pipeline shares one explicit layout, so a group
1997 /// built against one binds under all of them.
1998 fn frame_pipelines(
1999 &mut self,
2000 device: &wgpu::Device,
2001 format: wgpu::TextureFormat,
2002 depth: bool,
2003 copy_mode: RegionCopyMode,
2004 ) -> FramePipelines {
2005 let alpha_variant = if depth {
2006 EnginePipeline::StripDepthAlpha
2007 } else {
2008 EnginePipeline::StripAlpha
2009 };
2010 let punch_variant = if depth {
2011 EnginePipeline::StripDepthDestOut
2012 } else {
2013 EnginePipeline::StripDestOut
2014 };
2015 let uber = ClassSet::only(PaintClass::Uber);
2016 if self.resolved.format != Some(format) {
2017 self.resolved = ResolvedPipelines {
2018 format: Some(format),
2019 ..ResolvedPipelines::default()
2020 };
2021 }
2022
2023 // The surface's own pass is unconditional — it is what the frame's
2024 // bind groups are built against even when it draws nothing — so a frame
2025 // with no surface run still resolves one pipeline for it: the Uber one,
2026 // which the warm-up already built.
2027 let surface_classes = if self.scratch.surface_classes.is_empty() {
2028 uber
2029 } else {
2030 self.scratch.surface_classes
2031 };
2032 let mut frame = FramePipelines {
2033 alpha: (
2034 alpha_variant,
2035 self.class_pipelines(device, format, alpha_variant, surface_classes),
2036 ),
2037 opaque: None,
2038 page: None,
2039 punch: None,
2040 filter: None,
2041 copy: None,
2042 };
2043 if depth && !self.scratch.opaque.is_empty() {
2044 frame.opaque = Some(self.class_pipelines(
2045 device,
2046 format,
2047 EnginePipeline::StripOpaque,
2048 self.scratch.opaque_classes,
2049 ));
2050 }
2051 if self.scratch.page_rounds() > 0 {
2052 let page_classes = if self.scratch.page_classes.is_empty() {
2053 uber
2054 } else {
2055 self.scratch.page_classes
2056 };
2057 frame.page = Some(self.class_pipelines(
2058 device,
2059 format,
2060 EnginePipeline::StripIntermediate,
2061 page_classes,
2062 ));
2063 }
2064 if self.scratch.punches() {
2065 frame.punch = Some((
2066 punch_variant,
2067 self.class_pipelines(
2068 device,
2069 format,
2070 punch_variant,
2071 ClassSet::only(self.routing.punch_class()),
2072 ),
2073 ));
2074 }
2075 if self.scratch.filter_passes > 0 {
2076 // Takes the frame's format like every other variant and ignores
2077 // it: a filter pass only ever writes a pooled page, so its own
2078 // description is pinned to `INTERMEDIATE_FORMAT`. One filter
2079 // pipeline therefore serves a renderer for its whole life,
2080 // whatever its surface is reconfigured to.
2081 frame.filter = Some(
2082 self.pipelines
2083 .get_or_create(device, &EnginePipeline::Filter.desc(&self.shaders, format))
2084 .clone(),
2085 );
2086 }
2087
2088 if self.scratch.copies > 0 && copy_mode != RegionCopyMode::Unavailable {
2089 // Pinned to `INTERMEDIATE_FORMAT` like the filter pipeline — a copy
2090 // only ever writes a pooled page — so one serves the renderer for
2091 // its whole life whatever the frame's own format.
2092 frame.copy = Some(
2093 self.pipelines
2094 .get_or_create(device, &EnginePipeline::Copy.desc(&self.shaders, format))
2095 .clone(),
2096 );
2097 }
2098
2099 if let Some(pipeline) = frame.alpha.1.any() {
2100 self.resources
2101 .ensure_bind_groups(device, frame.alpha.0, pipeline, format);
2102 self.resources
2103 .ensure_external_groups(device, frame.alpha.0, pipeline, &self.textures);
2104 }
2105 if let Some(pipeline) = frame.opaque.as_ref().and_then(ClassPipelines::any) {
2106 self.resources.ensure_bind_groups(
2107 device,
2108 EnginePipeline::StripOpaque,
2109 pipeline,
2110 format,
2111 );
2112 }
2113 if let Some(pipeline) = frame.page.as_ref().and_then(ClassPipelines::any) {
2114 self.resources.ensure_bind_groups(
2115 device,
2116 EnginePipeline::StripIntermediate,
2117 pipeline,
2118 format,
2119 );
2120 // A layer's own round draws through this variant, so an external
2121 // texture inside an isolated layer needs its group here too.
2122 self.resources.ensure_external_groups(
2123 device,
2124 EnginePipeline::StripIntermediate,
2125 pipeline,
2126 &self.textures,
2127 );
2128 }
2129 if let Some((variant, pipelines)) = frame.punch.as_ref()
2130 && let Some(pipeline) = pipelines.any()
2131 {
2132 self.resources
2133 .ensure_bind_groups(device, *variant, pipeline, format);
2134 }
2135 self.resources
2136 .ensure_page_configs(device, self.scratch.page_rounds());
2137
2138 frame
2139 }
2140
2141 /// The pipelines `state` draws `classes` with — each class folded to what
2142 /// the state serves — resolved through this renderer's memo first and the
2143 /// pipeline cache (warm-up, steal, or an inline compile on a class's first
2144 /// use) only on a miss.
2145 fn class_pipelines(
2146 &mut self,
2147 device: &wgpu::Device,
2148 format: wgpu::TextureFormat,
2149 state: EnginePipeline,
2150 classes: ClassSet,
2151 ) -> ClassPipelines {
2152 let mut out = ClassPipelines::default();
2153 for class in classes.iter() {
2154 let class = state.fold_class(class);
2155 if out.get(class).is_some() {
2156 continue;
2157 }
2158 let slot = ResolvedPipelines::slot(state, class);
2159 let pipeline = match self.resolved.entries.get(slot).and_then(Option::as_ref) {
2160 Some(pipeline) => pipeline.clone(),
2161 None => {
2162 let pipeline = self
2163 .pipelines
2164 .get_or_create(device, &state.class_desc(&self.shaders, format, class))
2165 .clone();
2166 if let Some(entry) = self.resolved.entries.get_mut(slot) {
2167 *entry = Some(pipeline.clone());
2168 }
2169 pipeline
2170 }
2171 };
2172 out.set(class, pipeline);
2173 }
2174 out
2175 }
2176
2177 /// Records the frame's surface opening, every round's own pass, and the
2178 /// hole punch — the pass list the module doc's *The frame's passes*
2179 /// states — logging each pass it begins into [`Self::frame_passes`].
2180 ///
2181 /// Every pass opened here is ended before the method returns, which is the
2182 /// half of the encode contract a caller cannot check for itself.
2183 ///
2184 /// Each pass names the [`EngineSpan`] it is charged to: the frame's own
2185 /// surface passes are [`EngineSpan::Main`], a layer page round and a
2186 /// filter pass are [`EngineSpan::Composite`]. Several passes per span is
2187 /// the ordinary case and they sum. A pass takes its query pair only when
2188 /// it is really begun, so no pair is ever taken for a pass that was not
2189 /// recorded.
2190 #[expect(
2191 clippy::too_many_arguments,
2192 reason = "one frame's full recording state, each piece owned by a \
2193 different part of the renderer; bundling them would move the \
2194 same assembly one call up"
2195 )]
2196 fn record_frame(
2197 &mut self,
2198 device: &wgpu::Device,
2199 queue: &wgpu::Queue,
2200 encoder: &mut wgpu::CommandEncoder,
2201 target: &EngineTarget<'_>,
2202 depth_view: Option<&wgpu::TextureView>,
2203 base_color: Color,
2204 pipelines: &FramePipelines,
2205 copy_mode: RegionCopyMode,
2206 timestamps: FrameTimestamps<'_>,
2207 ) {
2208 let merge = self.pass_merge;
2209 let clear = clear_color(base_color, target.output);
2210 let depth_load = self.depth.load_op(target.depth.is_some());
2211
2212 // Under the kill switch the clear is a pass of its own, recorded first
2213 // and drawing nothing, with the opaque strips in a pass of their own
2214 // after it. It is timed alongside the frame's other surface
2215 // passes even though a backend that samples its counters at the
2216 // vertex/fragment stage boundaries may write nothing for it — an
2217 // untimed pass is a measurement gap, not a wrong measurement (see
2218 // `frust_gpu::diag`).
2219 if !merge {
2220 drop(encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
2221 label: Some(FramePass::Clear.label()),
2222 color_attachments: &[Some(color_attachment(
2223 target.view,
2224 wgpu::LoadOp::Clear(clear),
2225 ))],
2226 depth_stencil_attachment: depth_view.map(|view| depth_attachment(view, depth_load)),
2227 timestamp_writes: timestamps.writes(EngineSpan::Main),
2228 occlusion_query_set: None,
2229 multiview_mask: None,
2230 }));
2231 self.frame_passes.push(FramePass::Clear);
2232 }
2233
2234 // Every field below is reached through `self.<field>` rather than
2235 // through a method: the page pool is borrowed mutably for the whole
2236 // walk while the instance buffer, the bind groups and the plan are
2237 // borrowed immutably, and only disjoint field borrows let those
2238 // coexist.
2239 let Some(instances) = self.resources.instances.as_ref() else {
2240 // No instance has ever been uploaded, so the frame draws nothing
2241 // at all — and still resolves to a clean surface: the opening with
2242 // nothing to draw is exactly the clear.
2243 if merge {
2244 SurfaceOpening {
2245 view: target.view,
2246 depth: depth_view,
2247 clear,
2248 depth_load,
2249 opaque: None,
2250 }
2251 .record(
2252 encoder,
2253 None,
2254 timestamps.writes(EngineSpan::Main),
2255 &mut self.frame_passes,
2256 );
2257 }
2258 #[cfg(debug_assertions)]
2259 self.check_pass_list(merge, false, false, false);
2260 return;
2261 };
2262 let dim = self.caps.resource_texture_dim;
2263 let opaque_count = self.scratch.opaque.len() as u32;
2264 // The alpha region starts where the opaque one ends, so every segment's
2265 // own index is relative to that.
2266 let base = opaque_count;
2267
2268 // The opaque strips, once for the whole frame and ahead of every
2269 // surface draw: the depth they write is what every blended instance
2270 // the frame draws onto its own target, composites included, is then
2271 // tested against.
2272 //
2273 // Once rather than ahead of each surface round, because a frame can
2274 // take several of those (the scheduler cuts one short to hand a page
2275 // group back) and a second recording would re-draw opaque coverage at
2276 // equal stored depth over composites the round before it had already
2277 // blended. Ahead of the layer rounds or after them makes no difference
2278 // to those: a layer round writes a pooled page and reads neither this
2279 // target nor the depth attachment.
2280 let opaque = if opaque_count > 0
2281 && let Some(opaque) = pipelines.opaque.as_ref()
2282 && let Some(opaque_groups) =
2283 self.resources.bind_groups.get(&EnginePipeline::StripOpaque)
2284 {
2285 Some(PassPlan {
2286 pass: FramePass::Opaque,
2287 view: target.view,
2288 load: wgpu::LoadOp::Load,
2289 depth: depth_view,
2290 state: EnginePipeline::StripOpaque,
2291 pipelines: opaque,
2292 groups: opaque_groups,
2293 resources: &opaque_groups.resources,
2294 composites: &[],
2295 // An external paint is never claimed opaque, so the
2296 // depth-writing pass never holds a run.
2297 external_keys: &[],
2298 instances,
2299 base: 0,
2300 // The whole opaque region, `(0, opaque_count)`, cut into its
2301 // own class runs when the frame was planned.
2302 segments: &self.scratch.opaque_segments,
2303 composite_class: self.routing.composite_class(),
2304 })
2305 } else {
2306 None
2307 };
2308 let opaque_drawn = opaque.is_some();
2309
2310 // The surface opening: the clear, then the opaque strips, then — when
2311 // one reaches the surface before anything else does — the first
2312 // surface round's own strips, all in ONE pass. Owed until the first
2313 // pass that touches the surface takes it: the first surface round
2314 // that draws, or a punch issued ahead of any (which takes it as a
2315 // pass of its own, drawing no round). A page or filter round never
2316 // takes it — neither reads nor writes the surface or its depth — so
2317 // the opening simply waits past them. Whatever is still owed at the
2318 // end of the walk is recorded there, which is how a frame whose
2319 // surface draws nothing still resolves to a clean surface.
2320 //
2321 // Under the kill switch the clear was recorded above and the opaque
2322 // strips are recorded here, as passes of their own, and nothing is
2323 // owed.
2324 let mut opening = if merge {
2325 Some(SurfaceOpening {
2326 view: target.view,
2327 depth: depth_view,
2328 clear,
2329 depth_load,
2330 opaque,
2331 })
2332 } else {
2333 if let Some(opaque) = opaque.as_ref() {
2334 record_pass(
2335 encoder,
2336 opaque,
2337 timestamps.writes(EngineSpan::Main),
2338 &mut self.frame_passes,
2339 );
2340 }
2341 None
2342 };
2343
2344 // The destination-out pass's own recording state, resolved once for the
2345 // frame: a punch can now land at any of the walk's cuts, and every one
2346 // of them erases the same target through the same pipeline.
2347 let punch_pass = pipelines.punch.as_ref().and_then(|(variant, punch)| {
2348 Some(PunchPass {
2349 view: target.view,
2350 depth: depth_view,
2351 state: *variant,
2352 pipelines: punch,
2353 class: self.routing.punch_class(),
2354 groups: self.resources.bind_groups.get(variant)?,
2355 instances,
2356 base,
2357 })
2358 });
2359 let punch_drawn = punch_pass.is_some();
2360
2361 // The frame's live pages — the two ping-pong groups and the one spill
2362 // page beside them — each holding the finished page a later round
2363 // composites (see [`crate::schedule`]).
2364 let mut live: [Option<PooledTexture>; MAX_LIVE_PAGES] = [const { None }; MAX_LIVE_PAGES];
2365 let mut page_slot = 0_usize;
2366 // The fragment region copies' own instance slots, one per copy round.
2367 let mut copy_slot = 0_u32;
2368
2369 for plan in &self.scratch.rounds {
2370 let own = match plan.page {
2371 None => None,
2372 // A round continuing a page an earlier round of the same layer
2373 // opened takes that very texture back out of its group: a fresh
2374 // one from the pool would hold the previous holder's pixels
2375 // instead of the half already drawn.
2376 Some(page) if page.continued => match live[page.parity.index()].take() {
2377 Some(pooled) => Some((page.parity, pooled)),
2378 // Unreachable: the round that opened the page put it in
2379 // this group, and no round between the two releases it.
2380 None => continue,
2381 },
2382 Some(page) => match self.targets.acquire(
2383 device,
2384 page.size.width,
2385 page.size.height,
2386 PAGE_LABEL,
2387 ) {
2388 IntermediateTexture::Texture(pooled) => Some((page.parity, pooled)),
2389 // Unreachable: every page extent was checked against this
2390 // pool's own ceiling before the first pass was recorded.
2391 // Skipping the round draws less rather than taking a device
2392 // error mid-frame.
2393 IntermediateTexture::TooLarge { .. } => continue,
2394 },
2395 };
2396
2397 // A region-copy round seeds its page with a region of the frame's
2398 // own target, read back after everything painted before it in
2399 // painter order — so the opening, which clears the target and
2400 // draws the opaque strips, is recorded first if it is still owed
2401 // (the scheduler already cut the surface round ahead of it).
2402 if let Some(source) = plan.copy {
2403 if let Some((_, pooled)) = own.as_ref() {
2404 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
2405 RegionCopyPass {
2406 mode: copy_mode,
2407 target: target.view,
2408 page: pooled,
2409 source,
2410 pipeline: pipelines.copy.as_ref(),
2411 resources: self.region_copies.as_ref(),
2412 slot: copy_slot,
2413 }
2414 .record(
2415 device,
2416 queue,
2417 encoder,
2418 timestamps.writes(EngineSpan::Composite),
2419 );
2420 copy_slot = copy_slot.saturating_add(1);
2421 self.frame_passes.push(FramePass::Copy);
2422 }
2423 settle_pages(&mut self.targets, &mut live, plan.released, own);
2424 continue;
2425 }
2426
2427 // A filter round is only a filter pass: no strip instance, no
2428 // composite, no viewport uniform of its own — the pass maps NDC
2429 // against the destination extent its own instance carries, which is
2430 // why that instance is written here, where the pool's quantized
2431 // extent is finally known.
2432 if let Some(filter) = plan.filter {
2433 if let Some((_, pooled)) = own.as_ref()
2434 && let Some(pipeline) = pipelines.filter.as_ref()
2435 && let Some(filters) = self.filters.as_ref()
2436 {
2437 let (width, height) = pooled.size();
2438 filters.write_instance(
2439 queue,
2440 filter.instance,
2441 &FilterInstanceData::new(
2442 &filter.step,
2443 filter.data_offset,
2444 // Both pages hold the layer at their own origin, so
2445 // neither region is offset within its page.
2446 (0, 0),
2447 (0, 0),
2448 SizeU16::from_wh(
2449 u16::try_from(width).unwrap_or(u16::MAX),
2450 u16::try_from(height).unwrap_or(u16::MAX),
2451 ),
2452 filter.original,
2453 ),
2454 );
2455 // A group with no live page samples the transparent
2456 // placeholder, which filters nothing — the same "draw less,
2457 // never wrong" answer an unresolvable paint gets.
2458 // Unreachable: the round that wrote this pass's source is
2459 // the one before it, and nothing between the two releases
2460 // that group.
2461 let source = live[filter.source.index()].as_ref().map_or(
2462 &self.resources.placeholders.layer_input,
2463 PooledTexture::view,
2464 );
2465 filters.record_pass_timed(
2466 device,
2467 encoder,
2468 &FilterPassPlan {
2469 label: FILTER_LABEL,
2470 pipeline,
2471 dest: pooled.view(),
2472 source,
2473 instance: filter.instance,
2474 },
2475 timestamps.writes(EngineSpan::Composite),
2476 );
2477 self.frame_passes.push(FramePass::Filter);
2478 }
2479
2480 settle_pages(&mut self.targets, &mut live, plan.released, own);
2481 continue;
2482 }
2483
2484 // The off-screen texture this round draws into, when it is not the
2485 // surface: the pooled page acquired above, or the retained page the
2486 // store bound to this round's body (`crate::cache::retained`). A
2487 // retained round whose page is missing — unreachable, the store
2488 // committed one for every body a round renders — draws nothing
2489 // rather than drawing onto the surface.
2490 let retained_page = plan
2491 .retained
2492 .and_then(|retained| self.retained.page_for(retained.bracket));
2493 if plan.retained.is_some() && retained_page.is_none() {
2494 settle_pages(&mut self.targets, &mut live, plan.released, own);
2495 continue;
2496 }
2497 let offscreen = own
2498 .as_ref()
2499 .map(|(_, pooled)| (pooled.view(), pooled.size()))
2500 .or(retained_page);
2501 let continued = plan.page.is_some_and(|page| page.continued)
2502 || plan.retained.is_some_and(|retained| retained.continued);
2503
2504 // The round's viewport uniform. A page's is written here rather
2505 // than with the frame's other uploads because only the pool knows
2506 // the extent it quantized the request up to, and NDC is computed
2507 // against the attachment's real extent. Distinct buffers, so the
2508 // write ordering against the frame's own config never matters.
2509 let config = match offscreen {
2510 None => &self.resources.config,
2511 Some((_, (width, height))) => {
2512 let Some(config) = self.resources.page_configs.get(page_slot) else {
2513 continue;
2514 };
2515 queue.write_buffer(
2516 config,
2517 0,
2518 bytemuck::bytes_of(&GpuConfig::new(width, height, dim, dim)),
2519 );
2520 page_slot = page_slot.saturating_add(1);
2521 config
2522 }
2523 };
2524
2525 let variant = match offscreen {
2526 None => pipelines.alpha.0,
2527 Some(_) => EnginePipeline::StripIntermediate,
2528 };
2529 let class_pipelines = match (offscreen, pipelines.page.as_ref()) {
2530 (None, _) => &pipelines.alpha.1,
2531 (Some(_), Some(page)) => page,
2532 (Some(_), None) => continue,
2533 };
2534 // The pipeline group 0 is built against: any of the pass's class
2535 // pipelines will do, since every strip pipeline shares one layout.
2536 let Some(pipeline) = class_pipelines.any() else {
2537 continue;
2538 };
2539 let Some(groups) = self.resources.bind_groups.get(&variant) else {
2540 continue;
2541 };
2542
2543 // A page round binds its own viewport uniform; the root round's is
2544 // already the one the shared set carries.
2545 let page_resources = offscreen.map(|_| {
2546 resources_bind_group(
2547 device,
2548 pipeline,
2549 &self.resources.alphas.view,
2550 config,
2551 &self.resources.placeholders.layer_input,
2552 )
2553 });
2554 let resources = page_resources.as_ref().unwrap_or(&groups.resources);
2555
2556 let segments = self
2557 .scratch
2558 .segments
2559 .get(plan.segments.clone())
2560 .unwrap_or(&[]);
2561 let mut composites: Vec<wgpu::BindGroup> = Vec::new();
2562 for segment in segments {
2563 // A backdrop's draws sample the page holding its blurred
2564 // region through the same group-0 layer input a composite
2565 // reads, so its group is built here, in segment order, beside
2566 // the composites' own.
2567 if let Segment::Backdrop(_, _, parity) = *segment {
2568 let view = live[parity.index()].as_ref().map_or(
2569 &self.resources.placeholders.layer_input,
2570 PooledTexture::view,
2571 );
2572 composites.push(resources_bind_group(
2573 device,
2574 pipeline,
2575 &self.resources.alphas.view,
2576 config,
2577 view,
2578 ));
2579 }
2580 if let Segment::Composite(_, source) = *segment {
2581 // A group with no live page — or a retained bracket with no
2582 // page bound — samples the transparent placeholder, which
2583 // composites nothing: the same "draw less, never wrong"
2584 // answer an unresolvable paint gets.
2585 let view = match source {
2586 CompositeSource::Page(parity) => live[parity.index()].as_ref().map_or(
2587 &self.resources.placeholders.layer_input,
2588 PooledTexture::view,
2589 ),
2590 CompositeSource::Retained(bracket) => self
2591 .retained
2592 .page_for(bracket)
2593 .map_or(&self.resources.placeholders.layer_input, |(view, _)| view),
2594 };
2595 composites.push(resources_bind_group(
2596 device,
2597 pipeline,
2598 &self.resources.alphas.view,
2599 config,
2600 view,
2601 ));
2602 }
2603 }
2604
2605 // A page round draws into an off-screen layer page, the frame's own
2606 // rounds into the surface — the split the two spans name.
2607 let span = if offscreen.is_some() {
2608 EngineSpan::Composite
2609 } else {
2610 EngineSpan::Main
2611 };
2612
2613 let (view, pass, load, depth) = match offscreen {
2614 Some((view, _)) => (
2615 view,
2616 FramePass::Page,
2617 // A pooled texture holds whatever its last holder left
2618 // there, and a retained one whatever this body's last frame
2619 // left, so a layer's first round into a page clears it —
2620 // and a round continuing that same page loads it, because
2621 // clearing again would wipe the half already drawn.
2622 if continued {
2623 wgpu::LoadOp::Load
2624 } else {
2625 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT)
2626 },
2627 None,
2628 ),
2629 None => (
2630 target.view,
2631 FramePass::Surface,
2632 wgpu::LoadOp::Load,
2633 depth_view,
2634 ),
2635 };
2636
2637 // A surface plan with nothing of its own to draw records no pass:
2638 // loading and storing the target unchanged is exactly nothing. A
2639 // punch cut falling at the very start of a round — a `ClearRect`
2640 // recorded before anything the round draws — is how one arises, and
2641 // so is a surface whose every span went to the opaque strips; the
2642 // opening stays owed past it either way. A *page* plan with no
2643 // segments still records its pass, because that is what clears the
2644 // pooled page.
2645 if !segments.is_empty() || offscreen.is_some() {
2646 let round = PassPlan {
2647 pass,
2648 view,
2649 load,
2650 depth,
2651 state: variant,
2652 pipelines: class_pipelines,
2653 groups,
2654 resources,
2655 composites: &composites,
2656 external_keys: self.resources.external_runs.keys(),
2657 instances,
2658 base,
2659 segments,
2660 composite_class: self.routing.composite_class(),
2661 };
2662 // The first surface round that draws takes the opening: one
2663 // pass clears, draws the opaque strips, then this round's own
2664 // segments in order — the pipeline switching between them
2665 // inside the pass — and ends where this round's pass would
2666 // have ended.
2667 let opening = if offscreen.is_none() {
2668 opening.take()
2669 } else {
2670 None
2671 };
2672 match opening {
2673 Some(opening) => opening.record(
2674 encoder,
2675 Some(&round),
2676 timestamps.writes(span),
2677 &mut self.frame_passes,
2678 ),
2679 None => record_pass(
2680 encoder,
2681 &round,
2682 timestamps.writes(span),
2683 &mut self.frame_passes,
2684 ),
2685 }
2686 }
2687
2688 settle_pages(&mut self.targets, &mut live, plan.released, own);
2689
2690 // The cut this plan ends at, erased once the ops before it have
2691 // been recorded and before the plan after it draws a thing. Inert
2692 // on every plan the walk did not cut, which is every plan of a
2693 // frame that punches nothing. A punch issued before the opening
2694 // was taken has to erase a cleared surface, so the opening is
2695 // recorded first, as a pass of its own.
2696 if let Some(punch) = punch_pass.as_ref()
2697 && plan.punch.1 > 0
2698 {
2699 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
2700 punch.record(encoder, plan.punch, timestamps, &mut self.frame_passes);
2701 }
2702 }
2703
2704 // An opening no surface round took — a frame whose surface draws
2705 // nothing of its own past the opaque strips, or nothing at all — is
2706 // recorded here, ahead of the trailing punch it must precede.
2707 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
2708
2709 // The punches past every op of the frame, in the position the pass held
2710 // unconditionally before painter order was restored.
2711 if let Some(punch) = punch_pass.as_ref() {
2712 punch.record(
2713 encoder,
2714 self.scratch.punch,
2715 timestamps,
2716 &mut self.frame_passes,
2717 );
2718 }
2719
2720 for page in live.into_iter().flatten() {
2721 self.targets.release(page);
2722 }
2723
2724 #[cfg(debug_assertions)]
2725 self.check_pass_list(merge, true, opaque_drawn, punch_drawn);
2726 #[cfg(not(debug_assertions))]
2727 let _ = (opaque_drawn, punch_drawn);
2728 }
2729
2730 /// Checks that the passes the walk just recorded are exactly the ones
2731 /// [`Scratch::plan_passes`] says a frame of this shape records — the
2732 /// host-testable statement of the pass list, held to the recording on
2733 /// every frame a debug build encodes.
2734 #[cfg(debug_assertions)]
2735 fn check_pass_list(&self, merge: bool, instances: bool, opaque: bool, punch: bool) {
2736 let mut planned = Vec::new();
2737 self.scratch.plan_passes(
2738 &PassShape {
2739 merge,
2740 instances,
2741 opaque,
2742 punch,
2743 },
2744 &mut planned,
2745 );
2746 debug_assert_eq!(
2747 planned, self.frame_passes,
2748 "the recorded pass list disagrees with the planned one"
2749 );
2750 }
2751}
2752
2753/// One render pass a frame recorded into the caller's encoder — what
2754/// [`EngineRenderer::last_frame_passes`] lists, in recording order.
2755///
2756/// The module doc's *The frame's passes* is the contract these name. The
2757/// glyph-atlas replay's passes are not among them: they are recorded on an
2758/// encoder of the engine's own, ahead of the frame (see [`crate::gpu::atlas`]).
2759#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
2760pub enum FramePass {
2761 /// The surface opening: clears the colour target (and the depth
2762 /// attachment, unless pre-cleared), then draws the opaque strips and the
2763 /// first surface round that draws, whichever of the two the frame has.
2764 /// The first pass over the surface whenever passes are merged.
2765 Opening,
2766 /// The clear alone, drawing nothing — the first pass of a frame recorded
2767 /// with passes unmerged.
2768 Clear,
2769 /// The opaque strips alone, depth-writing, loading what the clear left —
2770 /// recorded only with passes unmerged.
2771 Opaque,
2772 /// A surface round's blended strips, loading the surface.
2773 Surface,
2774 /// A layer page round, rendering into a pooled page.
2775 Page,
2776 /// One pass of a filter's sequence, between two pooled pages.
2777 Filter,
2778 /// A region copy: a region of the frame's own target read back into a
2779 /// pooled page by one pass that clears the page and draws the region into
2780 /// it — preceded, on a target that can only be copied, by a transfer into
2781 /// the renderer's staging texture (see [`RegionCopyMode`]) — ahead of a
2782 /// backdrop blur's filter passes.
2783 Copy,
2784 /// The destination-out hole punch, at a cut or after the last round.
2785 Punch,
2786}
2787
2788impl FramePass {
2789 /// The label the pass is recorded under, as a GPU capture shows it.
2790 #[must_use]
2791 pub const fn label(self) -> &'static str {
2792 match self {
2793 FramePass::Opening => "frust-engine surface opening",
2794 FramePass::Clear => "frust-engine clear",
2795 FramePass::Opaque => "frust-engine opaque strips",
2796 FramePass::Surface => "frust-engine alpha strips",
2797 FramePass::Page => PAGE_LABEL,
2798 FramePass::Filter => FILTER_LABEL,
2799 FramePass::Copy => "frust-engine region copy",
2800 FramePass::Punch => "frust-engine hole punch",
2801 }
2802 }
2803
2804 /// Whether the pass renders into the frame's own target rather than into
2805 /// a pooled page.
2806 #[must_use]
2807 pub const fn targets_surface(self) -> bool {
2808 !matches!(self, FramePass::Page | FramePass::Filter | FramePass::Copy)
2809 }
2810}
2811
2812/// What a frame's pass list depends on beyond its plan, as
2813/// [`Scratch::plan_passes`] reads it.
2814#[cfg(any(test, debug_assertions))]
2815#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2816struct PassShape {
2817 /// Whether the surface opening is merged (the renderer's
2818 /// [`EngineRenderer::pass_merge`]).
2819 merge: bool,
2820 /// Whether the frame has an instance buffer to draw from at all.
2821 instances: bool,
2822 /// Whether the frame draws opaque strips — depth in play, opaque spans to
2823 /// route, and their pipelines resolved.
2824 opaque: bool,
2825 /// Whether the frame records its hole punches.
2826 punch: bool,
2827}
2828
2829/// The pipelines one frame's passes are recorded with, resolved once before the
2830/// first `begin_render_pass`.
2831///
2832/// Only `alpha` is unconditional: the rest exist exactly when the frame has
2833/// work for them, so a plain frame compiles and binds nothing it will not draw.
2834#[derive(Debug)]
2835struct FramePipelines {
2836 /// The blended pass over the frame's own target, with its variant — which
2837 /// of the two it is depends on whether depth is in play.
2838 alpha: (EnginePipeline, ClassPipelines),
2839 /// The depth-writing pass, when depth is available and the frame has
2840 /// fully covered spans to route into it.
2841 opaque: Option<ClassPipelines>,
2842 /// The pass a layer page is rendered through, when the frame has one.
2843 page: Option<ClassPipelines>,
2844 /// The destination-out pass, with its variant, when the frame punches.
2845 punch: Option<(EnginePipeline, ClassPipelines)>,
2846 /// The pass one filter round runs through, when the frame filters a layer.
2847 filter: Option<wgpu::RenderPipeline>,
2848 /// The pass a fragment region copy runs through, when the frame copies a
2849 /// region of a target it can only sample ([`RegionCopyMode::Fragment`]).
2850 copy: Option<wgpu::RenderPipeline>,
2851}
2852
2853/// How a renderer cuts its passes into paint-class runs.
2854///
2855/// Settled from [`crate::config`] when the renderer is built
2856/// (`FRUST_ENGINE_UBER_PROGRAM`, `FRUST_ENGINE_RUN_SPLIT_MIN`) and overridable
2857/// per renderer, so a caller can compare the two arms in one process.
2858#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2859struct ClassRouting {
2860 /// The kill switch: every run is drawn with [`PaintClass::Uber`] and no
2861 /// segment is split.
2862 uber: bool,
2863 /// The shortest run worth a pipeline switch of its own.
2864 min_run: u32,
2865}
2866
2867impl Default for ClassRouting {
2868 fn default() -> Self {
2869 Self {
2870 uber: false,
2871 min_run: crate::gpu::strips::RUN_SPLIT_MIN,
2872 }
2873 }
2874}
2875
2876impl ClassRouting {
2877 /// The routing a renderer starts with, from the process configuration.
2878 fn from_config() -> Self {
2879 Self {
2880 uber: config::uber_program_forced(),
2881 min_run: config::run_split_min(),
2882 }
2883 }
2884
2885 /// The class a composite quad is drawn with.
2886 fn composite_class(self) -> PaintClass {
2887 if self.uber {
2888 PaintClass::Uber
2889 } else {
2890 PaintClass::LayerComposite
2891 }
2892 }
2893
2894 /// The class a hole punch is drawn with.
2895 fn punch_class(self) -> PaintClass {
2896 if self.uber {
2897 PaintClass::Uber
2898 } else {
2899 PaintClass::SolidCoverage
2900 }
2901 }
2902}
2903
2904/// One strip state's pipelines, by paint class — the ones a pass resolved for
2905/// the classes its runs name.
2906#[derive(Debug, Default)]
2907struct ClassPipelines {
2908 by_class: [Option<wgpu::RenderPipeline>; PaintClass::ALL.len()],
2909}
2910
2911impl ClassPipelines {
2912 /// The pipeline drawing `class`, if the frame resolved one.
2913 fn get(&self, class: PaintClass) -> Option<&wgpu::RenderPipeline> {
2914 self.by_class.get(class.index()).and_then(Option::as_ref)
2915 }
2916
2917 fn set(&mut self, class: PaintClass, pipeline: wgpu::RenderPipeline) {
2918 if let Some(slot) = self.by_class.get_mut(class.index()) {
2919 *slot = Some(pipeline);
2920 }
2921 }
2922
2923 /// Any one of the pipelines — what a bind group is built against, every
2924 /// strip pipeline sharing one explicit layout.
2925 fn any(&self) -> Option<&wgpu::RenderPipeline> {
2926 self.by_class.iter().flatten().next()
2927 }
2928}
2929
2930/// The strip pipelines a renderer has resolved so far, by `(state, class)`,
2931/// for the target format they were built against.
2932///
2933/// A memo in front of the pipeline cache: a steady-state frame takes every
2934/// pipeline it draws with from here, by indexing a dense `state x class` table,
2935/// rather than rebuilding each description (a vertex-layout allocation) and
2936/// packing its cache key — which keeps the frame's `pipelines` phase from
2937/// growing with the number of classes it draws.
2938#[derive(Debug)]
2939struct ResolvedPipelines {
2940 /// The format every entry was built for; a change drops them all.
2941 format: Option<wgpu::TextureFormat>,
2942 /// Indexed by [`Self::slot`].
2943 entries: [Option<wgpu::RenderPipeline>; RESOLVED_SLOTS],
2944}
2945
2946/// One [`ResolvedPipelines`] entry per render state and paint class.
2947const RESOLVED_SLOTS: usize = EnginePipeline::ALL.len() * PaintClass::ALL.len();
2948
2949impl Default for ResolvedPipelines {
2950 fn default() -> Self {
2951 Self {
2952 format: None,
2953 entries: [const { None }; RESOLVED_SLOTS],
2954 }
2955 }
2956}
2957
2958impl ResolvedPipelines {
2959 /// The table slot of `state` drawing `class`.
2960 const fn slot(state: EnginePipeline, class: PaintClass) -> usize {
2961 state.index() * PaintClass::ALL.len() + class.index()
2962 }
2963}
2964
2965/// One pass's full recording state, assembled before the pass is begun.
2966///
2967/// A struct rather than an argument list because the composite groups have to
2968/// be built (and so borrowed) before `begin_render_pass` takes the encoder, and
2969/// naming them together is what makes that ordering obvious at the call site.
2970///
2971/// The attachment half (`pass`, `view`, `load`, `depth`) describes the pass the
2972/// plan begins when it is recorded on its own; the rest is what it draws, which
2973/// is all the [surface opening](SurfaceOpening) reads of the plans it draws
2974/// inside its own pass.
2975struct PassPlan<'a> {
2976 /// Which of the frame's passes this is — its label, and its entry in
2977 /// [`EngineRenderer::last_frame_passes`].
2978 pass: FramePass,
2979 view: &'a wgpu::TextureView,
2980 load: wgpu::LoadOp<wgpu::Color>,
2981 depth: Option<&'a wgpu::TextureView>,
2982 /// The render state the pass draws with, which folds each run's class to
2983 /// one it serves.
2984 state: EnginePipeline,
2985 /// The state's pipeline per paint class — at least every class the pass's
2986 /// segments name, folded.
2987 pipelines: &'a ClassPipelines,
2988 /// The pipeline variant's own groups 1-3.
2989 groups: &'a StripBindGroups,
2990 /// Group 0 for the pass's ordinary strip segments.
2991 resources: &'a wgpu::BindGroup,
2992 /// Group 0 per composite segment, in the order the segments name them.
2993 composites: &'a [wgpu::BindGroup],
2994 /// The externally bound texture each [`Segment::External`] slot names,
2995 /// indexed by slot — the frame's own
2996 /// [`ExternalRuns::keys`](crate::gpu::bindings::ExternalRuns::keys).
2997 /// Empty for a pass that draws no external texture, which is every pass of
2998 /// every frame that records no `SceneTexture`.
2999 external_keys: &'a [u64],
3000 instances: &'a wgpu::Buffer,
3001 /// The instance index every segment's own index is relative to.
3002 base: u32,
3003 segments: &'a [Segment],
3004 /// The class a [`Segment::Composite`] is drawn with —
3005 /// [`PaintClass::LayerComposite`], or Uber under the kill switch.
3006 composite_class: PaintClass,
3007}
3008
3009/// Records one pass: load the colour target, then draw each segment in order.
3010///
3011/// `timestamps` is the query pair the pass's GPU time is stamped into, `None`
3012/// for an untimed pass — which is every pass of every frame encoded without a
3013/// timestamp sink (see [`crate::diag`]). The pass is logged into `log` as it
3014/// is begun.
3015fn record_pass(
3016 encoder: &mut wgpu::CommandEncoder,
3017 plan: &PassPlan<'_>,
3018 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3019 log: &mut Vec<FramePass>,
3020) {
3021 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3022 label: Some(plan.pass.label()),
3023 color_attachments: &[Some(color_attachment(plan.view, plan.load))],
3024 depth_stencil_attachment: plan
3025 .depth
3026 .map(|view| depth_attachment(view, wgpu::LoadOp::Load)),
3027 timestamp_writes: timestamps,
3028 occlusion_query_set: None,
3029 multiview_mask: None,
3030 });
3031 log.push(plan.pass);
3032 draw_segments(&mut pass, plan);
3033}
3034
3035/// Draws `plan`'s segments, in order, into an already begun `pass`.
3036///
3037/// Two groups are re-bound per segment. Group 0, because a composite reads its
3038/// page through it while an ordinary strip reads the placeholder; and group 1,
3039/// because a run of instances sampling an externally bound texture needs that
3040/// texture bound where the atlas placeholder otherwise sits. Groups 2-3 are the
3041/// variant's own and never change within a plan. A plan with one segment —
3042/// every frame that records no layer and no external texture — sets them all
3043/// exactly once.
3044///
3045/// A segment naming a slot with no bind group behind it draws nothing rather
3046/// than drawing with whatever group 1 last held: the texture was unbound
3047/// between the frame's paint resolution and its recording, and a wrongly
3048/// sampled run is worse than a missing one.
3049///
3050/// Each segment is one paint-class run, and the pipeline is set only where
3051/// the class changes from the segment before — so a plan of one class (every
3052/// plan under the Uber kill switch) sets its pipeline exactly once. A segment
3053/// whose class has no resolved pipeline draws nothing, for the same reason; it
3054/// cannot occur, since the frame resolves every class its segments name before
3055/// recording.
3056///
3057/// Everything a plan draws with is set here, its first pipeline included, so
3058/// two plans drawn back to back into one pass — the opening's opaque strips
3059/// and then its round's — each start from their own state rather than from
3060/// whatever the plan before them left bound.
3061fn draw_segments(pass: &mut wgpu::RenderPass<'_>, plan: &PassPlan<'_>) {
3062 pass.set_vertex_buffer(0, plan.instances.slice(..));
3063
3064 let mut composite = 0_usize;
3065 let mut bound: Option<PaintClass> = None;
3066 for segment in plan.segments {
3067 // Claimed before the class lookup, so a composite whose pipeline is
3068 // missing still consumes its own group and the next one pairs up.
3069 let class = plan.state.fold_class(match *segment {
3070 Segment::Strips(_, _, class) | Segment::External(_, _, _, class) => class,
3071 Segment::Composite(..) | Segment::Backdrop(..) => plan.composite_class,
3072 });
3073 let (group, images, range) = match *segment {
3074 Segment::Strips(first, count, _) => {
3075 if count == 0 {
3076 continue;
3077 }
3078 (
3079 plan.resources,
3080 None,
3081 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3082 )
3083 }
3084 Segment::External(first, count, slot, _) => {
3085 if count == 0 {
3086 continue;
3087 }
3088 let Some(images) = plan
3089 .external_keys
3090 .get(slot as usize)
3091 .and_then(|key| plan.groups.externals.get(key))
3092 else {
3093 continue;
3094 };
3095 (
3096 plan.resources,
3097 Some(images),
3098 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3099 )
3100 }
3101 Segment::Composite(first, _) => {
3102 let Some(group) = plan.composites.get(composite) else {
3103 continue;
3104 };
3105 composite = composite.saturating_add(1);
3106 (
3107 group,
3108 None,
3109 GpuStrip::instance_range(plan.base.saturating_add(first), 1),
3110 )
3111 }
3112 // Its group is claimed before its count is read, like a
3113 // composite's, so the groups after it stay paired.
3114 Segment::Backdrop(first, count, _) => {
3115 let Some(group) = plan.composites.get(composite) else {
3116 continue;
3117 };
3118 composite = composite.saturating_add(1);
3119 if count == 0 {
3120 continue;
3121 }
3122 (
3123 group,
3124 None,
3125 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3126 )
3127 }
3128 };
3129 if bound != Some(class) {
3130 let Some(pipeline) = plan.pipelines.get(class) else {
3131 continue;
3132 };
3133 pass.set_pipeline(pipeline);
3134 bound = Some(class);
3135 }
3136 plan.groups.bind_with(pass, group, images);
3137 pass.draw(GpuStrip::vertex_range(), range);
3138 }
3139}
3140
3141/// The pass that opens the frame's own surface: the clear, the opaque strips,
3142/// and the first surface round that draws, in one render pass.
3143///
3144/// Merging is pixel-identical to recording the three apart, because nothing
3145/// between them differs but the pass boundary: the clear becomes the pass's
3146/// `LoadOp::Clear` on colour (and on depth, unless the caller stated its
3147/// attachment is already populated), and the two strip plans draw in the order
3148/// their passes ran, each through its own pipelines and bind groups, against
3149/// the one colour and depth attachment both already shared. What it saves is
3150/// the pass boundary itself: on a tile-based GPU every pass over the surface
3151/// is a whole-framebuffer load into tile memory and a store back out, and the
3152/// clear and the opaque pass were two of those that drew little or nothing.
3153///
3154/// Recorded exactly once per frame, by whichever comes first of the first
3155/// surface round that draws (which it then carries) and the first punch
3156/// (ahead of which it is recorded alone) — or at the end of the walk when
3157/// neither came. See [`EngineRenderer::set_pass_merge`] for the switch that
3158/// records the three apart again.
3159struct SurfaceOpening<'a> {
3160 view: &'a wgpu::TextureView,
3161 depth: Option<&'a wgpu::TextureView>,
3162 /// The frame's base colour, already in the target's alpha convention.
3163 clear: wgpu::Color,
3164 /// The depth clear, or a load when the caller's attachment is pre-cleared
3165 /// (see [`crate::gpu::depth`]).
3166 depth_load: wgpu::LoadOp<f32>,
3167 /// The opaque strips, when the frame has any to draw.
3168 opaque: Option<PassPlan<'a>>,
3169}
3170
3171impl SurfaceOpening<'_> {
3172 /// Records the opening, drawing `round`'s segments after the opaque strips
3173 /// when a surface round is folded into it.
3174 fn record(
3175 &self,
3176 encoder: &mut wgpu::CommandEncoder,
3177 round: Option<&PassPlan<'_>>,
3178 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3179 log: &mut Vec<FramePass>,
3180 ) {
3181 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3182 label: Some(FramePass::Opening.label()),
3183 color_attachments: &[Some(color_attachment(
3184 self.view,
3185 wgpu::LoadOp::Clear(self.clear),
3186 ))],
3187 depth_stencil_attachment: self
3188 .depth
3189 .map(|view| depth_attachment(view, self.depth_load)),
3190 timestamp_writes: timestamps,
3191 occlusion_query_set: None,
3192 multiview_mask: None,
3193 });
3194 log.push(FramePass::Opening);
3195 if let Some(opaque) = self.opaque.as_ref() {
3196 draw_segments(&mut pass, opaque);
3197 }
3198 if let Some(round) = round {
3199 draw_segments(&mut pass, round);
3200 }
3201 }
3202}
3203
3204/// Records the surface opening as a pass of its own if it is still owed, and
3205/// marks it taken.
3206fn record_owed_opening(
3207 opening: &mut Option<SurfaceOpening<'_>>,
3208 encoder: &mut wgpu::CommandEncoder,
3209 timestamps: FrameTimestamps<'_>,
3210 log: &mut Vec<FramePass>,
3211) {
3212 if let Some(opening) = opening.take() {
3213 opening.record(encoder, None, timestamps.writes(EngineSpan::Main), log);
3214 }
3215}
3216
3217/// Everything the frame's hole-punch passes are recorded with, resolved once
3218/// before the round walk begins.
3219///
3220/// A frame can now record several: the punches are issued at the painter-order
3221/// positions they were hoisted from, so a surface round carrying two of them is
3222/// cut twice and each cut erases through this same pipeline and these same
3223/// groups (see [`crate::compile::clear`]). Resolving them once is what keeps
3224/// that from becoming a per-cut lookup, and holding the borrows in one value is
3225/// what keeps them out of the page pool's way — the walk holds that mutably
3226/// throughout.
3227struct PunchPass<'a> {
3228 view: &'a wgpu::TextureView,
3229 depth: Option<&'a wgpu::TextureView>,
3230 /// Which destination-out state the punch draws with.
3231 state: EnginePipeline,
3232 /// That state's pipelines, holding at least [`Self::class`].
3233 pipelines: &'a ClassPipelines,
3234 /// The class every punch run is drawn with: the solid program, or Uber
3235 /// under the kill switch.
3236 class: PaintClass,
3237 /// The destination-out variant's own groups 1-3.
3238 groups: &'a StripBindGroups,
3239 instances: &'a wgpu::Buffer,
3240 /// The instance index the punch's own `(first, count)` is relative to — the
3241 /// alpha region's start, the same one every round's segments use.
3242 base: u32,
3243}
3244
3245impl PunchPass<'_> {
3246 /// Records `punch`'s instances as one destination-out pass, or nothing at
3247 /// all when the cut issued none — which is every cut of every frame that
3248 /// records no `ClearRect`. A pass that is not recorded takes no query pair.
3249 fn record(
3250 &self,
3251 encoder: &mut wgpu::CommandEncoder,
3252 punch: (u32, u32),
3253 timestamps: FrameTimestamps<'_>,
3254 log: &mut Vec<FramePass>,
3255 ) {
3256 let (first, count) = punch;
3257 if count == 0 {
3258 return;
3259 }
3260 record_pass(
3261 encoder,
3262 &PassPlan {
3263 pass: FramePass::Punch,
3264 view: self.view,
3265 load: wgpu::LoadOp::Load,
3266 depth: self.depth,
3267 state: self.state,
3268 pipelines: self.pipelines,
3269 groups: self.groups,
3270 resources: &self.groups.resources,
3271 composites: &[],
3272 // A punch erases with a solid source; it samples nothing.
3273 external_keys: &[],
3274 instances: self.instances,
3275 base: self.base,
3276 // Every punch instance is the same solid source, so the run
3277 // is one class whole — no split to plan.
3278 segments: &[Segment::Strips(first, count, self.class)],
3279 composite_class: self.class,
3280 },
3281 timestamps.writes(EngineSpan::Main),
3282 log,
3283 );
3284 }
3285}
3286
3287/// Hands back the page groups a finished round consumed, and parks the page it
3288/// wrote in its own group.
3289///
3290/// The same bookkeeping after every round, strip and filter alike: a page is
3291/// free the moment the pass that sampled it ends, which is what bounds a chain
3292/// of any depth — and a filter layer's own pair of pages — to the two ping-pong
3293/// groups, and every shape this scheduler serves to
3294/// [`MAX_LIVE_PAGES`](crate::schedule::MAX_LIVE_PAGES) live intermediates.
3295///
3296/// A round that is not continuing a page of its own takes a *fresh* texture out
3297/// of the pool rather than the group's current occupant, and the occupant it
3298/// displaces goes back here. That is what keeps a filter pass from ever holding
3299/// one texture as both its attachment and its source: a filter round never
3300/// continues a page — it clears — so what it writes is always a different
3301/// texture from the one the pass before it wrote and this pass reads.
3302fn settle_pages(
3303 targets: &mut IntermediateTargets,
3304 live: &mut [Option<PooledTexture>; MAX_LIVE_PAGES],
3305 released: [bool; MAX_LIVE_PAGES],
3306 own: Option<(PageParity, PooledTexture)>,
3307) {
3308 for (index, slot) in live.iter_mut().enumerate() {
3309 if released.get(index).copied().unwrap_or(false)
3310 && let Some(page) = slot.take()
3311 {
3312 targets.release(page);
3313 }
3314 }
3315 if let Some((parity, pooled)) = own
3316 && let Some(previous) = live[parity.index()].replace(pooled)
3317 {
3318 targets.release(previous);
3319 }
3320}
3321
3322/// The GPU resources a frame's [filter](crate::filters) rounds are executed
3323/// with, beyond the two pooled pages they ping-pong between.
3324///
3325/// Three of them, and each is the engine's only one of its kind: the
3326/// filter-data texture holding every filter in the frame's 48-byte parameter
3327/// block, the bilinear sampler
3328/// ([`filter_sampler`](crate::gpu::targets::filter_sampler)) the blur kernels
3329/// read their source page through, and the instance buffer one quad per pass is
3330/// drawn from.
3331///
3332/// Public because this *is* executing a filter pass — the renderer holds one
3333/// and drives it over the pages the scheduler named, and `tests/filters.rs`
3334/// drives the same type over pages of its own on real hardware. That second
3335/// caller is not a convenience: `frust_scene` carries no *layer*-filter
3336/// command, so a filter layer cannot reach [`EngineRenderer::encode`] through a
3337/// `Scene` at all, and driving this type directly is how the drop shadow's
3338/// WGSL is exercised on a device. The blur's is also reached through `encode`
3339/// itself, by a backdrop blur's pass rounds.
3340#[derive(Debug)]
3341pub struct FilterResources {
3342 /// Every filter in the frame's parameter block, back to back.
3343 data: ResourceTexture,
3344 /// Group 0, naming [`Self::data`]'s view. Rebuilt whenever that texture is,
3345 /// and valid for the renderer's whole life otherwise: a filter pipeline's
3346 /// description does not depend on the frame's target format, so there is
3347 /// only ever one layout to have derived it from.
3348 data_group: Option<wgpu::BindGroup>,
3349 sampler: wgpu::Sampler,
3350 instances: Option<wgpu::Buffer>,
3351 instance_capacity: u64,
3352 /// Reusable staging for the parameter-block upload, padded to the
3353 /// texture's own footprint.
3354 staging: Vec<u8>,
3355}
3356
3357impl FilterResources {
3358 /// A renderer's filter resources, with nothing uploaded yet.
3359 #[must_use]
3360 pub fn new(device: &wgpu::Device) -> Self {
3361 Self {
3362 data: ResourceTexture::new(
3363 device,
3364 &filter_data_texture_descriptor(gpu::MIN_RESOURCE_TEXTURE_HEIGHT),
3365 ),
3366 data_group: None,
3367 sampler: filter_sampler(device),
3368 instances: None,
3369 instance_capacity: 0,
3370 staging: Vec::new(),
3371 }
3372 }
3373
3374 /// Uploads this frame's parameter `blocks` and reserves room for `passes`
3375 /// pass instances, growing either resource if the frame outgrew it.
3376 ///
3377 /// `pipeline` is the one [`EnginePipeline::Filter`] describes; it is needed
3378 /// because wgpu derives a pipeline's bind-group layouts from its shader
3379 /// module, so the group naming the filter-data texture can only be built
3380 /// against the pipeline that will bind it.
3381 ///
3382 /// A block count no filter-data texture could hold (see
3383 /// [`filter_data_texture_height`]) leaves the group unbuilt, which leaves
3384 /// every filter pass of the frame issuing no draw — the page is still
3385 /// cleared, so the layer composites as transparent rather than as whatever
3386 /// its page last held. Unreachable in practice, and "draw less, never
3387 /// wrong" when it is not.
3388 pub fn prepare(
3389 &mut self,
3390 device: &wgpu::Device,
3391 queue: &wgpu::Queue,
3392 pipeline: &wgpu::RenderPipeline,
3393 blocks: &[GpuFilterData],
3394 passes: u32,
3395 ) {
3396 let Some(height) = filter_data_texture_height(blocks.len()) else {
3397 self.data_group = None;
3398 return;
3399 };
3400 if height > self.data.height {
3401 self.data = ResourceTexture::new(device, &filter_data_texture_descriptor(height));
3402 self.data_group = None;
3403 }
3404 if self.data_group.is_none() {
3405 self.data_group = Some(device.create_bind_group(&wgpu::BindGroupDescriptor {
3406 label: Some("frust-engine filter data"),
3407 layout: &pipeline.get_bind_group_layout(0),
3408 entries: &[wgpu::BindGroupEntry {
3409 binding: 0,
3410 resource: wgpu::BindingResource::TextureView(&self.data.view),
3411 }],
3412 }));
3413 }
3414
3415 // A queue write covers the whole texture extent, so the blocks are
3416 // padded out to its footprint; the trailing texels are never addressed,
3417 // because a pass names its own block by a texel offset the host handed
3418 // it.
3419 let footprint = gpu::resource_texture_bytes(self.data.width, self.data.height);
3420 self.staging.clear();
3421 self.staging
3422 .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
3423 let bytes: &[u8] = bytemuck::cast_slice(blocks);
3424 if let Some(head) = self.staging.get_mut(..bytes.len()) {
3425 head.copy_from_slice(bytes);
3426 }
3427 queue.write_texture(
3428 self.data.copy_target(),
3429 &self.staging,
3430 resource_layout(
3431 gpu::resource_bytes_per_row(self.data.width),
3432 self.data.height,
3433 ),
3434 self.data.extent(),
3435 );
3436
3437 self.reserve_instances(device, passes);
3438 }
3439
3440 /// Grows the instance buffer if this frame's pass count outgrew it.
3441 fn reserve_instances(&mut self, device: &wgpu::Device, passes: u32) {
3442 let stride = size_of::<FilterInstanceData>() as u64;
3443 let required = u64::from(passes).saturating_mul(stride).max(stride);
3444 if self.instances.is_some() && self.instance_capacity >= required {
3445 return;
3446 }
3447 let capacity = required
3448 .checked_next_power_of_two()
3449 .unwrap_or(required)
3450 .max(MIN_FILTER_INSTANCE_CAPACITY.saturating_mul(stride));
3451 self.instance_capacity = capacity;
3452 self.instances = Some(device.create_buffer(&wgpu::BufferDescriptor {
3453 label: Some("frust-engine filter instances"),
3454 size: capacity,
3455 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
3456 mapped_at_creation: false,
3457 }));
3458 }
3459
3460 /// Writes one pass's instance into slot `index` of the instance buffer.
3461 ///
3462 /// Separate from [`Self::prepare`] because a pass's `dest_texture_size` is
3463 /// the extent the texture pool *quantized* its destination page up to, and
3464 /// NDC is computed against the attachment's real extent — only the round
3465 /// that acquires the page knows it. A queue write issued while the frame is
3466 /// being recorded still lands ahead of the command buffers it is submitted
3467 /// with, which is the same ordering a page's viewport uniform already
3468 /// relies on.
3469 ///
3470 /// A slot past the reserved capacity is dropped rather than written, which
3471 /// leaves that pass drawing whatever the slot last held; unreachable, since
3472 /// `prepare` reserved one slot per pass of this very frame.
3473 pub fn write_instance(&self, queue: &wgpu::Queue, index: u32, instance: &FilterInstanceData) {
3474 let Some(buffer) = self.instances.as_ref() else {
3475 return;
3476 };
3477 let stride = size_of::<FilterInstanceData>() as u64;
3478 let offset = u64::from(index).saturating_mul(stride);
3479 if offset.saturating_add(stride) > self.instance_capacity {
3480 return;
3481 }
3482 queue.write_buffer(buffer, offset, bytemuck::bytes_of(instance));
3483 }
3484
3485 /// Records one filter pass into `encoder`: clear the destination page, then
3486 /// draw the one instanced quad that filters `plan`'s source into it.
3487 ///
3488 /// The clear is unconditional and the draw is not. A filter pass writes only
3489 /// the region its step names — a decimated one a quarter of the texels the
3490 /// pass before it did — and the kernels sample past that region without
3491 /// bounds checks, so whatever surrounds it has to be transparent rather than
3492 /// a previous holder's pixels. That has to hold even on the path where the
3493 /// pass itself cannot be issued, or the layer's composite would sample the
3494 /// page's previous tenant instead of nothing.
3495 ///
3496 /// The pass is opened and closed here, on the caller's own encoder: a filter
3497 /// round is not an exception to [`EngineRenderer::encode`]'s contract.
3498 pub fn record_pass(
3499 &self,
3500 device: &wgpu::Device,
3501 encoder: &mut wgpu::CommandEncoder,
3502 plan: &FilterPassPlan<'_>,
3503 ) {
3504 self.record_pass_timed(device, encoder, plan, None);
3505 }
3506
3507 /// [`Self::record_pass`], stamping the pass's GPU time into `timestamps`.
3508 ///
3509 /// A separate method rather than a field on [`FilterPassPlan`]: the plan is
3510 /// public and built by struct literal outside this crate, so a new required
3511 /// field would break every one of those call sites to serve a diagnostic
3512 /// they do not use. `None` records exactly what [`Self::record_pass`] does.
3513 pub fn record_pass_timed(
3514 &self,
3515 device: &wgpu::Device,
3516 encoder: &mut wgpu::CommandEncoder,
3517 plan: &FilterPassPlan<'_>,
3518 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3519 ) {
3520 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3521 label: Some(plan.label),
3522 color_attachments: &[Some(color_attachment(
3523 plan.dest,
3524 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
3525 ))],
3526 // A pooled page carries no depth attachment, which is also why the
3527 // filter pipeline declares no depth state.
3528 depth_stencil_attachment: None,
3529 timestamp_writes: timestamps,
3530 occlusion_query_set: None,
3531 multiview_mask: None,
3532 });
3533
3534 let (Some(data_group), Some(instances)) =
3535 (self.data_group.as_ref(), self.instances.as_ref())
3536 else {
3537 return;
3538 };
3539
3540 let source = device.create_bind_group(&wgpu::BindGroupDescriptor {
3541 label: Some("frust-engine filter source"),
3542 layout: &plan.pipeline.get_bind_group_layout(1),
3543 entries: &[
3544 wgpu::BindGroupEntry {
3545 binding: 0,
3546 resource: wgpu::BindingResource::TextureView(plan.source),
3547 },
3548 wgpu::BindGroupEntry {
3549 binding: 1,
3550 resource: wgpu::BindingResource::Sampler(&self.sampler),
3551 },
3552 ],
3553 });
3554
3555 pass.set_pipeline(plan.pipeline);
3556 pass.set_vertex_buffer(0, instances.slice(..));
3557 pass.set_bind_group(0, data_group, &[]);
3558 pass.set_bind_group(1, &source, &[]);
3559 pass.draw(
3560 0..FILTER_QUAD_VERTICES,
3561 plan.instance..plan.instance.saturating_add(1),
3562 );
3563 }
3564}
3565
3566/// One filter pass's full recording state.
3567///
3568/// The two pages are named as views rather than as parities because
3569/// [`FilterResources`] holds no pool: which texture each group is holding is the
3570/// caller's bookkeeping, whether that caller is the frame path or a test.
3571#[derive(Debug)]
3572pub struct FilterPassPlan<'a> {
3573 /// Label for captures and validation messages.
3574 pub label: &'a str,
3575 /// The pipeline [`EnginePipeline::Filter`] describes.
3576 pub pipeline: &'a wgpu::RenderPipeline,
3577 /// The page this pass writes. Cleared to transparent before it is written.
3578 pub dest: &'a wgpu::TextureView,
3579 /// The page this pass reads — the one the pass before it wrote.
3580 pub source: &'a wgpu::TextureView,
3581 /// Which instance of the filter instance buffer this pass draws.
3582 pub instance: u32,
3583}
3584
3585/// A colour attachment over `view` with `load`, keeping what it stores.
3586fn color_attachment(
3587 view: &wgpu::TextureView,
3588 load: wgpu::LoadOp<wgpu::Color>,
3589) -> wgpu::RenderPassColorAttachment<'_> {
3590 wgpu::RenderPassColorAttachment {
3591 view,
3592 depth_slice: None,
3593 resolve_target: None,
3594 ops: wgpu::Operations {
3595 load,
3596 store: wgpu::StoreOp::Store,
3597 },
3598 }
3599}
3600
3601/// A depth attachment over `view` with `load`, keeping what it stores so the
3602/// pass after it tests against the same buffer.
3603fn depth_attachment(
3604 view: &wgpu::TextureView,
3605 load: wgpu::LoadOp<f32>,
3606) -> wgpu::RenderPassDepthStencilAttachment<'_> {
3607 wgpu::RenderPassDepthStencilAttachment {
3608 view,
3609 depth_ops: Some(wgpu::Operations {
3610 load,
3611 store: wgpu::StoreOp::Store,
3612 }),
3613 stencil_ops: None,
3614 }
3615}
3616
3617/// The frame's base colour as a clear value.
3618///
3619/// The strip pipelines blend premultiplied and the shader emits premultiplied
3620/// colour, so a [`OutputAlpha::Premultiplied`] target's clear value has to be
3621/// premultiplied too — otherwise the background sits in a different alpha
3622/// convention from everything drawn over it. `Straight` is honoured here and
3623/// only here: the pipelines themselves are fixed premultiplied, so a straight
3624/// target's *drawn* content is premultiplied regardless.
3625fn clear_color(base_color: Color, output: OutputAlpha) -> wgpu::Color {
3626 let components = match output {
3627 OutputAlpha::Premultiplied => base_color.premultiply().components,
3628 OutputAlpha::Straight => base_color.components,
3629 };
3630 wgpu::Color {
3631 r: f64::from(components[0]),
3632 g: f64::from(components[1]),
3633 b: f64::from(components[2]),
3634 a: f64::from(components[3]),
3635 }
3636}
3637
3638/// Whether a frame cleared to `base_color` presents opaquely, and so whether
3639/// its alpha channel carries anything a hole punch could reveal.
3640///
3641/// The engine's only statement about the target's own alpha handling:
3642/// [`EngineTarget`] describes how the alpha it produces is *interpreted*
3643/// (premultiplied or straight), never whether it is used at all. A base colour
3644/// at full alpha seals every pixel of the surface, which is exactly the
3645/// presentation [`crate::compile::clear`]'s contract says to skip the
3646/// destination-out pass on.
3647fn is_opaque(base_color: Color) -> bool {
3648 base_color.components[3] >= 1.0
3649}
3650
3651/// The target extent on the `u16` device grid the strip pipeline addresses.
3652fn grid_size(width: u32, height: u32) -> Result<(u16, u16), EngineError> {
3653 let width = u16::try_from(width).map_err(|_| EngineError::TargetTooLarge)?;
3654 let height = u16::try_from(height).map_err(|_| EngineError::TargetTooLarge)?;
3655 Ok((width, height))
3656}
3657
3658/// One unit of drawing inside a round's pass, in execution order.
3659///
3660/// Instance indices are relative to the alpha region of the shared instance
3661/// buffer, which is why [`PassPlan::base`] exists rather than the indices being
3662/// absolute: the opaque region is laid out first and its own segment addresses
3663/// from zero.
3664///
3665/// Every strip segment is also one paint-class run: it names the
3666/// [`PaintClass`] whose program draws it, and a pass sets that class's
3667/// pipeline where the class changes (see the module doc's *Paint-class runs*).
3668#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3669enum Segment {
3670 /// Ordinary strip instances, as `(first, count, class)`, drawn with the
3671 /// frame's own atlas binding.
3672 Strips(u32, u32, PaintClass),
3673 /// Strip instances sampling an externally bound texture, as
3674 /// `(first, count, slot, class)` — a *run*, in the sense
3675 /// [`crate::gpu::bindings::ExternalRuns`] gives the word: the maximal span
3676 /// of consecutive instances that share one texture, drawn with that
3677 /// texture's own group 1 bound.
3678 External(u32, u32, u32, PaintClass),
3679 /// One composite quad at `first`, sampling the finished page it names.
3680 Composite(u32, CompositeSource),
3681 /// A backdrop blur's own strip instances, as `(first, count, parity)`:
3682 /// its rectangle's coverage, each instance sampling the blurred page in
3683 /// `parity`'s group at its own position — drawn through the composite
3684 /// class with that page bound as group 0's layer input.
3685 Backdrop(u32, u32, PageParity),
3686}
3687
3688/// The page a [`Segment::Composite`] samples.
3689#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3690enum CompositeSource {
3691 /// The live pooled page in this group.
3692 Page(PageParity),
3693 /// The retained page of this retained bracket (see
3694 /// [`crate::schedule::RetainedSource`]).
3695 Retained(u32),
3696}
3697
3698/// The retained page one round renders a body into.
3699#[derive(Debug, Clone, Copy)]
3700struct RetainedRound {
3701 /// The retained bracket the page belongs to.
3702 bracket: u32,
3703 /// Whether an earlier round of the same body already rendered into the
3704 /// page this frame, so this one loads it rather than clearing it.
3705 continued: bool,
3706}
3707
3708/// One scheduled round, resolved to the instances and target it draws with.
3709///
3710/// One *plan* rather than one scheduled round: a surface round carrying a hole
3711/// punch is cut into a plan per span between its punches, so the punch pass can
3712/// be recorded at the painter-order position it was hoisted from (see
3713/// [`Scratch::cut_for_punches`]).
3714#[derive(Debug, Clone)]
3715struct RoundPlan {
3716 /// The page this round renders into, or `None` for the frame's own target
3717 /// and for a retained page.
3718 page: Option<PagePlan>,
3719 /// The retained page this round renders a body into, if it does — an
3720 /// off-screen round like a page one, into the store's texture.
3721 retained: Option<RetainedRound>,
3722 /// This round's slice of [`Scratch::segments`], in execution order.
3723 segments: Range<usize>,
3724 /// Page groups this round consumed, indexed by
3725 /// [`PageParity::index`]; each returns to the pool once the pass ends.
3726 released: [bool; MAX_LIVE_PAGES],
3727 /// The filter pass this round runs, on a filter round — which draws no
3728 /// strip and composites nothing, so its `segments` range is empty.
3729 filter: Option<FilterPlan>,
3730 /// The region of the frame's own target this round copies into its page,
3731 /// on a region-copy round — which, like a filter round, draws no strip.
3732 copy: Option<RectU16>,
3733 /// The hole punches to erase with once this plan's own pass has been
3734 /// recorded, as `(first, count)` into the alpha region — the cut this plan
3735 /// ends at.
3736 ///
3737 /// Only a *surface* plan ever carries one: a punch recorded inside an
3738 /// isolated layer was hoisted out of it to the frame root (see
3739 /// [`crate::compile::clear`]), so a page round is never cut and a filter
3740 /// round — which draws nothing of the frame's own — never is either.
3741 punch: (u32, u32),
3742}
3743
3744/// One filter pass, resolved to the instance slot it draws and the page group
3745/// it reads.
3746#[derive(Debug, Clone, Copy)]
3747struct FilterPlan {
3748 /// Which pass of the filter's sequence this is, and at what extents.
3749 step: FilterStep,
3750 /// The group holding the page this pass reads.
3751 source: PageParity,
3752 /// The texel this filter's parameter block starts at in the filter-data
3753 /// texture — where the fragment stage reads its kernel from.
3754 data_offset: u32,
3755 /// The filter layer's own extent, before any decimation; it bounds the
3756 /// transparent border a decimated pass overdraws.
3757 original: SizeU16,
3758 /// This pass's slot in the frame's filter instance buffer.
3759 instance: u32,
3760}
3761
3762/// The pooled page one round renders into.
3763#[derive(Debug, Clone, Copy)]
3764struct PagePlan {
3765 parity: PageParity,
3766 size: PageSize,
3767 /// Whether an earlier round of the same layer already rendered into this
3768 /// page, so this round takes that texture back rather than acquiring a
3769 /// fresh one, and loads it rather than clearing it.
3770 continued: bool,
3771}
3772
3773/// The frame's instance buffers and pass plan, retained so a steady-state frame
3774/// refills them rather than reallocating them.
3775///
3776/// The segments of every round live in one flat vector rather than a vector per
3777/// round, so a frame with layers costs no allocation once the first one has
3778/// grown these buffers.
3779#[derive(Debug, Default)]
3780struct Scratch {
3781 /// Fully-covered spans of the opaque draws targeting the frame's own
3782 /// surface, drawn unblended with depth write in one pass ahead of every
3783 /// round — including the surface's own, of which a frame may have several.
3784 opaque: Vec<GpuStrip>,
3785 /// Everything else, drawn premultiplied-blended in painter order and laid
3786 /// out round by round in execution order.
3787 alpha: Vec<GpuStrip>,
3788 /// Every round's segments, back to back; [`RoundPlan::segments`] slices it.
3789 segments: Vec<Segment>,
3790 /// The frame's rounds, each layer's before the round that composites it and
3791 /// the surface's last round last.
3792 rounds: Vec<RoundPlan>,
3793 /// The instances of the punches no cut reached — those recorded past every
3794 /// op of the frame — as `(first, count)` into the alpha region.
3795 ///
3796 /// Their pass is the last thing the frame records, which is where a
3797 /// `ClearRect` recorded last belongs in painter order anyway. Punches the
3798 /// walk *did* cut at are held on their own [`RoundPlan::punch`] instead.
3799 ///
3800 /// No round's segments name a punch instance, wherever it sits in the
3801 /// buffer, so dropping the punch passes leaves the frame exactly as the
3802 /// display list would read without the clear.
3803 punch: (u32, u32),
3804 /// The deepest painter's-order index inside each recorded layer, which is
3805 /// the depth its composite carries. Filled as the rounds are walked, which
3806 /// is sound because a layer's own round always precedes the round that
3807 /// composites it.
3808 layer_depth: Vec<u32>,
3809 /// One parameter block per *filter layer* of the frame, in the order the
3810 /// rounds first named them — which is the order the texel offsets in
3811 /// [`FilterPlan::data_offset`] were taken from.
3812 filter_blocks: Vec<GpuFilterData>,
3813 /// The layers `filter_blocks` holds, parallel to it, so a filter's second
3814 /// and later passes reuse the block its first one packed rather than
3815 /// repacking one per pass.
3816 filter_layers: Vec<u32>,
3817 /// How many filter passes this frame runs, and so how many instances its
3818 /// filter instance buffer has to hold.
3819 filter_passes: u32,
3820 /// How many region-copy rounds this frame runs — one per backdrop blur
3821 /// the scheduler served.
3822 copies: u32,
3823 /// How segments are cut into paint-class runs this frame — copied in from
3824 /// the renderer ahead of [`Self::build`].
3825 routing: ClassRouting,
3826 /// The opaque region's own segments: `(0, opaque.len())` cut into class
3827 /// runs, drawn by the one depth-writing pass.
3828 opaque_segments: Vec<Segment>,
3829 /// The classes the opaque pass's runs name.
3830 opaque_classes: ClassSet,
3831 /// The classes the frame's surface rounds name — composites included.
3832 surface_classes: ClassSet,
3833 /// The classes the frame's page rounds name — composites included.
3834 page_classes: ClassSet,
3835 /// Whether the round being planned renders into a page, which decides the
3836 /// class set [`Self::push_run`] records a run's class in.
3837 planning_page: bool,
3838 /// Reusable output of [`merge_class_runs`].
3839 class_runs: Vec<ClassRun>,
3840 /// Where the paint class changes in [`Self::alpha`], as `(first instance,
3841 /// class)` in ascending order — one entry per draw whose class differs
3842 /// from the draw before it. Every instance of a draw shares its paint, so
3843 /// these are the only places a class run can start, and a segment is split
3844 /// by walking them instead of re-reading every instance.
3845 alpha_marks: Vec<(u32, PaintClass)>,
3846 /// The same marks for [`Self::opaque`].
3847 opaque_marks: Vec<(u32, PaintClass)>,
3848}
3849
3850impl Scratch {
3851 /// Turns a scheduled frame into instances and a per-round pass plan,
3852 /// routing each instance to the pass that can draw it.
3853 ///
3854 /// A draw's anti-aliased spans always land in the blended buffer: partial
3855 /// coverage is not opaque however opaque the paint is. Its fully-covered
3856 /// spans land in the opaque buffer only when the paint is opaque, depth is
3857 /// available to re-establish their ordering against the blended ones, *and*
3858 /// the draw targets the frame's own surface — a page has no depth
3859 /// attachment, so a page round is a plain painter's-algorithm walk.
3860 ///
3861 /// `backdrops` is the frame's served backdrop blurs — `frame.backdrops`,
3862 /// or none at all on a target no region can be read back from. A draw the
3863 /// scheduler placed at a backdrop's own depth is that backdrop's rectangle
3864 /// and samples its blurred page instead of its recorded paint; and no draw
3865 /// at or past the first backdrop's depth takes the depth-writing pass,
3866 /// which is recorded ahead of every surface round — so ahead of the copy
3867 /// that reads the target back, which must not see what is painted over the
3868 /// backdrop.
3869 fn build(
3870 &mut self,
3871 frame: &CompiledFrame,
3872 rounds: &[Round],
3873 depth_active: bool,
3874 punches: bool,
3875 paint_slots: &[Option<ResolvedPaint>],
3876 backdrops: &[Backdrop],
3877 ) {
3878 self.opaque.clear();
3879 self.alpha.clear();
3880 self.segments.clear();
3881 self.rounds.clear();
3882 self.punch = (0, 0);
3883 self.layer_depth.clear();
3884 self.layer_depth.resize(frame.recorder.layers.len(), 0);
3885 // A `DrawRetained`'s layer holds no draw, so its composite's depth is
3886 // the slot the command itself was recorded at rather than the deepest
3887 // index inside it.
3888 for bracket in &frame.retained {
3889 if let RetainedKind::Draw { depth, .. } = bracket.kind
3890 && let Some(slot) = self.layer_depth.get_mut(bracket.layer as usize)
3891 {
3892 *slot = depth;
3893 }
3894 }
3895 self.filter_blocks.clear();
3896 self.filter_layers.clear();
3897 self.filter_passes = 0;
3898 self.copies = 0;
3899 self.opaque_segments.clear();
3900 self.alpha_marks.clear();
3901 self.opaque_marks.clear();
3902 self.opaque_classes = ClassSet::EMPTY;
3903 self.surface_classes = ClassSet::EMPTY;
3904 self.page_classes = ClassSet::EMPTY;
3905
3906 let draws = frame.draws();
3907 let strips = frame.strip_buf();
3908 // The punches still to be issued, in the order the compiler hoisted
3909 // them — which is depth order, since each took its own index from the
3910 // frame's monotonic painter-order counter. Empty on a frame with no
3911 // clear and on a target that disregards alpha (`punches`), and then
3912 // nothing below ever cuts: a non-punching frame is planned exactly as
3913 // it always was.
3914 let mut pending: &[ClearPunch] = if punches { &frame.clears } else { &[] };
3915 // Every draw before the first backdrop may take the depth-writing pass;
3916 // none from it on. `u32::MAX` on a frame with no backdrop, which is
3917 // then routed exactly as it always was.
3918 let opaque_limit = backdrops
3919 .first()
3920 .map_or(u32::MAX, |backdrop| backdrop.depth);
3921
3922 for round in rounds {
3923 let page = round.page();
3924 let retained = round.retained();
3925 let offscreen = page.is_some() || retained.is_some();
3926
3927 // A region-copy round draws nothing: it is planned as its page and
3928 // the region it copies. Any punch recorded ahead of the backdrop
3929 // it serves is erased first, as a cut of its own, so the copy reads
3930 // the target the display list says lies beneath the backdrop.
3931 if let Some(copy) = round.region_copy() {
3932 if let Some(page) = page {
3933 if !pending.is_empty()
3934 && let Some(backdrop) = backdrops
3935 .iter()
3936 .find(|backdrop| backdrop.layer == page.layer)
3937 {
3938 let mut first_segment = self.segments.len();
3939 let mut run_start = self.alpha.len() as u32;
3940 let mut run_external = None;
3941 self.cut_for_punches(
3942 &mut pending,
3943 strips,
3944 backdrop.depth,
3945 &mut first_segment,
3946 &mut run_start,
3947 &mut run_external,
3948 );
3949 }
3950 self.copies = self.copies.saturating_add(1);
3951 self.rounds.push(RoundPlan {
3952 page: Some(PagePlan {
3953 parity: page.parity,
3954 size: page.size,
3955 continued: false,
3956 }),
3957 retained: None,
3958 segments: self.segments.len()..self.segments.len(),
3959 released: released_groups(&round.released),
3960 filter: None,
3961 copy: Some(copy.source),
3962 punch: (0, 0),
3963 });
3964 }
3965 continue;
3966 }
3967
3968 let filter = self.plan_filter(frame, round, backdrops);
3969 // A page holds its layer — or one column band of it — at the page's
3970 // own origin, so every instance of the round is shifted into that
3971 // column and clipped to it (see `PageWindow`). A retained page holds
3972 // its body the same way.
3973 let window = match (page, retained) {
3974 (Some(page), _) => PageWindow::of(page),
3975 (None, Some(retained)) => PageWindow::of_retained(retained),
3976 (None, None) => PageWindow::ROOT,
3977 };
3978 let split_opaque = !offscreen && depth_active;
3979 self.planning_page = offscreen;
3980 // Only the frame's own target is punched — a punch inside an
3981 // isolated layer was hoisted out of it — so only a surface round is
3982 // ever cut.
3983 let cuts = !offscreen;
3984
3985 let mut first_segment = self.segments.len();
3986 let mut deepest = 0_u32;
3987 let mut run_start = self.alpha.len() as u32;
3988 // The external texture the open run is drawn with, `None` while it
3989 // is drawn with the frame's own atlas binding. A draw that names a
3990 // different one closes the run: there is a single external binding
3991 // to set (see [`crate::gpu::bindings`]).
3992 let mut run_external: Option<u32> = None;
3993
3994 for op in &round.ops {
3995 match op {
3996 RoundOp::Draws(range) => {
3997 let batch = draws
3998 .get(range.start as usize..range.end as usize)
3999 .unwrap_or(&[]);
4000 for draw in batch {
4001 deepest = deepest.max(draw.depth);
4002 // Ahead of the paint lookup rather than after it, so
4003 // the cut follows the order the display list was
4004 // recorded in rather than the subset of it that
4005 // survived lowering.
4006 if cuts && !pending.is_empty() {
4007 self.cut_for_punches(
4008 &mut pending,
4009 strips,
4010 draw.depth,
4011 &mut first_segment,
4012 &mut run_start,
4013 &mut run_external,
4014 );
4015 }
4016 // A backdrop's rectangle: drawn from the blurred
4017 // page this round samples rather than from the
4018 // transparent paint it was recorded with.
4019 if let Ok(at) = backdrops
4020 .binary_search_by_key(&draw.depth, |backdrop| backdrop.depth)
4021 {
4022 if let Some(backdrop) = backdrops.get(at)
4023 && let Some(sample) = round
4024 .samples
4025 .iter()
4026 .find(|sample| sample.layer == backdrop.layer)
4027 && let Some(run) = strips.get(draw.strip_range.clone())
4028 {
4029 let end = self.alpha.len() as u32;
4030 self.push_run(run_start, end, run_external);
4031 run_external = None;
4032 self.push_backdrop(
4033 backdrop,
4034 sample.parity,
4035 draw.depth,
4036 run,
4037 window,
4038 );
4039 run_start = self.alpha.len() as u32;
4040 }
4041 continue;
4042 }
4043 let Some(paint) = pack_paint(&draw.paint, draw.depth, paint_slots)
4044 else {
4045 continue;
4046 };
4047 if paint.glyph_quad {
4048 // One rect instance, from the quad alone: the
4049 // draw's strips are its rectangle's coverage,
4050 // which the instance covers whole.
4051 let Some(quad) = glyph_quad(&frame.glyph_quads, draw.depth) else {
4052 continue;
4053 };
4054 if paint.external != run_external {
4055 let end = self.alpha.len() as u32;
4056 self.push_run(run_start, end, run_external);
4057 run_start = end;
4058 run_external = paint.external;
4059 }
4060 mark_class(
4061 &mut self.alpha_marks,
4062 self.alpha.len() as u32,
4063 paint.class,
4064 );
4065 if let Some(instance) = window.place_quad(quad, paint) {
4066 self.alpha.push(instance);
4067 }
4068 continue;
4069 }
4070 let Some(run) = strips.get(draw.strip_range.clone()) else {
4071 continue;
4072 };
4073 // Before the first of this draw's instances lands,
4074 // so the run that closes holds exactly the
4075 // instances drawn with the texture it names.
4076 if paint.external != run_external {
4077 let end = self.alpha.len() as u32;
4078 self.push_run(run_start, end, run_external);
4079 run_start = end;
4080 run_external = paint.external;
4081 }
4082 let to_opaque =
4083 paint.opaque && split_opaque && draw.depth < opaque_limit;
4084 let class = paint.class;
4085 mark_class(&mut self.alpha_marks, self.alpha.len() as u32, class);
4086 if to_opaque {
4087 mark_class(&mut self.opaque_marks, self.opaque.len() as u32, class);
4088 }
4089
4090 // A generation's last strip is its sentinel, which
4091 // is what carries the preceding strip's extent — so
4092 // every instance comes from a pair, and the
4093 // sentinel itself never becomes one.
4094 for pair in run.windows(2) {
4095 self.push_span(&pair[0], &pair[1], paint, to_opaque, window);
4096 }
4097 }
4098 }
4099 RoundOp::Composite(composite) => {
4100 let depth = self
4101 .layer_depth
4102 .get(composite.layer as usize)
4103 .copied()
4104 .unwrap_or(0);
4105 // A composite carries the deepest index inside its
4106 // layer, so a layer recorded after a punch is cut
4107 // against it exactly like a draw would be — which is
4108 // what keeps a translucent layer over the slot from
4109 // being erased by it.
4110 if cuts && !pending.is_empty() {
4111 self.cut_for_punches(
4112 &mut pending,
4113 strips,
4114 depth,
4115 &mut first_segment,
4116 &mut run_start,
4117 &mut run_external,
4118 );
4119 }
4120
4121 // Consecutive draw batches merge into one segment; a
4122 // composite is what breaks the run, because it binds a
4123 // different page as its colour source.
4124 let end = self.alpha.len() as u32;
4125 self.push_run(run_start, end, run_external);
4126
4127 deepest = deepest.max(depth);
4128 let source = match composite.retained {
4129 Some(retained) => CompositeSource::Retained(retained.bracket),
4130 None => CompositeSource::Page(composite.parity),
4131 };
4132 self.segments.push(Segment::Composite(end, source));
4133 self.note_class(self.routing.composite_class());
4134 self.alpha
4135 .push(composite_instance(composite, window.origin(), depth));
4136 run_start = self.alpha.len() as u32;
4137 // A composite draws through group 1's own atlas
4138 // binding, so the run after it starts un-bound again.
4139 run_external = None;
4140 }
4141 }
4142 }
4143
4144 let end = self.alpha.len() as u32;
4145 self.push_run(run_start, end, run_external);
4146
4147 // Accumulated rather than assigned: a layer whose round was cut
4148 // renders in several rounds, and the depth its composite carries is
4149 // the deepest index across all of them.
4150 let own_layer = page
4151 .map(|page| page.layer)
4152 .or(retained.map(|retained| retained.layer));
4153 if let Some(layer) = own_layer
4154 && let Some(slot) = self.layer_depth.get_mut(layer as usize)
4155 {
4156 *slot = (*slot).max(deepest);
4157 }
4158
4159 let released = released_groups(&round.released);
4160
4161 self.rounds.push(RoundPlan {
4162 page: page.map(|page| PagePlan {
4163 parity: page.parity,
4164 size: page.size,
4165 continued: page.continued,
4166 }),
4167 retained: retained.map(|retained| RetainedRound {
4168 bracket: retained.bracket,
4169 continued: retained.continued,
4170 }),
4171 segments: first_segment..self.segments.len(),
4172 released,
4173 filter,
4174 copy: None,
4175 // The round's last plan draws to its own end; a punch past
4176 // every op of the frame is issued after the whole walk instead.
4177 punch: (0, 0),
4178 });
4179 }
4180
4181 // Whatever no op was recorded after: a `ClearRect` recorded last, which
4182 // is the ordinary shape of a platform-view slot.
4183 if !pending.is_empty() {
4184 self.punch = self.push_punches(pending, strips);
4185 }
4186
4187 // The opaque region is one segment of the one depth-writing pass, cut
4188 // into class runs like any other.
4189 self.class_runs.clear();
4190 merge_class_runs(
4191 marked_runs(&self.opaque_marks, 0, self.opaque.len() as u32),
4192 self.routing.min_run,
4193 self.routing.uber,
4194 &mut self.class_runs,
4195 );
4196 debug_assert!(
4197 runs_draw(&self.class_runs, &self.opaque),
4198 "the opaque marks disagree with the instances they mark"
4199 );
4200 for run in &self.class_runs {
4201 self.opaque_segments
4202 .push(Segment::Strips(run.first, run.count, run.class));
4203 self.opaque_classes.insert(run.class);
4204 }
4205 }
4206
4207 /// Records that the round being planned draws a run of `class`.
4208 fn note_class(&mut self, class: PaintClass) {
4209 if self.planning_page {
4210 self.page_classes.insert(class);
4211 } else {
4212 self.surface_classes.insert(class);
4213 }
4214 }
4215
4216 /// Closes the open span of a surface round at `depth`, issuing every punch
4217 /// recorded before it as a pass of its own.
4218 ///
4219 /// This is the whole painter-order restoration: the ops recorded before the
4220 /// punch become a plan that ends here, the punch's own instances follow
4221 /// them in the buffer, and the ops recorded after it start a plan of their
4222 /// own — so the erase lands between the two rather than after both. A punch
4223 /// pass is a pass of its own because it draws through a different pipeline
4224 /// (destination-out) than the strips around it, not because of what it
4225 /// covers.
4226 ///
4227 /// Does nothing when no pending punch is shallower than `depth`, which is
4228 /// every op of every frame that records no clear.
4229 fn cut_for_punches(
4230 &mut self,
4231 pending: &mut &[ClearPunch],
4232 strips: &[Strip],
4233 depth: u32,
4234 first_segment: &mut usize,
4235 run_start: &mut u32,
4236 run_external: &mut Option<u32>,
4237 ) {
4238 let cut = pending
4239 .iter()
4240 .take_while(|punch| punch.depth < depth)
4241 .count();
4242 if cut == 0 {
4243 return;
4244 }
4245 let (issued, rest) = pending.split_at(cut);
4246 *pending = rest;
4247
4248 // Close the run this cut interrupts, so the plan's segments name only
4249 // what was recorded before the punch.
4250 let end = self.alpha.len() as u32;
4251 self.push_run(*run_start, end, *run_external);
4252 // The punch's own instances go in next and are drawn solid, so the run
4253 // resumed after the cut starts with nothing bound.
4254 *run_external = None;
4255
4256 let punch = self.push_punches(issued, strips);
4257 self.rounds.push(RoundPlan {
4258 page: None,
4259 retained: None,
4260 segments: *first_segment..self.segments.len(),
4261 // The pages a cut round consumed are handed back when its LAST plan
4262 // ends: nothing between two plans of one round acquires a page, and
4263 // a composite in an earlier plan still has to sample the page it
4264 // names.
4265 released: [false; MAX_LIVE_PAGES],
4266 filter: None,
4267 copy: None,
4268 punch,
4269 });
4270 *first_segment = self.segments.len();
4271 *run_start = self.alpha.len() as u32;
4272 }
4273
4274 /// Records the open run of instances `first..end` as segments drawn with
4275 /// the texture in `external` bound — one per paint-class run the span
4276 /// splits into — or nothing at all when the run is empty.
4277 ///
4278 /// The one place a strip segment is created, so both run-breaking rules —
4279 /// a segment holds instances sharing one group-1 binding, and one paint
4280 /// class ([`merge_class_runs`]) — are stated once rather than at each of
4281 /// the four points a run can close. The class runs tile the span in order,
4282 /// so the segments still draw `first..end` front to back exactly as one
4283 /// segment did.
4284 fn push_run(&mut self, first: u32, end: u32, external: Option<u32>) {
4285 if end.checked_sub(first).filter(|count| *count > 0).is_none() {
4286 return;
4287 }
4288 let Some(span) = self.alpha.get(first as usize..end as usize) else {
4289 return;
4290 };
4291 self.class_runs.clear();
4292 merge_class_runs(
4293 marked_runs(&self.alpha_marks, first, end),
4294 self.routing.min_run,
4295 self.routing.uber,
4296 &mut self.class_runs,
4297 );
4298 debug_assert!(
4299 runs_draw(&self.class_runs, span),
4300 "the alpha marks disagree with the instances they mark"
4301 );
4302 for index in 0..self.class_runs.len() {
4303 let run = self.class_runs[index];
4304 let at = first.saturating_add(run.first);
4305 self.segments.push(match external {
4306 Some(slot) => Segment::External(at, run.count, slot, run.class),
4307 None => Segment::Strips(at, run.count, run.class),
4308 });
4309 self.note_class(run.class);
4310 }
4311 }
4312
4313 /// Appends a backdrop blur's rectangle — the strip run `run` of its one
4314 /// draw, at painter-order `depth` — as instances sampling the blurred page
4315 /// in `parity`'s group, and the [`Segment::Backdrop`] drawing them.
4316 ///
4317 /// Each instance reads the page at its own position relative to the
4318 /// backdrop's region, which the page holds at its origin, so every pixel
4319 /// of the rectangle takes the blur of the backdrop under that very pixel;
4320 /// the run's own coverage — the rectangle's edges, and any coverage-mask
4321 /// clip it was generated under — weights it exactly as it would a fill.
4322 /// Never opaque: the blur of a backdrop reaching transparent edges is not.
4323 fn push_backdrop(
4324 &mut self,
4325 backdrop: &Backdrop,
4326 parity: PageParity,
4327 depth: u32,
4328 run: &[Strip],
4329 window: PageWindow,
4330 ) {
4331 let class = self.routing.composite_class();
4332 let paint = PackedPaint {
4333 payload: PaintPayload::Layer(backdrop.region.x0, backdrop.region.y0),
4334 paint: LAYER_PAINT_SOURCE | u32::from(u8::MAX),
4335 depth_index: depth,
4336 opaque: false,
4337 external: None,
4338 class,
4339 glyph_quad: false,
4340 };
4341 let first = self.alpha.len() as u32;
4342 for pair in run.windows(2) {
4343 self.push_span(&pair[0], &pair[1], paint, false, window);
4344 }
4345 let count = (self.alpha.len() as u32).saturating_sub(first);
4346 if count > 0 {
4347 self.segments.push(Segment::Backdrop(first, count, parity));
4348 self.note_class(class);
4349 }
4350 }
4351
4352 /// Turns `punches` into destination-out instances, appended to the alpha
4353 /// region and named by no round's segments, as `(first, count)`.
4354 ///
4355 /// Each punch is a strip run like any other, drawn with an opaque source so
4356 /// the blend state's `1 − src.a` reaches zero exactly where the coverage is
4357 /// full, and carrying the painter-order depth it was hoisted from — which
4358 /// still orders it against the frame's one depth-writing pass, whose opaque
4359 /// coverage is recorded once ahead of every round and so cannot be cut.
4360 fn push_punches(&mut self, punches: &[ClearPunch], strips: &[Strip]) -> (u32, u32) {
4361 let first = self.alpha.len() as u32;
4362
4363 for punch in punches {
4364 let paint = PackedPaint {
4365 payload: PaintPayload::Solid(PUNCH_SOURCE),
4366 paint: SOLID_PAINT,
4367 depth_index: punch.depth,
4368 // An erase never joins the depth-writing pass: it establishes
4369 // no colour for a later fragment to be rejected against.
4370 opaque: false,
4371 external: None,
4372 class: PaintClass::SolidCoverage,
4373 glyph_quad: false,
4374 };
4375 let Some(run) = strips.get(punch.strip_range.clone()) else {
4376 continue;
4377 };
4378 for pair in run.windows(2) {
4379 // Only the frame's own target is punched, so a punch's
4380 // instances are never shifted and never clipped.
4381 self.push_span(&pair[0], &pair[1], paint, false, PageWindow::ROOT);
4382 }
4383 }
4384
4385 (first, (self.alpha.len() as u32).saturating_sub(first))
4386 }
4387
4388 /// Emits the instances the `strip`/`next` pair describes: the strip's own
4389 /// alpha-sampled span, plus the solid span filling the gap to `next` when
4390 /// the winding between them says there is one.
4391 ///
4392 /// `window` shifts each instance's *geometry* into the round's target and
4393 /// clips it to the column that target holds, while the paint is still
4394 /// sampled at the scene position the strip was rasterized at — a layer's
4395 /// contents move into its page, the gradient or image painting them does
4396 /// not. An instance the window culls entirely is not emitted at all.
4397 fn push_span(
4398 &mut self,
4399 strip: &Strip,
4400 next: &Strip,
4401 paint: PackedPaint,
4402 to_opaque: bool,
4403 window: PageWindow,
4404 ) {
4405 let values = paint.values_at(strip.x, strip.y);
4406 let mut span = GpuStrip::from_strip_pair(strip, next, values);
4407 if window.place(&mut span, paint) {
4408 self.alpha.push(span);
4409 }
4410 // A gap starts where the strip ends rather than where it begins, so a
4411 // position-sampled paint is re-evaluated at the gap's own origin — the
4412 // gap is a different piece of the scene, not a continuation of the
4413 // span's sampling. `place` re-evaluates it once more if the window's
4414 // own left edge moves the instance again.
4415 if let Some(mut gap) = GpuStrip::gap_fill(strip, next, values) {
4416 gap.payload = paint.payload_at(gap.x, gap.y);
4417 if window.place(&mut gap, paint) {
4418 if to_opaque {
4419 self.opaque.push(gap);
4420 } else {
4421 self.alpha.push(gap);
4422 }
4423 }
4424 }
4425 }
4426
4427 /// Resolves `round`'s filter pass, if it has one, packing the layer's
4428 /// parameter block on first sight and claiming the pass's instance slot.
4429 ///
4430 /// `None` for an ordinary round, and also for the two shapes that cannot
4431 /// occur: a filter round with no page (the scheduler always gives one a
4432 /// page) and a recorded kind [`served_filter`] does not recognise (the
4433 /// scheduler refuses one before it plans a round). Both answer by leaving
4434 /// the round's pass unissued — its
4435 /// page is still cleared — rather than by asserting (E17).
4436 fn plan_filter(
4437 &mut self,
4438 frame: &CompiledFrame,
4439 round: &Round,
4440 backdrops: &[Backdrop],
4441 ) -> Option<FilterPlan> {
4442 let pass = round.filter_pass()?;
4443 let page = round.page()?;
4444 let recorded = frame.recorder.layers.get(pass.layer as usize)?;
4445 // A backdrop's layer is a regular one: its blur is the backdrop
4446 // record's, not the recording's.
4447 let data_offset = match backdrops
4448 .iter()
4449 .find(|backdrop| backdrop.layer == pass.layer)
4450 {
4451 Some(backdrop) => self.pack_filter_block(pass.layer, || {
4452 Some(GpuFilterData::from(GpuGaussianBlur::from(&backdrop.blur())))
4453 })?,
4454 None => self.filter_block(pass.layer, &recorded.kind)?,
4455 };
4456
4457 let instance = self.filter_passes;
4458 self.filter_passes = self.filter_passes.saturating_add(1);
4459 Some(FilterPlan {
4460 step: pass.step,
4461 source: pass.source,
4462 data_offset,
4463 original: SizeU16::from(page.bounds),
4464 instance,
4465 })
4466 }
4467
4468 /// The texel offset of `layer`'s parameter block, packing the block on
4469 /// first sight.
4470 ///
4471 /// A linear scan rather than a map: a frame's filter layers are counted in
4472 /// ones (a filter layer is served only directly under the surface), and one
4473 /// allocation-free vector beats a hash map that would have to be cleared
4474 /// every frame.
4475 fn filter_block(&mut self, layer: u32, kind: &RecordedLayerKind) -> Option<u32> {
4476 // The one dispatch every filter-recognising site in this crate shares
4477 // (`schedule::layer_role`/`filter_rounds`), so the block packed here is
4478 // for the same filter the scheduler planned the passes of.
4479 self.pack_filter_block(layer, || {
4480 Some(match served_filter(layer, kind).ok()? {
4481 ServedFilter::Blur(blur) => GpuFilterData::from(GpuGaussianBlur::from(&blur)),
4482 ServedFilter::DropShadow(shadow) => {
4483 GpuFilterData::from(GpuDropShadow::from(&shadow))
4484 }
4485 })
4486 })
4487 }
4488
4489 /// The texel offset of `layer`'s parameter block, packing the one `make`
4490 /// answers on first sight — [`Self::filter_block`]'s bookkeeping, shared
4491 /// with a backdrop blur, whose block comes from its own record.
4492 fn pack_filter_block(
4493 &mut self,
4494 layer: u32,
4495 make: impl FnOnce() -> Option<GpuFilterData>,
4496 ) -> Option<u32> {
4497 let index = match self.filter_layers.iter().position(|id| *id == layer) {
4498 Some(index) => index,
4499 None => {
4500 let block = make()?;
4501 self.filter_layers.push(layer);
4502 self.filter_blocks.push(block);
4503 self.filter_layers.len().saturating_sub(1)
4504 }
4505 };
4506
4507 u32::try_from(index)
4508 .ok()?
4509 .checked_mul(GpuFilterData::SIZE_TEXELS)
4510 }
4511
4512 /// Whether this frame records a hole-punch pass at all — at a cut inside a
4513 /// surface round, or after every round of the frame.
4514 ///
4515 /// Derived rather than counted alongside the instances: the two places a
4516 /// punch can land are the two places its instances are recorded from, and
4517 /// one answer read off both is one fewer field to keep in step.
4518 fn punches(&self) -> bool {
4519 self.punch.1 > 0 || self.rounds.iter().any(|plan| plan.punch.1 > 0)
4520 }
4521
4522 /// How many of this frame's rounds render *strips* into a pooled page, and
4523 /// so how many viewport uniforms of their own the frame needs.
4524 ///
4525 /// A filter round targets a page too and is deliberately not counted: it
4526 /// binds no viewport uniform at all, mapping NDC against the destination
4527 /// extent its own instance carries.
4528 fn page_rounds(&self) -> usize {
4529 self.rounds
4530 .iter()
4531 .filter(|plan| {
4532 (plan.page.is_some() || plan.retained.is_some())
4533 && plan.filter.is_none()
4534 && plan.copy.is_none()
4535 })
4536 .count()
4537 }
4538
4539 /// The passes a frame planned like this one records, in order, into
4540 /// `out` — the pass list of the module doc's *The frame's passes*, stated
4541 /// over the plan alone so it can be checked without a device.
4542 ///
4543 /// `record_frame` is what actually records them, and a debug build holds
4544 /// every frame it records to this list (`EngineRenderer::check_pass_list`).
4545 /// The two agree by construction on every shape the walk serves; the
4546 /// unreachable skips the walk guards against (a page the pool refuses, a
4547 /// pipeline the frame never resolved) are the only places they could part.
4548 #[cfg(any(test, debug_assertions))]
4549 fn plan_passes(&self, shape: &PassShape, out: &mut Vec<FramePass>) {
4550 out.clear();
4551 if !shape.merge {
4552 out.push(FramePass::Clear);
4553 }
4554 if !shape.instances {
4555 if shape.merge {
4556 out.push(FramePass::Opening);
4557 }
4558 return;
4559 }
4560 if !shape.merge && shape.opaque {
4561 out.push(FramePass::Opaque);
4562 }
4563
4564 let mut owed = shape.merge;
4565 for plan in &self.rounds {
4566 // A copy reads the surface back, so the opening — the clear and
4567 // the opaque strips — is recorded ahead of it.
4568 if plan.copy.is_some() {
4569 if owed {
4570 out.push(FramePass::Opening);
4571 owed = false;
4572 }
4573 out.push(FramePass::Copy);
4574 continue;
4575 }
4576 if plan.filter.is_some() {
4577 out.push(FramePass::Filter);
4578 continue;
4579 }
4580 if plan.page.is_some() || plan.retained.is_some() {
4581 out.push(FramePass::Page);
4582 } else if !plan.segments.is_empty() {
4583 out.push(if owed {
4584 FramePass::Opening
4585 } else {
4586 FramePass::Surface
4587 });
4588 owed = false;
4589 }
4590 if shape.punch && plan.punch.1 > 0 {
4591 if owed {
4592 out.push(FramePass::Opening);
4593 owed = false;
4594 }
4595 out.push(FramePass::Punch);
4596 }
4597 }
4598 if owed {
4599 out.push(FramePass::Opening);
4600 }
4601 if shape.punch && self.punch.1 > 0 {
4602 out.push(FramePass::Punch);
4603 }
4604 }
4605
4606 /// The instance bytes, opaque buffer first, so one vertex buffer serves
4607 /// every pass and each is a contiguous instance range into it.
4608 fn instance_bytes(&self) -> (&[u8], &[u8]) {
4609 (
4610 bytemuck::cast_slice(&self.opaque),
4611 bytemuck::cast_slice(&self.alpha),
4612 )
4613 }
4614}
4615
4616/// Records that the instances from `at` on carry `class`, unless the marks
4617/// already say so.
4618///
4619/// A draw that emitted nothing leaves its mark at the same index as the next
4620/// draw's, which then overwrites it — and drops it altogether when that leaves
4621/// two neighbouring marks of one class.
4622fn mark_class(marks: &mut Vec<(u32, PaintClass)>, at: u32, class: PaintClass) {
4623 match marks.last_mut() {
4624 Some(last) if last.1 == class => {}
4625 Some(last) if last.0 == at => {
4626 last.1 = class;
4627 let len = marks.len();
4628 if len >= 2 && marks[len - 2].1 == class {
4629 marks.pop();
4630 }
4631 }
4632 _ => marks.push((at, class)),
4633 }
4634}
4635
4636/// The class runs `marks` says cover instances `first..end`, relative to
4637/// `first` — the per-draw input [`merge_class_runs`] merges.
4638///
4639/// Every instance of a planned segment was emitted after its draw's mark, so
4640/// the mark at or before `first` names its class; a composite or punch
4641/// instance between two marks is never inside a segment's range.
4642fn marked_runs(
4643 marks: &[(u32, PaintClass)],
4644 first: u32,
4645 end: u32,
4646) -> impl Iterator<Item = ClassRun> + '_ {
4647 let from = marks
4648 .partition_point(|(at, _)| *at <= first)
4649 .saturating_sub(1);
4650 let tail = marks.get(from..).unwrap_or(&[]);
4651 tail.iter()
4652 .enumerate()
4653 .take_while(move |(_, (at, _))| *at < end)
4654 .filter_map(move |(index, (at, class))| {
4655 let start = (*at).max(first);
4656 let stop = tail.get(index + 1).map_or(end, |next| next.0.min(end));
4657 (stop > start).then(|| ClassRun {
4658 first: start - first,
4659 count: stop - start,
4660 class: *class,
4661 })
4662 })
4663}
4664
4665/// Whether `runs` tile `instances` front to back and each run's program draws
4666/// every instance in it, as far as the instance's own flag can tell
4667/// ([`PaintClass::admits_flag`]) — what the marks must agree with, checked in
4668/// debug builds.
4669fn runs_draw(runs: &[ClassRun], instances: &[GpuStrip]) -> bool {
4670 let mut next = 0_u32;
4671 for run in runs {
4672 if run.first != next || run.count == 0 {
4673 return false;
4674 }
4675 let Some(span) = instances.get(run.first as usize..(run.first + run.count) as usize) else {
4676 return false;
4677 };
4678 if !span
4679 .iter()
4680 .all(|instance| run.class.admits_flag(instance.paint_and_rect_flag))
4681 {
4682 return false;
4683 }
4684 next += run.count;
4685 }
4686 next as usize == instances.len()
4687}
4688
4689/// The column of device space one round's target holds: the origin its
4690/// instances are shifted to, and the edges they are clipped against.
4691///
4692/// A page holds its layer at the page's own origin, and its composite samples
4693/// exactly `(0, 0)`-to-its-own-extent back out
4694/// ([`Composite::source`](crate::schedule::Composite::source)). For a layer
4695/// [banded](crate::schedule::pages::page_bands) into column pages that makes
4696/// the band's own rectangle two things at once: the shift, and the *clip*. The
4697/// scheduler replays the layer's whole op list into every band — a band differs
4698/// only in which page it writes and where that page lands — so this is the site
4699/// that decides which part of each replayed strip belongs to the band at hand.
4700/// Emission is where the decision lives because it is the one place that knows
4701/// both the origin and the width; the alternative, clipping the ops
4702/// scheduler-side, would have to re-rasterize geometry the scheduler only holds
4703/// as draw ranges.
4704///
4705/// Two rules, and both halves of each matter:
4706///
4707/// - a span entirely outside the column contributes **no instance** — shifting
4708/// it by a saturating subtraction instead would clamp it onto the page's own
4709/// edge, stretching a span the scene drew elsewhere across content this band
4710/// really holds;
4711/// - a span straddling an edge has its geometry **and** its first alpha column
4712/// ([`GpuStrip::col_idx_or_rect_frac`]) advanced by the same amount, because
4713/// the shader reads that column unshifted and steps one column per pixel of
4714/// the instance (`shaders/strip.wgsl`'s `col_offset`): advancing the x
4715/// without the column would sample another pixel's coverage.
4716///
4717/// The frame's own surface is [`ROOT`](Self::ROOT), a window over the whole
4718/// device grid that shifts nothing and clips nothing. A layer that fits one
4719/// page is one band covering all of it, and a layer's bounds are the
4720/// tile-aligned union of its own draws' bounds, so the clip is a no-op for
4721/// every layer that is not banded.
4722#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4723struct PageWindow {
4724 /// Left edge of the column in device space — what every instance's `x` is
4725 /// shifted by, and what one left of it is culled against.
4726 x0: u16,
4727 /// Right edge of the column in device space, exclusive.
4728 x1: u16,
4729 /// Top edge of the column in device space — what every instance's `y` is
4730 /// shifted by. A band spans its layer's whole height, so the vertical axis
4731 /// is shifted and never clipped.
4732 y0: u16,
4733}
4734
4735impl PageWindow {
4736 /// The whole device grid: the window a surface round carries.
4737 const ROOT: Self = Self {
4738 x0: 0,
4739 x1: u16::MAX,
4740 y0: 0,
4741 };
4742
4743 /// The window `page` renders through — its own tile-aligned bounds, which
4744 /// are one column band's for a banded layer and the whole layer's for
4745 /// every other.
4746 fn of(page: &PageTarget) -> Self {
4747 Self {
4748 x0: page.bounds.x0,
4749 x1: page.bounds.x1,
4750 y0: page.bounds.y0,
4751 }
4752 }
4753
4754 /// The window a retained page's round renders through: the body's own
4755 /// tile-aligned bounds, never banded.
4756 fn of_retained(target: &RetainedTarget) -> Self {
4757 Self {
4758 x0: target.bounds.x0,
4759 x1: target.bounds.x1,
4760 y0: target.bounds.y0,
4761 }
4762 }
4763
4764 /// The origin instances are shifted by.
4765 fn origin(self) -> (u16, u16) {
4766 (self.x0, self.y0)
4767 }
4768
4769 /// Clips `span` — a strip instance still carrying its scene coordinates —
4770 /// to this window and shifts it into the target, answering whether any of
4771 /// it survived.
4772 ///
4773 /// `paint` is the draw's own paint, needed because a left-clipped instance
4774 /// starts at a different scene position than the one it was rasterized at:
4775 /// a position-sampled paint (a gradient, an image) is re-evaluated there,
4776 /// exactly as a gap fill is at its own origin, so the paint keeps landing
4777 /// where the scene put it rather than being squeezed into the clipped span.
4778 fn place(self, span: &mut GpuStrip, paint: PackedPaint) -> bool {
4779 let end = span.x.saturating_add(span.width);
4780 if span.width == 0 || end <= self.x0 || span.x >= self.x1 {
4781 return false;
4782 }
4783
4784 let cut = self.x0.saturating_sub(span.x);
4785 if cut > 0 {
4786 span.x = self.x0;
4787 span.width = span.width.saturating_sub(cut);
4788 // The dense part of an instance starts at its left edge, so the
4789 // columns cut off the geometry are cut off the coverage too.
4790 let dense = cut.min(span.dense_width_or_rect_height);
4791 span.dense_width_or_rect_height = span.dense_width_or_rect_height.saturating_sub(dense);
4792 // A sparse instance names no column at all: the fragment stage
4793 // reads `col + dense_width` as "does this instance sample
4794 // coverage", so leaving a non-zero column on one whose dense part
4795 // is gone would send it to the alpha texture for coverage it never
4796 // wrote.
4797 span.col_idx_or_rect_frac = if span.dense_width_or_rect_height == 0 {
4798 0
4799 } else {
4800 span.col_idx_or_rect_frac.saturating_add(u32::from(dense))
4801 };
4802 span.payload = paint.payload_at(span.x, span.y);
4803 }
4804
4805 // Past the right edge is outside the region the composite samples, so
4806 // it reaches no pixel of the parent either way; clipping it keeps a
4807 // band's instances inside the band's own rectangle rather than relying
4808 // on the page's quantized extent to swallow the overhang.
4809 let over = end.saturating_sub(self.x1);
4810 if over > 0 {
4811 span.width = span.width.saturating_sub(over);
4812 span.dense_width_or_rect_height = span.dense_width_or_rect_height.min(span.width);
4813 }
4814
4815 span.x = span.x.saturating_sub(self.x0);
4816 span.y = span.y.saturating_sub(self.y0);
4817 span.width > 0
4818 }
4819
4820 /// The rect instance drawing `quad` with `paint`, clipped to this window
4821 /// and shifted into its target, or `None` when none of it survives.
4822 ///
4823 /// The same two rules [`Self::place`] keeps, for a whole rectangle: a quad
4824 /// outside the column contributes nothing, and one straddling an edge is
4825 /// trimmed — its atlas texel advanced by exactly the columns cut off its
4826 /// left edge, so every surviving pixel still samples the texel it did. A
4827 /// quad's coverage is its rectangle whole (it lands on whole pixels), so
4828 /// there is no coverage column to advance and the instance carries no edge
4829 /// fractions at all.
4830 fn place_quad(self, quad: &GlyphQuad, paint: PackedPaint) -> Option<GpuStrip> {
4831 let end = quad.x.saturating_add(quad.width);
4832 if quad.width == 0 || quad.height == 0 || end <= self.x0 || quad.x >= self.x1 {
4833 return None;
4834 }
4835 let cut = self.x0.saturating_sub(quad.x);
4836 let x = quad.x.max(self.x0);
4837 let width = end.min(self.x1).saturating_sub(x);
4838 let instance = GpuStrip::from_rect(
4839 x.saturating_sub(self.x0),
4840 quad.y.saturating_sub(self.y0),
4841 width,
4842 quad.height,
4843 0,
4844 StripDraw {
4845 payload: pack_u16_pair(quad.atlas_x.saturating_add(cut), quad.atlas_y),
4846 paint: paint.paint,
4847 depth_index: paint.depth_index,
4848 },
4849 );
4850 (width > 0).then_some(instance)
4851 }
4852}
4853
4854/// The glyph quad the draw at `depth` is, out of a frame's quads in depth
4855/// order — `None` only for a frame whose quads disagree with its draws, which
4856/// then draws nothing for that glyph rather than something wrong.
4857fn glyph_quad(quads: &[GlyphQuad], depth: u32) -> Option<&GlyphQuad> {
4858 let found = quads
4859 .binary_search_by_key(&depth, |quad| quad.depth)
4860 .ok()
4861 .and_then(|index| quads.get(index));
4862 debug_assert!(
4863 found.is_some(),
4864 "a glyph-quad draw with no quad at depth {depth}"
4865 );
4866 found
4867}
4868
4869/// The scheduler roles `plan` gives `frame`'s retained layers, written into
4870/// `roles` (cleared first). A body the store refused gets none and is
4871/// scheduled as the ordinary layer it lowered to.
4872fn retained_roles(frame: &CompiledFrame, plan: &RetainedPlan, roles: &mut RetainedLayers) {
4873 roles.clear();
4874 for (index, (bracket, decision)) in frame.retained.iter().zip(&plan.decisions).enumerate() {
4875 let id = u32::try_from(index).unwrap_or(u32::MAX);
4876 let role = match *decision {
4877 RetainedDecision::Record { .. } => RetainedRole::Record { bracket: id },
4878 RetainedDecision::Draw { dest, origin } => RetainedRole::Draw {
4879 bracket: id,
4880 dest,
4881 origin,
4882 },
4883 RetainedDecision::Hidden => RetainedRole::Hidden,
4884 RetainedDecision::Inline(_) => continue,
4885 };
4886 roles.set(bracket.layer, role);
4887 }
4888}
4889
4890/// Which of `frame`'s retained brackets the schedule renders into a retained
4891/// page — the bodies whose pages the store may commit. A body the store
4892/// admitted can still render nothing, nested inside a layer that composites
4893/// nothing.
4894fn retained_rendered(frame: &CompiledFrame, rounds: &[Round]) -> Vec<bool> {
4895 let mut rendered = vec![false; frame.retained.len()];
4896 for round in rounds {
4897 if let Some(target) = round.retained()
4898 && let Some(slot) = rendered.get_mut(target.bracket as usize)
4899 {
4900 *slot = true;
4901 }
4902 }
4903 rendered
4904}
4905
4906/// The single quad that composites a finished page onto `origin`-shifted
4907/// target.
4908///
4909/// A whole-rectangle instance with no fractional edges: a layer's bounds are
4910/// tile-aligned, so the quad covers whole pixels and the fragment stage leaves
4911/// its coverage at one. The payload is the page texel the quad's top-left
4912/// corner samples — the page holds the layer at its own origin, so that is
4913/// `Composite::source`'s origin and not the layer's device position.
4914fn composite_instance(composite: &Composite, origin: (u16, u16), depth: u32) -> GpuStrip {
4915 let bounds = composite.bounds;
4916 let source = composite.source();
4917
4918 GpuStrip::from_rect(
4919 bounds.x0.saturating_sub(origin.0),
4920 bounds.y0.saturating_sub(origin.1),
4921 bounds.width(),
4922 bounds.height(),
4923 0,
4924 StripDraw {
4925 payload: pack_u16_pair(source.x0, source.y0),
4926 paint: LAYER_PAINT_SOURCE | u32::from(pack_opacity(composite.opacity)),
4927 depth_index: depth,
4928 },
4929 )
4930}
4931
4932/// A layer's constant opacity as the eight bits a composite instance carries.
4933///
4934/// Clamped rather than refused: the scheduler already admits only opacities
4935/// strictly between zero and one, and rounding is what keeps a 0.5 layer at
4936/// exactly the 128 the reference renderer's own packing produces.
4937fn pack_opacity(opacity: f32) -> u8 {
4938 (opacity.clamp(0.0, 1.0) * 255.0).round() as u8
4939}
4940
4941/// Where a strip instance's payload comes from.
4942///
4943/// A solid paint carries its colour there; every other paint reads its colour
4944/// from a record instead, and spends the payload on the scene position the
4945/// paint is sampled at — which is why the payload is per instance rather than
4946/// per draw.
4947#[derive(Debug, Clone, Copy)]
4948enum PaintPayload {
4949 /// A premultiplied RGBA8 colour, the same for every instance of the draw.
4950 Solid(u32),
4951 /// The instance's own scene-space origin, packed as a `u16` pair.
4952 Position,
4953 /// The page texel under the instance's own origin, for a page holding the
4954 /// device region starting at `(x, y)` at its own origin: a backdrop blur's
4955 /// rectangle sampling its blurred page.
4956 Layer(u16, u16),
4957}
4958
4959/// One draw's paint, resolved to the values its instances repeat.
4960#[derive(Debug, Clone, Copy)]
4961struct PackedPaint {
4962 payload: PaintPayload,
4963 /// The packed paint descriptor, without [`RECT_STRIP_FLAG`](crate::gpu::strips::RECT_STRIP_FLAG).
4964 paint: u32,
4965 depth_index: u32,
4966 /// Whether every pixel this paint produces is opaque, and so whether its
4967 /// fully-covered spans may take the depth-writing pass.
4968 opaque: bool,
4969 /// The external-texture slot this paint's instances have to be drawn with
4970 /// bound, or `None` for every paint the frame's atlas binding serves.
4971 ///
4972 /// What breaks a pass's instances into runs: the strip pipelines have one
4973 /// external binding, so consecutive instances may share a segment only
4974 /// while this stays equal (see [`crate::gpu::bindings::ExternalRuns`]).
4975 external: Option<u32>,
4976 /// The strip program class this paint's instances are drawn with — from
4977 /// the resolved record, so an image is told apart by the path it samples
4978 /// through (see [`PaintClass::of_image`]).
4979 class: PaintClass,
4980 /// Whether this is a glyph run's shared record, whose draws are each one
4981 /// glyph quad ([`crate::compile::GlyphQuad`]) rather than a strip run.
4982 glyph_quad: bool,
4983}
4984
4985impl PackedPaint {
4986 /// The payload an instance whose scene origin is `(x, y)` carries.
4987 fn payload_at(self, x: u16, y: u16) -> u32 {
4988 match self.payload {
4989 PaintPayload::Solid(rgba) => rgba,
4990 PaintPayload::Position => pack_u16_pair(x, y),
4991 PaintPayload::Layer(x0, y0) => {
4992 pack_u16_pair(x.saturating_sub(x0), y.saturating_sub(y0))
4993 }
4994 }
4995 }
4996
4997 /// The per-instance values for an instance whose scene origin is `(x, y)`.
4998 fn values_at(self, x: u16, y: u16) -> StripDraw {
4999 StripDraw {
5000 payload: self.payload_at(x, y),
5001 paint: self.paint,
5002 depth_index: self.depth_index,
5003 }
5004 }
5005}
5006
5007/// Two `u16`s in one word, low half first — the packing the shader's
5008/// `unpack_u16_pair` reverses.
5009fn pack_u16_pair(x: u16, y: u16) -> u32 {
5010 u32::from(x) | (u32::from(y) << 16)
5011}
5012
5013/// The shader values for `paint`, resolved against the frame's paint slots.
5014///
5015/// Answers `None` for an indexed paint the frame could not resolve — one whose
5016/// entry is missing from `paint_slots` entirely, or one the lowering refused
5017/// because the engine has nowhere to hold its pixels yet (see the module
5018/// header). The draw is then skipped rather than stamped in a wrong colour.
5019fn pack_paint(
5020 paint: &Paint,
5021 depth: u32,
5022 paint_slots: &[Option<ResolvedPaint>],
5023) -> Option<PackedPaint> {
5024 match paint {
5025 Paint::Solid(color) => Some(PackedPaint {
5026 payload: PaintPayload::Solid(color.as_premul_rgba8().to_u32()),
5027 paint: SOLID_PAINT,
5028 depth_index: depth,
5029 opaque: color.is_opaque(),
5030 external: None,
5031 class: PaintClass::SolidCoverage,
5032 glyph_quad: false,
5033 }),
5034 Paint::Indexed(indexed) => {
5035 let Some(Some(resolved)) = paint_slots.get(indexed.index()) else {
5036 INDEXED_PAINT_WARNING.call_once(|| {
5037 log::warn!(
5038 "an indexed paint could not be lowered to a GPU record; those draws are \
5039 skipped (logged once)"
5040 );
5041 });
5042 return None;
5043 };
5044 Some(PackedPaint {
5045 payload: PaintPayload::Position,
5046 paint: pack_paint_descriptor(resolved.paint_type, resolved.texel_offset),
5047 depth_index: depth,
5048 opaque: resolved.opaque,
5049 external: resolved.external,
5050 class: resolved.class,
5051 glyph_quad: resolved.paint_type == PaintType::Glyph,
5052 })
5053 }
5054 }
5055}
5056
5057/// One encoded paint after lowering: where its record landed and how a strip
5058/// instance names it.
5059#[derive(Debug, Clone, Copy)]
5060struct ResolvedPaint {
5061 /// How the fragment shader reads the record.
5062 paint_type: PaintType,
5063 /// The texel the record starts at in the encoded-paint texture.
5064 texel_offset: u32,
5065 /// Whether every pixel the paint produces is opaque.
5066 opaque: bool,
5067 /// The external-texture slot this paint samples, for a paint that reads a
5068 /// caller-owned texture rather than the atlas (see
5069 /// [`crate::gpu::bindings`]). `None` — the ordinary case — means the
5070 /// frame's atlas binding serves it.
5071 external: Option<u32>,
5072 /// The strip program class the paint is drawn with ([`class_of_record`]).
5073 class: PaintClass,
5074}
5075
5076/// The strip program class a lowered record is drawn with.
5077///
5078/// The paint type decides it for every record but an image, whose record also
5079/// names the path it samples through: nearest or bilinear from the atlas is
5080/// [`PaintClass::Image`], bicubic or the external texture
5081/// [`PaintClass::ImageFiltered`]. The bits are the ones the shader reads —
5082/// quality in bits 0-1 of `image_params`, the source kind in bit 14.
5083fn class_of_record(record: &GpuEncodedPaint) -> PaintClass {
5084 match record {
5085 GpuEncodedPaint::Image(image) => PaintClass::of_image(
5086 image.image_params & IMAGE_QUALITY_BITS,
5087 (image.image_params >> IMAGE_SOURCE_KIND_SHIFT) & 1 == 1,
5088 ),
5089 _ => PaintClass::of_flag(pack_paint_descriptor(record.paint_type(), 0)),
5090 }
5091}
5092
5093/// Bits 0-1 of an image record's `image_params`: the sampling quality.
5094const IMAGE_QUALITY_BITS: u32 = 0b11;
5095
5096/// The bit of an image record's `image_params` marking an external texture.
5097const IMAGE_SOURCE_KIND_SHIFT: u32 = 14;
5098
5099/// One resource texture and the extent it currently holds.
5100#[derive(Debug)]
5101struct ResourceTexture {
5102 texture: wgpu::Texture,
5103 view: wgpu::TextureView,
5104 width: u32,
5105 height: u32,
5106}
5107
5108impl ResourceTexture {
5109 fn new(device: &wgpu::Device, descriptor: &wgpu::TextureDescriptor<'_>) -> Self {
5110 let texture = device.create_texture(descriptor);
5111 let view = texture.create_view(&wgpu::TextureViewDescriptor::default());
5112 Self {
5113 texture,
5114 view,
5115 width: descriptor.size.width,
5116 height: descriptor.size.height,
5117 }
5118 }
5119
5120 /// The whole texture as a copy destination.
5121 fn copy_target(&self) -> wgpu::TexelCopyTextureInfo<'_> {
5122 wgpu::TexelCopyTextureInfo {
5123 texture: &self.texture,
5124 mip_level: 0,
5125 origin: wgpu::Origin3d::ZERO,
5126 aspect: wgpu::TextureAspect::All,
5127 }
5128 }
5129
5130 /// The extent one full-texture upload covers.
5131 fn extent(&self) -> wgpu::Extent3d {
5132 wgpu::Extent3d {
5133 width: self.width,
5134 height: self.height,
5135 depth_or_array_layers: 1,
5136 }
5137 }
5138}
5139
5140/// The GPU resources one renderer holds across frames.
5141#[derive(Debug)]
5142struct FrameResources {
5143 alphas: ResourceTexture,
5144 paints: ResourceTexture,
5145 gradients: ResourceTexture,
5146 /// Stand-ins for the strip shader's bindings a given pass has nothing real
5147 /// for: the layer input (a real page when a composite is being drawn, this
5148 /// otherwise), the glyph/image atlas array until the first image is
5149 /// resident, and an externally bound texture, which nothing writes yet.
5150 /// Every declared binding has to be bound for a pass to validate, whether
5151 /// or not an instance samples it.
5152 placeholders: Placeholders,
5153 /// The linear, clamp-to-edge sampler the image-sampler program family
5154 /// filters bilinear atlas images through, bound beside the atlas array in
5155 /// every strip group 1; `None` for the manual family, whose layout
5156 /// declares no sampler (see [`EngineRenderer::with_image_sampler`]).
5157 atlas_sampler: Option<wgpu::Sampler>,
5158 config: wgpu::Buffer,
5159 /// One viewport uniform per page round of the busiest frame so far.
5160 ///
5161 /// A round's NDC mapping is against the extent of the attachment it writes,
5162 /// and a page's extent is neither the frame's nor the same from one round
5163 /// to the next, so each needs a buffer of its own — a bind group holds the
5164 /// whole buffer, not an offset into one. Grown only, like every other
5165 /// retained resource here.
5166 page_configs: Vec<wgpu::Buffer>,
5167 instances: Option<wgpu::Buffer>,
5168 instance_capacity: u64,
5169 /// The GPU records this frame's indexed paints resolve against, in
5170 /// serialization order — which is the order the texel offsets in
5171 /// `paint_slots` were taken from.
5172 paints_data: Vec<GpuEncodedPaint>,
5173 /// One slot per encoded paint of the frame, indexed by
5174 /// [`Paint::Indexed`](vello_common::paint::Paint::Indexed): `None` for one
5175 /// the engine could not lower. Kept parallel to the *encoded* paints
5176 /// rather than compacted to the lowered ones, because a draw names its
5177 /// paint by the compiler's index.
5178 paint_slots: Vec<Option<ResolvedPaint>>,
5179 /// Reusable per-paint ramp residency, filled from a frame's LUT requests
5180 /// before its paints are lowered.
5181 paint_ramps: Vec<Option<CachedRamp>>,
5182 /// Reusable staging for the encoded-paint upload, padded to the texture's
5183 /// footprint.
5184 paint_staging: Vec<u8>,
5185 /// The distinct external textures this frame's paints sample, in the order
5186 /// they were first named — the slot numbering the pass segments and the
5187 /// group-1 bind groups both address by.
5188 external_runs: ExternalRuns,
5189 /// One set of bind groups per strip render state, built against any one
5190 /// of that state's class pipelines. Every strip pipeline — each state,
5191 /// each paint class — is built against one explicit layout
5192 /// ([`crate::gpu::bindings::strip_pipeline_layout`]), so a state's set binds
5193 /// under all of its classes; keeping one set per state is what the map
5194 /// already did when each pipeline derived a layout of its own.
5195 bind_groups: HashMap<EnginePipeline, StripBindGroups>,
5196 /// The target format the live bind groups were built against; a change
5197 /// means different pipelines, so the whole map is dropped rather than
5198 /// accumulating a set per format ever rendered to.
5199 bind_group_format: Option<wgpu::TextureFormat>,
5200 /// The real image atlas array, created lazily by the first frame that
5201 /// makes an image resident and grown from then on — `None` is exactly
5202 /// [`Placeholders::atlas_array`]'s domain, a renderer that has never
5203 /// drawn an image.
5204 atlas: Option<AtlasArray>,
5205 /// The atlas rectangle every image the renderer has ever drawn currently
5206 /// holds, keyed by the stable [`ImageId`] the compiler's own residency
5207 /// minted it. See [`FrameResources::resolve_paints`] for how this is
5208 /// kept in step with residency across frames without reaching into the
5209 /// compiler's own cache.
5210 image_registry: HashMap<ImageId, ResidentImage>,
5211 /// How many atlas regions have been declined — a write or clear the array
5212 /// refused, or one the budget says the array could not hold. Counted rather
5213 /// than dropped silently, because each one is a region the frame believed
5214 /// it had filled.
5215 refused_regions: u64,
5216}
5217
5218impl FrameResources {
5219 fn new(device: &wgpu::Device, dim: u32, image_sampler: bool) -> Self {
5220 let min = gpu::MIN_RESOURCE_TEXTURE_HEIGHT;
5221 Self {
5222 alphas: ResourceTexture::new(device, &gpu::alpha_texture_descriptor(dim, min)),
5223 paints: ResourceTexture::new(
5224 device,
5225 &gpu::paint_texture::encoded_paints_texture_descriptor(dim, min),
5226 ),
5227 gradients: ResourceTexture::new(device, &gradient_texture_descriptor(dim, min)),
5228 placeholders: Placeholders::new(device),
5229 atlas_sampler: image_sampler.then(|| atlas_sampler(device)),
5230 config: device.create_buffer(&config_descriptor("frust-engine config uniform")),
5231 page_configs: Vec::new(),
5232 instances: None,
5233 instance_capacity: 0,
5234 paints_data: Vec::new(),
5235 paint_slots: Vec::new(),
5236 paint_ramps: Vec::new(),
5237 paint_staging: Vec::new(),
5238 external_runs: ExternalRuns::new(),
5239 bind_groups: HashMap::new(),
5240 bind_group_format: None,
5241 atlas: None,
5242 image_registry: HashMap::new(),
5243 refused_regions: 0,
5244 }
5245 }
5246
5247 /// Drops the atlas array, everything recorded about what lives in it, and
5248 /// the bind groups naming it.
5249 ///
5250 /// What a re-budget needs: the rectangles the registry holds were allocated
5251 /// in a geometry that no longer exists, so keeping any of them would point
5252 /// a paint at a rectangle of a texture that is gone.
5253 fn reset_atlas(&mut self) {
5254 self.atlas = None;
5255 self.image_registry.clear();
5256 self.bind_groups.clear();
5257 }
5258
5259 /// Count `regions` atlas regions as declined, saying so once.
5260 ///
5261 /// Once, not per region: a budget and an array that disagree disagree about
5262 /// every region, and a per-frame line would bury the fact under itself. The
5263 /// count on [`EngineRenderer::refused_atlas_regions`] is the measure.
5264 fn note_refused_regions(&mut self, regions: u64) {
5265 self.refused_regions = self.refused_regions.saturating_add(regions);
5266 ATLAS_REFUSAL_WARNING.call_once(|| {
5267 log::warn!(
5268 "an atlas region was refused by the image atlas array; those images are skipped \
5269 and their uploads re-offered on a later frame (logged once — see \
5270 EngineRenderer::refused_atlas_regions for the count)"
5271 );
5272 });
5273 }
5274
5275 /// Services `frame`'s LUT and image residency, then lowers its encoded
5276 /// paints into the records the shader samples.
5277 ///
5278 /// Leaves `paints_data` holding the lowered records in serialization order
5279 /// and `paint_slots` naming, per *encoded* paint, the texel its record
5280 /// starts at — or `None` where the paint could not be lowered.
5281 ///
5282 /// A solid-only frame leaves both empty and touches neither the cache nor
5283 /// the paint texture, so it costs exactly what it did before paints were
5284 /// wired up. Image residency is still serviced even then, since an image
5285 /// can be evicted on a frame that draws nothing at all (see
5286 /// [`Self::update_image_registry`]).
5287 fn resolve_paints(
5288 &mut self,
5289 frame: &CompiledFrame,
5290 cache: &mut GradientCache,
5291 budget: AtlasBudget,
5292 externals: &ExternalTextures,
5293 ) {
5294 self.paints_data.clear();
5295 self.paint_slots.clear();
5296 self.external_runs.clear();
5297
5298 // Kept in step every frame, not only when this frame's own paints
5299 // need it: an image reaped by the compiler's age-based eviction while
5300 // nothing draws it must still be forgotten here, or a later draw that
5301 // reuses its freed rectangle's `ImageId` would read the stale entry.
5302 self.update_image_registry(frame, budget);
5303
5304 if frame.encoded_paints.is_empty() {
5305 return;
5306 }
5307
5308 // Ramp residency next, for the whole frame: a ramp's offset is only
5309 // meaningful once the cache has finished baking this frame's misses,
5310 // and a record built before that would name a ramp that had not been
5311 // packed yet.
5312 self.paint_ramps.clear();
5313 self.paint_ramps.resize(frame.encoded_paints.len(), None);
5314 for request in &frame.lut_requests {
5315 let ramp = resolve_lut_request(*request, &frame.encoded_paints, cache);
5316 if let Some(slot) = self.paint_ramps.get_mut(request.paint_index) {
5317 *slot = ramp;
5318 }
5319 }
5320
5321 let mut texel_offset = 0;
5322 // Ascending, like the walk below, so one cursor marks them.
5323 let mut glyph_records = frame.glyph_run_paints.iter().copied().peekable();
5324 for (index, paint) in frame.encoded_paints.iter().enumerate() {
5325 // A glyph run's shared record: lowered like any atlas image, but
5326 // drawn by glyph quads (see `crate::compile::GlyphQuad`).
5327 let glyph_record = glyph_records.next_if_eq(&index).is_some();
5328 // An external texture's slot travels with its record: the record
5329 // itself only says "sample the external binding", and which
5330 // texture that binding holds is settled per run when the pass is
5331 // recorded rather than per paint.
5332 let mut external = None;
5333 let lowered = match paint {
5334 EncodedPaint::Image(image) => image_id(image)
5335 .and_then(|id| self.image_registry.get(&id))
5336 .and_then(|resident| {
5337 lower_encoded_image(image, resident)
5338 .map(|record| fit_minified(record, resident))
5339 }),
5340 // A paint naming a texture nothing is bound under lowers to
5341 // nothing, so its draws are skipped rather than sampling
5342 // whichever texture the binding happens to hold.
5343 EncodedPaint::ExternalTexture(entry) => externals
5344 .view(entry.texture_id.0)
5345 .and_then(|_| self.external_runs.slot_of(entry.texture_id.0))
5346 .map(|slot| {
5347 external = Some(slot);
5348 lower_encoded_external(entry)
5349 }),
5350 _ => {
5351 let ramp = self.paint_ramps.get(index).copied().flatten();
5352 lower_encoded_paint(paint, ramp)
5353 }
5354 };
5355 let slot = lowered.map(|record| {
5356 let resolved = ResolvedPaint {
5357 paint_type: if glyph_record {
5358 PaintType::Glyph
5359 } else {
5360 record.paint_type()
5361 },
5362 texel_offset,
5363 opaque: !paint.may_have_transparency(),
5364 external,
5365 class: if glyph_record {
5366 PaintClass::Glyph
5367 } else {
5368 class_of_record(&record)
5369 },
5370 };
5371 texel_offset += record.texel_len();
5372 self.paints_data.push(record);
5373 resolved
5374 });
5375 self.paint_slots.push(slot);
5376 }
5377 }
5378
5379 /// Keeps [`Self::image_registry`] in step with the compiler's own image
5380 /// residency, without reaching into it: the residency's rectangles are
5381 /// not reachable from here (see [`crate::gpu::paint_texture::lower_encoded_paint`]'s
5382 /// doc for why), so this reconstructs the same information from what a
5383 /// compiled frame already reports.
5384 ///
5385 /// Two passes, and each reads the frame's plan directly rather than
5386 /// inferring anything from draw order. First, every region this frame's
5387 /// residency reaped is forgotten — `frame.image_evictions` names it by
5388 /// rectangle, and a rectangle uniquely identifies the one image that held it
5389 /// (padding is always zero in this engine, so the reported and the stored
5390 /// rectangle are the same value; see [`crate::cache::images`]'s module doc).
5391 /// Second, every entry of `frame.image_uploads` is registered under the
5392 /// [`ImageId`] it carries.
5393 ///
5394 /// The order matters and the id does. Evictions run first so a same-frame
5395 /// evict-then-reallocate that reuses a rectangle registers the new tenant
5396 /// rather than having it removed again. And the upload naming its own id is
5397 /// what makes this sound under a *re-offered* plan: an upload the previous
5398 /// frame did not service is reported again alongside no new draw of its own,
5399 /// so a walk pairing uploads positionally against this frame's encoded
5400 /// paints would hand a fresh image the stale upload's rectangle.
5401 ///
5402 /// A region the atlas array could not hold at `budget`'s geometry and this
5403 /// frame's depth is counted and left unregistered instead. Its draws are
5404 /// then skipped, which is the whole point: a registered rectangle nothing
5405 /// wrote would be sampled as whatever the texture happened to contain.
5406 fn update_image_registry(&mut self, frame: &CompiledFrame, budget: AtlasBudget) {
5407 if !frame.image_evictions.is_empty() {
5408 // A set rather than `Vec::contains`: both sides of this scan scale
5409 // with content — the registry with how many images and glyph slots
5410 // are resident, the plan with how hard the atlas is churning — and
5411 // their product is the frame path's, not a report's.
5412 let cleared: HashSet<AtlasRegion> = frame.image_evictions.iter().copied().collect();
5413 self.image_registry
5414 .retain(|_, resident| !cleared.contains(&resident.region));
5415 }
5416
5417 let mut refused = 0_u64;
5418 for upload in &frame.image_uploads {
5419 if !budget.contains(upload.region, frame.atlas_layers) {
5420 refused = refused.saturating_add(1);
5421 continue;
5422 }
5423 self.image_registry.insert(
5424 upload.id,
5425 ResidentImage {
5426 id: upload.id,
5427 region: upload.region,
5428 natural: upload.natural,
5429 padding: u32::from(ATLAS_PADDING),
5430 may_have_transparency: upload.may_have_transparency,
5431 },
5432 );
5433 }
5434 // The glyph half of the same registry, and the reason it is a second
5435 // loop rather than a branch inside the first: a glyph slot carries no
5436 // pixels and is *not* re-offered across frames, because the pixels are
5437 // produced by the replay pass rather than uploaded from here. Its
5438 // rectangle is reported by every draw that names it (see
5439 // [`crate::compile::GlyphSlot`]), so registering it here is what makes
5440 // a handle `glifo` recycled resolve against its current occupant.
5441 for slot in &frame.glyph_slots {
5442 if !budget.contains(slot.region, frame.atlas_layers) {
5443 refused = refused.saturating_add(1);
5444 continue;
5445 }
5446 self.image_registry.insert(
5447 slot.id,
5448 ResidentImage {
5449 id: slot.id,
5450 region: slot.region,
5451 // Never minified: a glyph is rasterized straight into the
5452 // rectangle it was allocated, so the natural extent and the
5453 // resident one are the same value by construction.
5454 natural: slot.region.size,
5455 padding: slot.padding,
5456 may_have_transparency: true,
5457 },
5458 );
5459 }
5460
5461 if refused > 0 {
5462 self.note_refused_regions(refused);
5463 }
5464 }
5465
5466 /// Grows or creates the image atlas array to hold `layers` layers at
5467 /// `budget`'s per-layer extent, clearing the live bind groups when it
5468 /// does — a bind group built against the old (or absent) atlas view would
5469 /// otherwise sample nothing, or a freed texture.
5470 ///
5471 /// A frame that has never made an image resident (`layers == 0`) leaves
5472 /// the atlas unset, so the strip shader's binding stays on
5473 /// [`Placeholders::atlas_array`] until the first one is.
5474 ///
5475 /// Growth submits a maintenance command buffer of its own rather than
5476 /// recording into the frame's encoder — see [`crate::gpu::atlas`]. So this
5477 /// must be called after the frame's last fallible step (a refused frame
5478 /// must submit nothing) and before its atlas writes are issued (the copy has
5479 /// to precede them, and a submit flushes whatever is already queued).
5480 fn ensure_atlas(
5481 &mut self,
5482 device: &wgpu::Device,
5483 queue: &wgpu::Queue,
5484 budget: AtlasBudget,
5485 layers: u32,
5486 ) -> bool {
5487 let grew = match &mut self.atlas {
5488 None if layers == 0 => false,
5489 None => {
5490 self.atlas = Some(AtlasArray::with_layers(
5491 device,
5492 budget.atlas_size.0,
5493 budget.atlas_size.1,
5494 layers,
5495 ));
5496 true
5497 }
5498 Some(atlas) => atlas.ensure_layers(device, queue, layers),
5499 };
5500 if grew {
5501 self.bind_groups.clear();
5502 }
5503 grew
5504 }
5505
5506 /// Flushes `frame`'s atlas evictions and uploads against the live atlas
5507 /// array, evictions first — a rectangle this frame's residency freed may
5508 /// already hold a fresh upload by the time this runs (see
5509 /// [`crate::cache::images`]'s module doc), so clearing after writing
5510 /// would erase the image that just moved in.
5511 ///
5512 /// Answers whether the whole plan reached the array — which is what tells
5513 /// the compiler it may stop re-offering it. A region the array declined is
5514 /// counted (see [`Self::note_refused_regions`]) and the plan stays pending,
5515 /// so a later frame that has grown the array writes it rather than the
5516 /// image being lost.
5517 ///
5518 /// An absent array with a plan to service is that same disagreement rather
5519 /// than a quiet no-op: both are driven by the same `frame.atlas_layers`, so
5520 /// an empty plan and an absent atlas normally agree.
5521 fn upload_atlas(&mut self, queue: &wgpu::Queue, frame: &CompiledFrame) -> bool {
5522 let refused = match self.atlas.as_ref() {
5523 None => (frame.image_evictions.len() + frame.image_uploads.len()) as u64,
5524 Some(atlas) => {
5525 let mut refused = 0_u64;
5526 for region in &frame.image_evictions {
5527 if !atlas.clear_region(queue, *region) {
5528 refused = refused.saturating_add(1);
5529 }
5530 }
5531 for upload in &frame.image_uploads {
5532 if !atlas.write_region(queue, upload.region, upload.pixels.data_as_u8_slice()) {
5533 refused = refused.saturating_add(1);
5534 }
5535 }
5536 refused
5537 }
5538 };
5539
5540 if refused == 0 {
5541 return true;
5542 }
5543 self.note_refused_regions(refused);
5544 false
5545 }
5546
5547 /// Makes sure there is one viewport uniform per page round of this frame.
5548 ///
5549 /// Grown only: the buffers are 32 bytes each and a frame that once needed
5550 /// four keeps them rather than reallocating on the next frame that does.
5551 fn ensure_page_configs(&mut self, device: &wgpu::Device, rounds: usize) {
5552 while self.page_configs.len() < rounds {
5553 self.page_configs
5554 .push(device.create_buffer(&config_descriptor("frust-engine page config uniform")));
5555 }
5556 }
5557
5558 /// The texels this frame's encoded paints occupy.
5559 fn paint_texels(&self) -> u32 {
5560 self.paints_data
5561 .iter()
5562 .map(GpuEncodedPaint::texel_len)
5563 .sum()
5564 }
5565
5566 /// The height the gradient LUT texture must grow to for every ramp the
5567 /// cache has packed, or `None` when it already fits.
5568 ///
5569 /// # Errors
5570 ///
5571 /// [`EngineError::PaintCapacity`]: a LUT set past the resource dimension
5572 /// squared has nowhere to live, and the frame path reports it rather than
5573 /// asserting.
5574 fn grown_gradient_height(&self, cache: &GradientCache) -> Result<Option<u32>, EngineError> {
5575 let width = self.gradients.width;
5576 let texels = u32::try_from(cache.luts_size() / BYTES_PER_TEXEL as usize)
5577 .map_err(|_| EngineError::PaintCapacity)?;
5578 let required = texels.div_ceil(width).max(gpu::MIN_RESOURCE_TEXTURE_HEIGHT);
5579 if required > width {
5580 return Err(EngineError::PaintCapacity);
5581 }
5582 Ok((required > self.gradients.height).then_some(required))
5583 }
5584
5585 fn resize_alphas(&mut self, device: &wgpu::Device, height: Option<u32>) {
5586 if let Some(height) = height {
5587 self.alphas = ResourceTexture::new(
5588 device,
5589 &gpu::alpha_texture_descriptor(self.alphas.width, height),
5590 );
5591 self.bind_groups.clear();
5592 }
5593 }
5594
5595 fn resize_paints(&mut self, device: &wgpu::Device, height: Option<u32>) {
5596 if let Some(height) = height {
5597 self.paints = ResourceTexture::new(
5598 device,
5599 &gpu::paint_texture::encoded_paints_texture_descriptor(self.paints.width, height),
5600 );
5601 self.bind_groups.clear();
5602 }
5603 }
5604
5605 fn resize_gradients(&mut self, device: &wgpu::Device, height: Option<u32>) {
5606 if let Some(height) = height {
5607 self.gradients = ResourceTexture::new(
5608 device,
5609 &gradient_texture_descriptor(self.gradients.width, height),
5610 );
5611 self.bind_groups.clear();
5612 }
5613 }
5614
5615 /// Uploads the frame's coverage, paint records, colour ramps, atlas
5616 /// evictions/uploads and config, answering whether the atlas plan was
5617 /// serviced in full (see [`Self::upload_atlas`]).
5618 fn upload(
5619 &mut self,
5620 queue: &wgpu::Queue,
5621 frame: &mut CompiledFrame,
5622 cache: &mut GradientCache,
5623 size: (u16, u16),
5624 dim: u32,
5625 ) -> bool {
5626 let atlas_serviced = self.upload_atlas(queue, frame);
5627
5628 let alphas = &self.alphas;
5629 gpu::with_padded_alphas(
5630 &mut frame.strips.alphas,
5631 alphas.width,
5632 alphas.height,
5633 |bytes| {
5634 queue.write_texture(
5635 alphas.copy_target(),
5636 bytes,
5637 resource_layout(gpu::resource_bytes_per_row(alphas.width), alphas.height),
5638 alphas.extent(),
5639 );
5640 },
5641 );
5642
5643 if !self.paints_data.is_empty() {
5644 let footprint = gpu::resource_texture_bytes(self.paints.width, self.paints.height);
5645 self.paint_staging
5646 .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
5647 // The height was checked to fit before any pass was recorded, so a
5648 // buffer too short for the records is unreachable; skipping the
5649 // upload rather than unwrapping keeps the frame path total anyway.
5650 if GpuEncodedPaint::serialize_to_buffer(&self.paints_data, &mut self.paint_staging)
5651 .is_ok()
5652 {
5653 queue.write_texture(
5654 self.paints.copy_target(),
5655 &self.paint_staging,
5656 resource_layout(
5657 gpu::resource_bytes_per_row(self.paints.width),
5658 self.paints.height,
5659 ),
5660 self.paints.extent(),
5661 );
5662 }
5663 }
5664
5665 if cache.has_changed() {
5666 let layout = GradientTextureLayout {
5667 width: self.gradients.width,
5668 height: self.gradients.height,
5669 };
5670 if let Some(upload) = cache.begin_upload(layout) {
5671 queue.write_texture(
5672 self.gradients.copy_target(),
5673 &upload,
5674 resource_layout(upload.bytes_per_row(), self.gradients.height),
5675 self.gradients.extent(),
5676 );
5677 }
5678 cache.mark_synced();
5679 }
5680
5681 // The strip program addresses all three resource textures by the
5682 // config's width shifts alone — the LUT by the paint texture's — so
5683 // the config built from `dim` is only sound while each of them really
5684 // is `dim` wide (see `gpu::config`'s module doc). Every resize above
5685 // grows height only, so this holds by construction; the assertion is
5686 // what notices a change that breaks it.
5687 debug_assert!(
5688 resource_widths_agree(
5689 dim,
5690 self.alphas.width,
5691 self.paints.width,
5692 self.gradients.width
5693 ),
5694 "strip resource textures must all be resource_texture_dim ({dim}) wide: \
5695 alphas {}, paints {}, gradients {}",
5696 self.alphas.width,
5697 self.paints.width,
5698 self.gradients.width,
5699 );
5700 let config = GpuConfig::new(u32::from(size.0), u32::from(size.1), dim, dim);
5701 queue.write_buffer(&self.config, 0, bytemuck::bytes_of(&config));
5702
5703 atlas_serviced
5704 }
5705
5706 /// Grows the instance buffer if this frame outgrew it, then uploads the
5707 /// opaque and alpha instances back to back.
5708 fn upload_instances(&mut self, device: &wgpu::Device, queue: &wgpu::Queue, scratch: &Scratch) {
5709 let (opaque, alpha) = scratch.instance_bytes();
5710 let required = (opaque.len() + alpha.len()) as u64;
5711 if required == 0 {
5712 return;
5713 }
5714
5715 let buffer = match &mut self.instances {
5716 Some(buffer) if self.instance_capacity >= required => buffer,
5717 slot => {
5718 let floor = MIN_INSTANCE_CAPACITY * size_of::<GpuStrip>() as u64;
5719 let capacity = required
5720 .checked_next_power_of_two()
5721 .unwrap_or(required)
5722 .max(floor);
5723 self.instance_capacity = capacity;
5724 slot.insert(device.create_buffer(&wgpu::BufferDescriptor {
5725 label: Some("frust-engine strip instances"),
5726 size: capacity,
5727 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
5728 mapped_at_creation: false,
5729 }))
5730 }
5731 };
5732 if !opaque.is_empty() {
5733 queue.write_buffer(buffer, 0, opaque);
5734 }
5735 if !alpha.is_empty() {
5736 queue.write_buffer(buffer, opaque.len() as u64, alpha);
5737 }
5738 }
5739
5740 /// Builds `variant`'s bind groups if it has none, first dropping every
5741 /// set when the target format changed under them.
5742 fn ensure_bind_groups(
5743 &mut self,
5744 device: &wgpu::Device,
5745 variant: EnginePipeline,
5746 pipeline: &wgpu::RenderPipeline,
5747 format: wgpu::TextureFormat,
5748 ) {
5749 if self.bind_group_format != Some(format) {
5750 self.bind_groups.clear();
5751 self.bind_group_format = Some(format);
5752 }
5753 if self.bind_groups.contains_key(&variant) {
5754 return;
5755 }
5756 let atlas_view = self
5757 .atlas
5758 .as_ref()
5759 .map(AtlasArray::view)
5760 .unwrap_or(&self.placeholders.atlas_array);
5761 let groups = StripBindGroups::new(
5762 device,
5763 pipeline,
5764 &self.alphas.view,
5765 &self.config,
5766 &self.placeholders,
5767 atlas_view,
5768 self.atlas_sampler.as_ref(),
5769 &self.paints.view,
5770 &self.gradients.view,
5771 );
5772 self.bind_groups.insert(variant, groups);
5773 }
5774
5775 /// Builds `variant`'s group 1 for every external texture this frame draws
5776 /// that it has none for yet.
5777 ///
5778 /// Called after [`Self::ensure_bind_groups`] has built the variant's own
5779 /// set, and only for the variants the frame records with, so a frame that
5780 /// draws no external texture builds nothing. What is built is retained
5781 /// alongside the rest of the variant's groups and dropped with them — on a
5782 /// format change, an atlas growth or a re-budget, all of which change the
5783 /// atlas view this group also holds.
5784 ///
5785 /// A texture with no registered view is skipped rather than substituted:
5786 /// its paint did not resolve either, so nothing in the frame names its
5787 /// slot.
5788 fn ensure_external_groups(
5789 &mut self,
5790 device: &wgpu::Device,
5791 variant: EnginePipeline,
5792 pipeline: &wgpu::RenderPipeline,
5793 externals: &ExternalTextures,
5794 ) {
5795 if self.external_runs.is_empty() {
5796 return;
5797 }
5798 // Destructured rather than reached through `self`: the group map is
5799 // borrowed mutably while the atlas view and the placeholders are read.
5800 let Self {
5801 bind_groups,
5802 external_runs,
5803 atlas,
5804 placeholders,
5805 atlas_sampler,
5806 ..
5807 } = self;
5808 let atlas_view = atlas
5809 .as_ref()
5810 .map(AtlasArray::view)
5811 .unwrap_or(&placeholders.atlas_array);
5812 let Some(groups) = bind_groups.get_mut(&variant) else {
5813 return;
5814 };
5815 for key in external_runs.keys() {
5816 if groups.externals.contains_key(key) {
5817 continue;
5818 }
5819 let Some(view) = externals.view(*key) else {
5820 continue;
5821 };
5822 groups.externals.insert(
5823 *key,
5824 images_bind_group(device, pipeline, atlas_view, view, atlas_sampler.as_ref()),
5825 );
5826 }
5827 }
5828
5829 /// Drops every bind group naming the texture registered under `key`.
5830 ///
5831 /// Called when that registration changes, because a group holds its view
5832 /// by value: keeping one past a re-bind would go on sampling the texture
5833 /// the caller replaced, and keeping one past an unbind would hold the
5834 /// caller's texture alive for as long as this renderer lives.
5835 fn forget_external(&mut self, key: u64) {
5836 for groups in self.bind_groups.values_mut() {
5837 groups.externals.remove(&key);
5838 }
5839 }
5840}
5841
5842/// Lower one atlas page's recorded commands into the strips that draw it,
5843/// answering whether the whole page could be expressed.
5844///
5845/// The caller-supplied half of the render-to-atlas seam: `gpu::atlas` owns the
5846/// pass, the orderings and the submit, while turning a command stream into
5847/// strips is compiler work and stays on this side of the edge — `compile`
5848/// already depends on `gpu::atlas`, so taking the reverse dependency would make
5849/// the two mutually recursive.
5850///
5851/// The stream is replayed as a scene in *page* space and compiled by
5852/// `lowering`, which is why that compiler is sized to the page rather than to
5853/// the surface. Only the four commands an outline glyph produces are lowered;
5854/// anything else — a clip path, a blend layer, a gradient paint, which is to
5855/// say every COLR shape — refuses the page whole rather than drawing part of
5856/// it. An indexed paint coming back out of the compile means the same thing:
5857/// the atlas pass binds no paint texture, so a record it would have to sample
5858/// cannot be drawn.
5859///
5860/// Refusing a page is a *last* line rather than the design, because refusal
5861/// cannot be made harmless here: `glifo` clears a recorder's commands whether
5862/// or not this answered `true`, and it offers no way to withdraw the entries
5863/// whose pixels those commands were going to be. Nothing that would reach this
5864/// refusal is therefore admitted to the atlas in the first place — a colour
5865/// face never takes the atlas route at all (`crate::text::atlas_policy`), so
5866/// what arrives here is the solid outline stream this lowers.
5867fn lower_atlas_page(
5868 recorder: &AtlasCommandRecorder,
5869 buffers: &mut AtlasPageBuffers,
5870 lowering: &mut SceneCompiler,
5871 page: (u16, u16),
5872) -> bool {
5873 let mut scene = Scene::new();
5874 {
5875 let mut builder = SceneBuilder::new(&mut scene);
5876 let mut transform = Affine::IDENTITY;
5877 let mut brush = Brush::Solid(Color::BLACK);
5878
5879 for command in &recorder.commands {
5880 match command {
5881 AtlasCommand::SetTransform(next) => transform = *next,
5882 AtlasCommand::SetPaint(AtlasPaint::Solid(color)) => brush = Brush::Solid(*color),
5883 AtlasCommand::FillPath(path) => {
5884 builder.push_transform(transform);
5885 builder.fill_path((**path).clone(), brush.clone());
5886 builder.pop_transform();
5887 }
5888 AtlasCommand::FillRect(rect) => {
5889 builder.push_transform(transform);
5890 builder.fill_rect(*rect, brush.clone());
5891 builder.pop_transform();
5892 }
5893 _ => return false,
5894 }
5895 }
5896 }
5897
5898 let Ok(frame) = lowering.compile(&scene, Affine::IDENTITY, page) else {
5899 return false;
5900 };
5901
5902 let strips = frame.strip_buf();
5903 for draw in frame.draws() {
5904 let Paint::Solid(color) = &draw.paint else {
5905 return false;
5906 };
5907 let Some(run) = strips.get(draw.strip_range.clone()) else {
5908 continue;
5909 };
5910 push_solid_strips(
5911 run,
5912 color.as_premul_rgba8().to_u32(),
5913 draw.depth,
5914 &mut buffers.instances,
5915 );
5916 }
5917 buffers.alphas.extend_from_slice(frame.alphas());
5918 true
5919}
5920
5921/// The [`ImageId`] an encoded image paint names, or `None` for the one
5922/// [`ImageSource`] variant no residency ever mints — the paint carrying its
5923/// pixels inline as a [`vello_common::pixmap::Pixmap`] rather than through a
5924/// handle. The compiler's own image encoding always produces the handle form
5925/// (see [`crate::compile::paint::encode_image`]), so a compiled frame never
5926/// exercises the `None` arm; it exists because the type itself admits both.
5927fn image_id(image: &EncodedImage) -> Option<ImageId> {
5928 match image.source {
5929 ImageSource::OpaqueId { id, .. } => Some(id),
5930 ImageSource::Pixmap(_) => None,
5931 }
5932}
5933
5934/// A lowered image record corrected for an atlas rectangle that holds a
5935/// *minified* copy of the source.
5936///
5937/// The record's transform maps a device position back onto the image's own
5938/// texels, and the compiler composed it against the source's declared extent —
5939/// it had to, since that is all it knows before residency is consulted. Scaling
5940/// its output by [`ResidentImage::minify_scale`] retargets it at the smaller
5941/// rectangle actually uploaded, which is the whole correction a downsampled
5942/// image needs: the record's `image_size` and `image_offset` already describe
5943/// the resident rectangle.
5944///
5945/// A record stored at full size is returned untouched, which is every image but
5946/// one larger than an atlas layer.
5947fn fit_minified(record: GpuEncodedPaint, resident: &ResidentImage) -> GpuEncodedPaint {
5948 let Some((x, y)) = resident.minify_scale() else {
5949 return record;
5950 };
5951 let GpuEncodedPaint::Image(mut image) = record else {
5952 return record;
5953 };
5954
5955 // `[a, b, c, d, tx, ty]`, mapping `(u, v)` to `(a·u + c·v + tx, b·u + d·v +
5956 // ty)`: the x row is scaled by one factor and the y row by the other.
5957 image.transform[0] *= x;
5958 image.transform[2] *= x;
5959 image.transform[4] *= x;
5960 image.transform[1] *= y;
5961 image.transform[3] *= y;
5962 image.transform[5] *= y;
5963
5964 GpuEncodedPaint::Image(image)
5965}
5966
5967/// The descriptor every `Config` uniform buffer is created with.
5968fn config_descriptor(label: &str) -> wgpu::BufferDescriptor<'_> {
5969 wgpu::BufferDescriptor {
5970 label: Some(label),
5971 size: GpuConfig::SIZE,
5972 usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,
5973 mapped_at_creation: false,
5974 }
5975}
5976
5977/// The texel copy layout of a full resource-texture upload.
5978fn resource_layout(bytes_per_row: u32, rows: u32) -> wgpu::TexelCopyBufferLayout {
5979 wgpu::TexelCopyBufferLayout {
5980 offset: 0,
5981 bytes_per_row: Some(bytes_per_row),
5982 rows_per_image: Some(rows),
5983 }
5984}
5985
5986/// The descriptor for a gradient LUT texture of `width` x `height` texels.
5987fn gradient_texture_descriptor(width: u32, height: u32) -> wgpu::TextureDescriptor<'static> {
5988 wgpu::TextureDescriptor {
5989 label: Some("frust-engine gradient texture"),
5990 size: wgpu::Extent3d {
5991 width,
5992 height,
5993 depth_or_array_layers: 1,
5994 },
5995 mip_level_count: 1,
5996 sample_count: 1,
5997 dimension: wgpu::TextureDimension::D2,
5998 format: GradientTextureLayout::FORMAT,
5999 usage: gpu::RESOURCE_TEXTURE_USAGES,
6000 view_formats: &[],
6001 }
6002}
6003
6004/// The 1x1 stand-ins for the strip shader's not-yet-written bindings.
6005#[derive(Debug)]
6006struct Placeholders {
6007 layer_input: wgpu::TextureView,
6008 atlas_array: wgpu::TextureView,
6009 external: wgpu::TextureView,
6010}
6011
6012impl Placeholders {
6013 fn new(device: &wgpu::Device) -> Self {
6014 Self {
6015 layer_input: placeholder_view(device, "frust-engine layer input placeholder", false),
6016 atlas_array: placeholder_view(device, "frust-engine atlas placeholder", true),
6017 external: placeholder_view(device, "frust-engine external placeholder", false),
6018 }
6019 }
6020}
6021
6022/// A 1x1 transparent `Rgba8Unorm` texture's view, as a plain 2D texture or as
6023/// a 2D array.
6024///
6025/// The array variant allocates **two** layers, not one, matching
6026/// [`gpu::atlas::atlas_texture_descriptor`]'s own floor: wgpu-hal 30.0.1's
6027/// GLES backend derives the GL target from the descriptor's layer count
6028/// alone, and a one-layer array descriptor binds as `GL_TEXTURE_2D` rather
6029/// than `GL_TEXTURE_2D_ARRAY` (see that function's doc comment for the
6030/// file:line and upstream issue refs). Only layer zero of the two is ever
6031/// sampled here.
6032fn placeholder_view(device: &wgpu::Device, label: &str, array: bool) -> wgpu::TextureView {
6033 let texture = device.create_texture(&wgpu::TextureDescriptor {
6034 label: Some(label),
6035 size: wgpu::Extent3d {
6036 width: 1,
6037 height: 1,
6038 depth_or_array_layers: if array { 2 } else { 1 },
6039 },
6040 mip_level_count: 1,
6041 sample_count: 1,
6042 dimension: wgpu::TextureDimension::D2,
6043 format: wgpu::TextureFormat::Rgba8Unorm,
6044 usage: wgpu::TextureUsages::TEXTURE_BINDING,
6045 view_formats: &[],
6046 });
6047 texture.create_view(&wgpu::TextureViewDescriptor {
6048 label: Some(label),
6049 dimension: Some(if array {
6050 wgpu::TextureViewDimension::D2Array
6051 } else {
6052 wgpu::TextureViewDimension::D2
6053 }),
6054 ..Default::default()
6055 })
6056}
6057
6058/// The four bind groups every strip pass sets.
6059///
6060/// The grouping is the shader's, not this module's: coverage plus config plus
6061/// layer input, atlas array plus external texture, encoded paints, gradient
6062/// ramps — four groups exactly, which is the downlevel ceiling with no
6063/// headroom left.
6064#[derive(Debug)]
6065struct StripBindGroups {
6066 resources: wgpu::BindGroup,
6067 images: wgpu::BindGroup,
6068 paints: wgpu::BindGroup,
6069 gradients: wgpu::BindGroup,
6070 /// Group 1 again, once per externally bound texture this variant has
6071 /// drawn: the same atlas array beside that texture's view instead of the
6072 /// placeholder. Keyed by the id a display list names the texture by.
6073 ///
6074 /// A second group rather than a fifth: the four-group ceiling has no
6075 /// headroom (see [`crate::gpu::pipelines`]), so an external texture is
6076 /// bound by re-setting the group the atlas already occupies — which is why
6077 /// a pass's instances are split into runs at all.
6078 externals: HashMap<u64, wgpu::BindGroup>,
6079}
6080
6081impl StripBindGroups {
6082 /// Takes the eight resources one by one rather than a `&FrameResources`
6083 /// so the caller can build a set while holding the map it lands in
6084 /// mutably.
6085 ///
6086 /// `atlas` is the live [`AtlasArray`] view once the renderer has one, or
6087 /// [`Placeholders::atlas_array`] until then — the caller picks, since
6088 /// only it knows which the frame's own `atlas_layers` calls for. `sampler`
6089 /// is [`FrameResources::atlas_sampler`].
6090 #[expect(
6091 clippy::too_many_arguments,
6092 reason = "one bind-group build's full resource list; a struct would \
6093 only rename the same borrows the caller already holds \
6094 mutably in `FrameResources`"
6095 )]
6096 fn new(
6097 device: &wgpu::Device,
6098 pipeline: &wgpu::RenderPipeline,
6099 alphas: &wgpu::TextureView,
6100 config: &wgpu::Buffer,
6101 placeholders: &Placeholders,
6102 atlas: &wgpu::TextureView,
6103 sampler: Option<&wgpu::Sampler>,
6104 paints: &wgpu::TextureView,
6105 gradients: &wgpu::TextureView,
6106 ) -> Self {
6107 let resources_group =
6108 resources_bind_group(device, pipeline, alphas, config, &placeholders.layer_input);
6109 let images = images_bind_group(device, pipeline, atlas, &placeholders.external, sampler);
6110 let paints = device.create_bind_group(&wgpu::BindGroupDescriptor {
6111 label: Some("frust-engine strip paints"),
6112 layout: &pipeline.get_bind_group_layout(2),
6113 entries: &[wgpu::BindGroupEntry {
6114 binding: 0,
6115 resource: wgpu::BindingResource::TextureView(paints),
6116 }],
6117 });
6118 let gradients = device.create_bind_group(&wgpu::BindGroupDescriptor {
6119 label: Some("frust-engine strip gradients"),
6120 layout: &pipeline.get_bind_group_layout(3),
6121 entries: &[wgpu::BindGroupEntry {
6122 binding: 0,
6123 resource: wgpu::BindingResource::TextureView(gradients),
6124 }],
6125 });
6126
6127 Self {
6128 resources: resources_group,
6129 images,
6130 paints,
6131 gradients,
6132 externals: HashMap::new(),
6133 }
6134 }
6135
6136 /// Sets all four groups on `pass`, taking group 0 from `resources` and
6137 /// group 1 from `images` when a segment names one, rather than from this
6138 /// set.
6139 ///
6140 /// Two of the four vary within a pass. Group 0 carries both the pass's
6141 /// viewport uniform and the layer input a composite samples, so a page
6142 /// round and every composite in it substitute their own. Group 1 carries
6143 /// the external texture a run is drawn with, so a segment that samples one
6144 /// substitutes the group holding it; every other segment takes this set's
6145 /// own, which pairs the atlas array with a placeholder nothing reads.
6146 /// Groups 2-3 are frame-wide and belong to the pipeline variant.
6147 fn bind_with(
6148 &self,
6149 pass: &mut wgpu::RenderPass<'_>,
6150 resources: &wgpu::BindGroup,
6151 images: Option<&wgpu::BindGroup>,
6152 ) {
6153 pass.set_bind_group(0, resources, &[]);
6154 pass.set_bind_group(1, images.unwrap_or(&self.images), &[]);
6155 pass.set_bind_group(2, &self.paints, &[]);
6156 pass.set_bind_group(3, &self.gradients, &[]);
6157 }
6158}
6159
6160/// Group 1 of a strip pass: the image atlas array, the caller-owned texture
6161/// an external image paint samples, and — in the image-sampler program family
6162/// only — the sampler the atlas array is filtered through.
6163///
6164/// One function for both shapes the group takes — `external` is
6165/// [`Placeholders::external`] for the frame-wide group, or a registered view
6166/// for the group a run of external instances is drawn with (see
6167/// [`crate::gpu::bindings`]). Built against one pipeline's group layout, as
6168/// [`resources_bind_group`] is; `sampler` is present exactly when that layout
6169/// declares one, which [`FrameResources::atlas_sampler`] already decides.
6170fn images_bind_group(
6171 device: &wgpu::Device,
6172 pipeline: &wgpu::RenderPipeline,
6173 atlas: &wgpu::TextureView,
6174 external: &wgpu::TextureView,
6175 sampler: Option<&wgpu::Sampler>,
6176) -> wgpu::BindGroup {
6177 let textures = [
6178 wgpu::BindGroupEntry {
6179 binding: 0,
6180 resource: wgpu::BindingResource::TextureView(atlas),
6181 },
6182 wgpu::BindGroupEntry {
6183 binding: 1,
6184 resource: wgpu::BindingResource::TextureView(external),
6185 },
6186 ];
6187 let with_sampler;
6188 let entries: &[wgpu::BindGroupEntry<'_>] = match sampler {
6189 Some(sampler) => {
6190 let [atlas, external] = textures;
6191 with_sampler = [
6192 atlas,
6193 external,
6194 wgpu::BindGroupEntry {
6195 binding: ATLAS_SAMPLER_BINDING,
6196 resource: wgpu::BindingResource::Sampler(sampler),
6197 },
6198 ];
6199 &with_sampler
6200 }
6201 None => &textures,
6202 };
6203 device.create_bind_group(&wgpu::BindGroupDescriptor {
6204 label: Some("frust-engine strip images"),
6205 layout: &pipeline.get_bind_group_layout(1),
6206 entries,
6207 })
6208}
6209
6210/// The sampler the image-sampler program family filters bilinear atlas
6211/// images through: linear in both directions, clamped to the edge, one mip.
6212///
6213/// Clamp-to-edge is a backstop rather than the mechanism: the shader already
6214/// clamps every sample position to the centres of the image's own edge texels
6215/// (`shaders/helpers.wgsl`'s `sampled_bilinear`), so the filter never reaches
6216/// past the image's rectangle in the first place.
6217fn atlas_sampler(device: &wgpu::Device) -> wgpu::Sampler {
6218 device.create_sampler(&wgpu::SamplerDescriptor {
6219 label: Some("frust-engine atlas sampler"),
6220 address_mode_u: wgpu::AddressMode::ClampToEdge,
6221 address_mode_v: wgpu::AddressMode::ClampToEdge,
6222 address_mode_w: wgpu::AddressMode::ClampToEdge,
6223 mag_filter: wgpu::FilterMode::Linear,
6224 min_filter: wgpu::FilterMode::Linear,
6225 mipmap_filter: wgpu::MipmapFilterMode::Nearest,
6226 ..Default::default()
6227 })
6228}
6229
6230/// The groups a round's `released` list names, as the per-group flags a
6231/// [`RoundPlan`] carries.
6232fn released_groups(released: &[PageParity]) -> [bool; MAX_LIVE_PAGES] {
6233 let mut groups = [false; MAX_LIVE_PAGES];
6234 for parity in released {
6235 if let Some(slot) = groups.get_mut(parity.index()) {
6236 *slot = true;
6237 }
6238 }
6239 groups
6240}
6241
6242/// How a region of a frame's own target is read back into a pooled page —
6243/// the mechanism every effect sampling what the frame has already painted
6244/// shares (a backdrop blur today; see [`crate::filters`]' *The backdrop
6245/// blur*).
6246///
6247/// Either way the region reaches the page through one pass of
6248/// [`EnginePipeline::Copy`] that clears the page and draws the region at its
6249/// origin: a pooled page is a render target and a sampled texture, never a
6250/// copy destination, so a texture copy cannot land in one directly. The modes
6251/// differ only in what that pass reads.
6252///
6253/// Decided per frame from the target texture itself, because the caller
6254/// decides what it renders into: a swapchain texture is `RENDER_ATTACHMENT`
6255/// only on this tier and is never widened, so `frust-render` renders a frame
6256/// carrying a backdrop into an intermediate of its own (in the swapchain's own
6257/// format, sampled and copyable) and presents it with one copy pass.
6258#[derive(Debug, Clone, Copy, PartialEq, Eq)]
6259pub enum RegionCopyMode {
6260 /// The target is `COPY_SRC` in the pages' own [`INTERMEDIATE_FORMAT`]: a
6261 /// `copy_texture_to_texture` lands the region in the renderer's own
6262 /// staging texture, unconverted, and the copy pass reads it from there —
6263 /// the route a target that cannot be sampled (a headless readback target)
6264 /// still has.
6265 Texture,
6266 /// The target is `TEXTURE_BINDING`: the copy pass reads the region
6267 /// straight out of it — any format the target is rendered in, converted on
6268 /// the way into the page's own.
6269 Fragment,
6270 /// The target can be neither copied nor sampled: no region is read back,
6271 /// and a backdrop blur draws nothing.
6272 Unavailable,
6273}
6274
6275impl RegionCopyMode {
6276 /// The mode a frame rendering into `target` reads regions back with.
6277 #[must_use]
6278 pub fn of(target: &wgpu::Texture) -> Self {
6279 Self::for_usage(target.usage(), target.format(), target.sample_count())
6280 }
6281
6282 /// [`Self::of`] over the three facts it reads, so the decision is
6283 /// host-testable without a device.
6284 ///
6285 /// Sampling is preferred whenever the target allows it — it costs the one
6286 /// pass alone, where a texture copy costs a transfer as well.
6287 #[must_use]
6288 pub fn for_usage(
6289 usage: wgpu::TextureUsages,
6290 format: wgpu::TextureFormat,
6291 sample_count: u32,
6292 ) -> Self {
6293 if sample_count != 1 {
6294 Self::Unavailable
6295 } else if usage.contains(wgpu::TextureUsages::TEXTURE_BINDING) {
6296 Self::Fragment
6297 } else if format == INTERMEDIATE_FORMAT && usage.contains(wgpu::TextureUsages::COPY_SRC) {
6298 Self::Texture
6299 } else {
6300 Self::Unavailable
6301 }
6302 }
6303}
6304
6305/// Reports a frame whose backdrop blurs are drawn as nothing because `target`
6306/// can be neither copied nor sampled, once per process.
6307fn note_backdrop_unreadable(target: &wgpu::Texture) {
6308 BACKDROP_UNREADABLE_WARNING.call_once(|| {
6309 log::warn!(
6310 "backdrop blur drawn as nothing: the frame's target ({:?}, {:?}) is neither \
6311 TEXTURE_BINDING nor COPY_SRC in {INTERMEDIATE_FORMAT:?}, so its backdrop cannot be \
6312 read back (logged once)",
6313 target.format(),
6314 target.usage()
6315 );
6316 });
6317}
6318
6319/// What a frame's region copies are recorded with beyond the copy pipeline:
6320/// the instance buffer — one 16-byte slot per copy a frame can run
6321/// ([`crate::filters::MAX_BACKDROPS_PER_FRAME`]), written as each copy is
6322/// recorded — and, for a [`RegionCopyMode::Texture`] frame, the staging
6323/// texture a region is transferred into before the copy pass reads it.
6324#[derive(Debug)]
6325struct RegionCopyResources {
6326 instances: wgpu::Buffer,
6327 /// `COPY_DST | TEXTURE_BINDING` in [`INTERMEDIATE_FORMAT`], grown to the
6328 /// largest region a frame has copied and kept, like the page pool's
6329 /// textures, for the renderer's life. Created by the first texture-mode
6330 /// frame.
6331 staging: Option<ResourceTexture>,
6332}
6333
6334impl RegionCopyResources {
6335 fn new(device: &wgpu::Device) -> Self {
6336 let slots = crate::filters::MAX_BACKDROPS_PER_FRAME as u64;
6337 Self {
6338 instances: device.create_buffer(&wgpu::BufferDescriptor {
6339 label: Some("frust-engine region copy instances"),
6340 size: slots.saturating_mul(COPY_INSTANCE_BYTES),
6341 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
6342 mapped_at_creation: false,
6343 }),
6344 staging: None,
6345 }
6346 }
6347
6348 /// Grows the staging texture to hold a `width` x `height` region, if it
6349 /// does not already.
6350 fn ensure_staging(&mut self, device: &wgpu::Device, width: u32, height: u32) {
6351 if width == 0 || height == 0 {
6352 return;
6353 }
6354 let (held_width, held_height) = self
6355 .staging
6356 .as_ref()
6357 .map_or((0, 0), |staging| (staging.width, staging.height));
6358 if held_width >= width && held_height >= height {
6359 return;
6360 }
6361 self.staging = Some(ResourceTexture::new(
6362 device,
6363 &wgpu::TextureDescriptor {
6364 label: Some("frust-engine region copy staging"),
6365 size: wgpu::Extent3d {
6366 width: width.max(held_width),
6367 height: height.max(held_height),
6368 depth_or_array_layers: 1,
6369 },
6370 mip_level_count: 1,
6371 sample_count: 1,
6372 dimension: wgpu::TextureDimension::D2,
6373 format: INTERMEDIATE_FORMAT,
6374 usage: wgpu::TextureUsages::COPY_DST | wgpu::TextureUsages::TEXTURE_BINDING,
6375 view_formats: &[],
6376 },
6377 ));
6378 }
6379}
6380
6381/// One region copy's full recording state: a region of the frame's own
6382/// target, read back into a pooled page at the page's origin.
6383struct RegionCopyPass<'a> {
6384 mode: RegionCopyMode,
6385 /// The frame's own target.
6386 target: &'a wgpu::TextureView,
6387 /// The page the region lands in.
6388 page: &'a PooledTexture,
6389 /// The device region copied — inside the target by construction
6390 /// ([`crate::filters::backdrop_region`] clamps it).
6391 source: RectU16,
6392 /// [`EnginePipeline::Copy`], on every frame that copies.
6393 pipeline: Option<&'a wgpu::RenderPipeline>,
6394 /// The copies' instance buffer and staging texture, on the same frames.
6395 resources: Option<&'a RegionCopyResources>,
6396 /// This copy's instance slot.
6397 slot: u32,
6398}
6399
6400impl RegionCopyPass<'_> {
6401 /// Records the copy into `encoder`: on a texture-mode frame the region is
6402 /// transferred into the staging texture first, then one pass clears the
6403 /// page and draws the region into it at the page's origin.
6404 ///
6405 /// The page holds the region and transparency around it. A copy whose
6406 /// pipeline, instance buffer or staging texture is missing — which cannot
6407 /// happen, `encode` resolves all three for every frame that copies — still
6408 /// clears the page, and the backdrop blurs nothing into nothing: "draw
6409 /// less, never wrong".
6410 fn record(
6411 &self,
6412 device: &wgpu::Device,
6413 queue: &wgpu::Queue,
6414 encoder: &mut wgpu::CommandEncoder,
6415 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
6416 ) {
6417 let width = self.source.width();
6418 let height = self.source.height();
6419 // The texture the pass reads, and where in it the region starts.
6420 let source = match (self.mode, self.resources) {
6421 (RegionCopyMode::Fragment, Some(_)) => {
6422 Some((self.target, (self.source.x0, self.source.y0)))
6423 }
6424 (RegionCopyMode::Texture, Some(resources)) => match resources.staging.as_ref() {
6425 Some(staging)
6426 if staging.width >= u32::from(width) && staging.height >= u32::from(height) =>
6427 {
6428 encoder.copy_texture_to_texture(
6429 wgpu::TexelCopyTextureInfo {
6430 texture: self.target.texture(),
6431 mip_level: 0,
6432 origin: wgpu::Origin3d {
6433 x: u32::from(self.source.x0),
6434 y: u32::from(self.source.y0),
6435 z: 0,
6436 },
6437 aspect: wgpu::TextureAspect::All,
6438 },
6439 staging.copy_target(),
6440 wgpu::Extent3d {
6441 width: u32::from(width),
6442 height: u32::from(height),
6443 depth_or_array_layers: 1,
6444 },
6445 );
6446 Some((&staging.view, (0, 0)))
6447 }
6448 _ => None,
6449 },
6450 _ => None,
6451 };
6452
6453 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
6454 label: Some(FramePass::Copy.label()),
6455 color_attachments: &[Some(color_attachment(
6456 self.page.view(),
6457 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
6458 ))],
6459 depth_stencil_attachment: None,
6460 timestamp_writes: timestamps,
6461 occlusion_query_set: None,
6462 multiview_mask: None,
6463 });
6464 let (Some((view, origin)), Some(pipeline), Some(resources)) =
6465 (source, self.pipeline, self.resources)
6466 else {
6467 return;
6468 };
6469 let offset = u64::from(self.slot).saturating_mul(COPY_INSTANCE_BYTES);
6470 if offset.saturating_add(COPY_INSTANCE_BYTES) > resources.instances.size() {
6471 return;
6472 }
6473 let (page_width, page_height) = self.page.size();
6474 let instance = [
6475 pack_u16_pair(0, 0),
6476 pack_u16_pair(origin.0, origin.1),
6477 pack_u16_pair(width, height),
6478 pack_u16_pair(
6479 u16::try_from(page_width).unwrap_or(u16::MAX),
6480 u16::try_from(page_height).unwrap_or(u16::MAX),
6481 ),
6482 ];
6483 queue.write_buffer(
6484 &resources.instances,
6485 offset,
6486 bytemuck::cast_slice(&instance),
6487 );
6488 let bind_group = device.create_bind_group(&wgpu::BindGroupDescriptor {
6489 label: Some("frust-engine region copy source"),
6490 layout: &pipeline.get_bind_group_layout(0),
6491 entries: &[wgpu::BindGroupEntry {
6492 binding: 0,
6493 resource: wgpu::BindingResource::TextureView(view),
6494 }],
6495 });
6496 pass.set_pipeline(pipeline);
6497 pass.set_vertex_buffer(0, resources.instances.slice(..));
6498 pass.set_bind_group(0, &bind_group, &[]);
6499 pass.draw(
6500 0..COPY_QUAD_VERTICES,
6501 self.slot..self.slot.saturating_add(1),
6502 );
6503 }
6504}
6505
6506/// Group 0 of a strip pass: the frame's coverage, the pass's own viewport
6507/// uniform, and the texture a composite reads its finished page from.
6508///
6509/// Built against one pipeline's group layout. Every strip pipeline names the
6510/// one explicit strip layout ([`crate::gpu::bindings::strip_pipeline_layout`]),
6511/// so the group binds under every strip state and paint class alike — which is
6512/// what lets one pass switch class pipelines between runs without rebinding.
6513fn resources_bind_group(
6514 device: &wgpu::Device,
6515 pipeline: &wgpu::RenderPipeline,
6516 alphas: &wgpu::TextureView,
6517 config: &wgpu::Buffer,
6518 layer_input: &wgpu::TextureView,
6519) -> wgpu::BindGroup {
6520 device.create_bind_group(&wgpu::BindGroupDescriptor {
6521 label: Some("frust-engine strip resources"),
6522 layout: &pipeline.get_bind_group_layout(0),
6523 entries: &[
6524 wgpu::BindGroupEntry {
6525 binding: 0,
6526 resource: wgpu::BindingResource::TextureView(alphas),
6527 },
6528 wgpu::BindGroupEntry {
6529 binding: 1,
6530 resource: config.as_entire_binding(),
6531 },
6532 wgpu::BindGroupEntry {
6533 binding: 2,
6534 resource: wgpu::BindingResource::TextureView(layer_input),
6535 },
6536 ],
6537 })
6538}
6539
6540// ---------------------------------------------------------------------------
6541// Command ranges (render-model Phase 4 overlay spike)
6542// ---------------------------------------------------------------------------
6543
6544/// How [`project_command_range`] treats the commands it is asked to carry
6545/// into a range — the one thing that differs between the renderer a range is
6546/// projected for.
6547#[derive(Debug, Clone, Copy, PartialEq, Eq)]
6548pub struct RangeOptions {
6549 /// Keep a `DrawRetained` inside the range. A renderer with a retained
6550 /// store behind it keeps them; one without (an overlay renderer built with
6551 /// retention off, whose compiler refuses a frame carrying one) has them
6552 /// dropped and counted in [`RangeCut::dropped_draw_retained`] instead.
6553 pub keep_draw_retained: bool,
6554}
6555
6556/// What cutting a command range out of a scene did — the bracket rule of the
6557/// overlay spike, reported so a caller (and a test) can see what a cut cost.
6558///
6559/// See [`project_command_range`] for the rule itself.
6560#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
6561pub struct RangeCut {
6562 /// The first command index the projection carried, after clamping.
6563 pub start: usize,
6564 /// One past the last command index the projection carried, after
6565 /// clamping.
6566 pub end: usize,
6567 /// Brackets open at `start` that were reopened ahead of the range.
6568 pub reopened: usize,
6569 /// Brackets still open at `end` that were closed after the range.
6570 pub closed: usize,
6571 /// `PushRetained` brackets spanning a cut, painted inline (their
6572 /// presentation scale folded into the body's transforms and their alpha
6573 /// as a plain layer) rather than recorded as a page.
6574 pub demoted_retained: usize,
6575 /// `DrawRetained`s dropped under `keep_draw_retained: false`.
6576 pub dropped_draw_retained: usize,
6577 /// Pops inside the range with no open bracket of their own kind — already
6578 /// ignored by every renderer, so dropped rather than carried.
6579 pub dropped_pops: usize,
6580}
6581
6582/// The bracket families the display list closes with a pop of their own.
6583#[derive(Debug, Clone, Copy, PartialEq, Eq)]
6584enum BracketKind {
6585 Clip,
6586 Layer,
6587 Retained,
6588}
6589
6590/// The bracket kind `command` opens, if it opens one.
6591fn opens(command: &frust_scene::Command) -> Option<BracketKind> {
6592 use frust_scene::Command;
6593 match command {
6594 Command::PushClip { .. }
6595 | Command::PushClipRounded { .. }
6596 | Command::PushClipPath { .. } => Some(BracketKind::Clip),
6597 Command::PushLayer { .. } => Some(BracketKind::Layer),
6598 Command::PushRetained { .. } => Some(BracketKind::Retained),
6599 _ => None,
6600 }
6601}
6602
6603/// The bracket kind `command` closes, if it is a pop.
6604fn closes(command: &frust_scene::Command) -> Option<BracketKind> {
6605 use frust_scene::Command;
6606 match command {
6607 Command::PopClip => Some(BracketKind::Clip),
6608 Command::PopLayer => Some(BracketKind::Layer),
6609 Command::PopRetained => Some(BracketKind::Retained),
6610 _ => None,
6611 }
6612}
6613
6614/// One bracket the projection holds open, and how it will be closed.
6615#[derive(Debug, Clone, Copy)]
6616enum OpenBracket {
6617 /// A clip or layer carried as recorded; closed with its own pop.
6618 Plain(BracketKind),
6619 /// A `PushRetained` carried as recorded; closed with `PopRetained`.
6620 Retained,
6621 /// A `PushRetained` spanning a cut, painted inline. `layer` says whether
6622 /// its alpha opened a layer to close; `prefix` is the correction in force
6623 /// before it, restored when it closes.
6624 Demoted { layer: bool, prefix: Affine },
6625 /// A `PushRetained` nested inside a demoted bracket: its presentation is
6626 /// ignored (only the outermost bracket's is honoured), so it opened
6627 /// nothing and closes nothing.
6628 Absorbed,
6629}
6630
6631impl OpenBracket {
6632 fn kind(self) -> BracketKind {
6633 match self {
6634 OpenBracket::Plain(kind) => kind,
6635 OpenBracket::Retained | OpenBracket::Demoted { .. } | OpenBracket::Absorbed => {
6636 BracketKind::Retained
6637 }
6638 }
6639 }
6640}
6641
6642/// Records `command` into `builder` with `prefix` composed ahead of its own
6643/// transform — the identity outside a demoted retained bracket, its
6644/// presentation correction inside one.
6645///
6646/// The builder's own transform stack is left as found: every arm pushes the
6647/// command's transform, records, and pops, so the recorded transform is
6648/// exactly `prefix * transform`.
6649fn replay(builder: &mut SceneBuilder<'_>, prefix: Affine, command: &frust_scene::Command) {
6650 use frust_scene::{Command, PathStyle};
6651 fn under(
6652 builder: &mut SceneBuilder<'_>,
6653 transform: Affine,
6654 record: impl FnOnce(&mut SceneBuilder<'_>),
6655 ) {
6656 builder.push_transform(transform);
6657 record(builder);
6658 builder.pop_transform();
6659 }
6660 match command {
6661 Command::FillRect {
6662 rect,
6663 brush,
6664 transform,
6665 } => under(builder, prefix * *transform, |b| {
6666 b.fill_rect(*rect, brush.clone())
6667 }),
6668 Command::RoundedRect {
6669 rect,
6670 radii,
6671 brush,
6672 transform,
6673 } => under(builder, prefix * *transform, |b| {
6674 b.fill_rounded_rect_radii(*rect, *radii, brush.clone());
6675 }),
6676 Command::Line {
6677 p0,
6678 p1,
6679 width,
6680 brush,
6681 transform,
6682 } => under(builder, prefix * *transform, |b| {
6683 b.stroke_line(*p0, *p1, *width, brush.clone());
6684 }),
6685 // A glyph run composes the builder's transform ahead of its own.
6686 Command::GlyphRun(run) => under(builder, prefix, |b| b.draw_glyph_run(run.clone())),
6687 Command::PushClip { rect, transform } => {
6688 under(builder, prefix * *transform, |b| b.push_clip(*rect));
6689 }
6690 Command::PushClipRounded {
6691 rect,
6692 radii,
6693 transform,
6694 } => under(builder, prefix * *transform, |b| {
6695 b.push_clip_rounded_radii(*rect, *radii);
6696 }),
6697 Command::PushClipPath { path, transform } => {
6698 under(builder, prefix * *transform, |b| {
6699 b.push_clip_path(path.clone())
6700 });
6701 }
6702 Command::PopClip => builder.pop_clip(),
6703 Command::Image {
6704 data,
6705 dest,
6706 transform,
6707 } => under(builder, prefix * *transform, |b| b.draw_image(data, *dest)),
6708 Command::BlurredRoundedRect {
6709 rect,
6710 radii,
6711 std_dev,
6712 color,
6713 transform,
6714 } => under(builder, prefix * *transform, |b| {
6715 b.draw_blurred_rounded_rect_radii(*rect, *radii, *std_dev, *color);
6716 }),
6717 Command::PushLayer {
6718 rect,
6719 alpha,
6720 transform,
6721 } => under(builder, prefix * *transform, |b| {
6722 b.push_layer(*rect, *alpha)
6723 }),
6724 Command::PopLayer => builder.pop_layer(),
6725 Command::ClearRect { rect, transform } => {
6726 under(builder, prefix * *transform, |b| b.clear_rect(*rect));
6727 }
6728 Command::Path {
6729 path,
6730 style,
6731 brush,
6732 transform,
6733 } => under(builder, prefix * *transform, |b| match style {
6734 PathStyle::Fill => b.fill_path(path.clone(), brush.clone()),
6735 PathStyle::Stroke { width, dash: None } => {
6736 b.stroke_path(path.clone(), *width, brush.clone());
6737 }
6738 PathStyle::Stroke {
6739 width,
6740 dash: Some(dash),
6741 } => b.stroke_path_dashed(path.clone(), *width, *dash, brush.clone()),
6742 }),
6743 Command::ShaderQuad {
6744 program,
6745 dest,
6746 transform,
6747 time,
6748 } => under(builder, prefix * *transform, |b| {
6749 b.draw_shader(program, *dest, *time);
6750 }),
6751 Command::PushRetained {
6752 key,
6753 rect,
6754 alpha,
6755 scale,
6756 transform,
6757 } => under(builder, prefix * *transform, |b| {
6758 b.push_retained(*key, *rect, *alpha, *scale);
6759 }),
6760 Command::PopRetained => builder.pop_retained(),
6761 Command::DrawRetained {
6762 key,
6763 rect,
6764 transform,
6765 alpha,
6766 scale,
6767 } => under(builder, prefix * *transform, |b| {
6768 b.draw_retained(*key, *rect, *alpha, *scale);
6769 }),
6770 Command::SceneTexture {
6771 id,
6772 dest,
6773 transform,
6774 } => {
6775 under(builder, prefix * *transform, |b| {
6776 b.scene_texture(*id, *dest)
6777 });
6778 }
6779 Command::BackdropBlur {
6780 rect,
6781 radius,
6782 transform,
6783 } => under(builder, prefix * *transform, |b| {
6784 b.backdrop_blur(*rect, *radius)
6785 }),
6786 }
6787}
6788
6789/// Writes the commands of `scene` in `range` into `out` (reset first) as a
6790/// self-contained display list, and reports what the cut did — the overlay
6791/// spike's range projection (render-model Phase 4, decision D10).
6792///
6793/// `range` is clamped to the scene: an `end` past the last command stops at
6794/// it, and a `start` past `end` projects nothing. The projection is a scene,
6795/// not a compiled-frame filter: the compiled frame's draws are ordered by
6796/// depth and grouped into rounds, while a command index is the one paint-order
6797/// position every producer of a cut (a platform view's place in the display
6798/// list) can name.
6799///
6800/// # The bracket rule
6801///
6802/// A cut can fall inside a bracket. The rule this spike chose:
6803///
6804/// - **Clips and layers spanning a cut are closed at it and reopened across
6805/// it.** Every clip or layer open at `start` is re-recorded, in its original
6806/// push order and with its original geometry, ahead of the range; every one
6807/// still open at `end` is popped after it, innermost first. Both halves of a
6808/// split clip therefore clip to the same shape, and both halves of a split
6809/// layer composite at the same alpha — which is exact for a clip, and for a
6810/// layer differs from the uncut frame only where content on both sides of
6811/// the cut overlaps inside it (each half is composited on its own).
6812/// - **A retained bracket spanning a cut is demoted to inline paint on both
6813/// sides.** A page recorded from half a body would be a stale page the
6814/// ledger then advertises, so neither side records one: the bracket's
6815/// presentation scale is folded into every body command's transform (the
6816/// same conjugation `crate::compile::layers::retained_correction` composes
6817/// ahead of the frame root) and its alpha below one becomes a plain layer —
6818/// exactly the lowering `frust_scene` prescribes for a renderer that does
6819/// not retain. A retained bracket nested inside a demoted one contributes
6820/// nothing, as a nested bracket's presentation never does. A retained
6821/// bracket wholly inside the range is carried as recorded.
6822/// - **A pop with no open bracket of its kind is dropped** (every renderer
6823/// ignores one anyway), and a `DrawRetained` is dropped when
6824/// `options.keep_draw_retained` is off.
6825///
6826/// Pops match the most recently opened bracket of their own kind, which is
6827/// the stack discipline of every well-nested display list.
6828///
6829/// Every carried command is re-recorded through a [`SceneBuilder`] with its
6830/// transform composed under the identity (or a demoted bracket's correction),
6831/// so geometry and paints are carried verbatim; the cost is one clone of each
6832/// carried command's payload, paid only by a caller that projects.
6833pub fn project_command_range(
6834 scene: &Scene,
6835 range: Range<usize>,
6836 options: RangeOptions,
6837 out: &mut Scene,
6838) -> RangeCut {
6839 project_command_ranges(scene, std::slice::from_ref(&range), options, out)
6840}
6841
6842/// [`project_command_range`] over several ranges at once: writes the
6843/// projection of each range of `ranges`, in the order given, one after the
6844/// other into `out` (reset first), and reports the cuts summed — the segmented
6845/// frame's way of drawing the commands *between* overlay cuts on the main
6846/// surface (render-model Phase 4, decision D10).
6847///
6848/// Each range is projected exactly as [`project_command_range`] projects it
6849/// alone — clamped to the scene, its open clips and layers reopened ahead of
6850/// it and closed after it, a retained bracket spanning either of its ends
6851/// demoted to inline paint — so every range is self-contained and the bracket
6852/// rule holds per range. A clip open across the gap between two ranges is
6853/// therefore closed at the end of the first and reopened at the start of the
6854/// second, never left open over the commands the gap removed.
6855///
6856/// The caller passes the ranges ascending and disjoint (the complement of a
6857/// frame's overlay cuts is); overlapping ranges are projected as given, each
6858/// command carried once per range it falls in. An empty range still reopens
6859/// and closes the brackets open at its start (drawing nothing), so a caller
6860/// that wants no trace of it leaves it out.
6861///
6862/// The returned [`RangeCut`]'s `start` is the first range's clamped start and
6863/// its `end` the last range's clamped end (both `0` for no range at all); its
6864/// counters are the sums over every range.
6865pub fn project_command_ranges(
6866 scene: &Scene,
6867 ranges: &[Range<usize>],
6868 options: RangeOptions,
6869 out: &mut Scene,
6870) -> RangeCut {
6871 let commands = scene.commands();
6872 let len = commands.len();
6873 let clamped: Vec<(usize, usize)> = ranges
6874 .iter()
6875 .map(|range| {
6876 let end = range.end.min(len);
6877 (range.start.min(end), end)
6878 })
6879 .collect();
6880 let mut cut = RangeCut {
6881 start: clamped.first().map_or(0, |&(start, _)| start),
6882 end: clamped.last().map_or(0, |&(_, end)| end),
6883 ..RangeCut::default()
6884 };
6885
6886 // Pass one, over the whole scene: where each bracket closes, and which
6887 // brackets are open at each range's start (in push order).
6888 let mut closed_at: Vec<Option<usize>> = vec![None; len];
6889 let mut stack: Vec<(BracketKind, usize)> = Vec::new();
6890 let mut open_at_start: Vec<Vec<usize>> = vec![Vec::new(); clamped.len()];
6891 let snapshot = |index: usize, stack: &[(BracketKind, usize)], open: &mut [Vec<usize>]| {
6892 for (slot, &(start, _)) in open.iter_mut().zip(&clamped) {
6893 if start == index {
6894 slot.extend(stack.iter().map(|&(_, at)| at));
6895 }
6896 }
6897 };
6898 for (index, command) in commands.iter().enumerate() {
6899 snapshot(index, &stack, &mut open_at_start);
6900 if let Some(kind) = opens(command) {
6901 stack.push((kind, index));
6902 } else if let Some(kind) = closes(command)
6903 && let Some(position) = stack.iter().rposition(|&(open, _)| open == kind)
6904 {
6905 let (_, at) = stack.remove(position);
6906 closed_at[at] = Some(index);
6907 }
6908 }
6909 snapshot(len, &stack, &mut open_at_start);
6910 // A retained bracket spans a cut of the range `start..end` when it opened
6911 // before the range, or is still open where the range ends short of the
6912 // scene's own end.
6913 let spans = |at: usize, start: usize, end: usize| {
6914 at < start || (end < len && closed_at[at].is_none_or(|c| c >= end))
6915 };
6916
6917 out.reset();
6918 let mut builder = SceneBuilder::new(out);
6919
6920 let push = |at: usize,
6921 (start, end): (usize, usize),
6922 builder: &mut SceneBuilder<'_>,
6923 open: &mut Vec<OpenBracket>,
6924 prefix: &mut Affine,
6925 cut: &mut RangeCut| {
6926 let command = &commands[at];
6927 match command {
6928 frust_scene::Command::PushRetained {
6929 rect,
6930 alpha,
6931 scale,
6932 transform,
6933 ..
6934 } => {
6935 if open
6936 .iter()
6937 .any(|bracket| matches!(bracket, OpenBracket::Demoted { .. }))
6938 {
6939 open.push(OpenBracket::Absorbed);
6940 } else if spans(at, start, end) {
6941 cut.demoted_retained += 1;
6942 let correction =
6943 crate::compile::layers::retained_correction(*rect, *scale, *transform);
6944 let layer = *alpha < 1.0;
6945 if layer {
6946 let corrected = *prefix * correction * *transform;
6947 builder.push_transform(corrected);
6948 builder.push_layer(*rect, *alpha);
6949 builder.pop_transform();
6950 }
6951 open.push(OpenBracket::Demoted {
6952 layer,
6953 prefix: *prefix,
6954 });
6955 *prefix *= correction;
6956 } else {
6957 replay(builder, *prefix, command);
6958 open.push(OpenBracket::Retained);
6959 }
6960 }
6961 _ => {
6962 if let Some(kind) = opens(command) {
6963 replay(builder, *prefix, command);
6964 open.push(OpenBracket::Plain(kind));
6965 }
6966 }
6967 }
6968 };
6969 let close =
6970 |bracket: OpenBracket, builder: &mut SceneBuilder<'_>, prefix: &mut Affine| match bracket {
6971 OpenBracket::Plain(BracketKind::Clip) => builder.pop_clip(),
6972 OpenBracket::Plain(BracketKind::Layer) => builder.pop_layer(),
6973 OpenBracket::Plain(BracketKind::Retained) | OpenBracket::Retained => {
6974 builder.pop_retained();
6975 }
6976 OpenBracket::Demoted {
6977 layer,
6978 prefix: before,
6979 } => {
6980 if layer {
6981 builder.pop_layer();
6982 }
6983 *prefix = before;
6984 }
6985 OpenBracket::Absorbed => {}
6986 };
6987
6988 let mut open: Vec<OpenBracket> = Vec::new();
6989 for (&bounds, reopen) in clamped.iter().zip(&open_at_start) {
6990 let (start, end) = bounds;
6991 // Every range starts from the identity with nothing open: the one
6992 // before it closed everything it opened.
6993 let mut prefix = Affine::IDENTITY;
6994 for &at in reopen {
6995 cut.reopened += 1;
6996 push(at, bounds, &mut builder, &mut open, &mut prefix, &mut cut);
6997 }
6998 for (index, command) in commands.iter().enumerate().take(end).skip(start) {
6999 if opens(command).is_some() {
7000 push(
7001 index,
7002 bounds,
7003 &mut builder,
7004 &mut open,
7005 &mut prefix,
7006 &mut cut,
7007 );
7008 } else if let Some(kind) = closes(command) {
7009 match open.iter().rposition(|bracket| bracket.kind() == kind) {
7010 Some(position) => {
7011 let bracket = open.remove(position);
7012 close(bracket, &mut builder, &mut prefix);
7013 }
7014 None => cut.dropped_pops += 1,
7015 }
7016 } else if matches!(command, frust_scene::Command::DrawRetained { .. })
7017 && !options.keep_draw_retained
7018 {
7019 cut.dropped_draw_retained += 1;
7020 } else {
7021 replay(&mut builder, prefix, command);
7022 }
7023 }
7024 while let Some(bracket) = open.pop() {
7025 cut.closed += 1;
7026 close(bracket, &mut builder, &mut prefix);
7027 }
7028 }
7029 cut
7030}
7031
7032/// The outermost `PushRetained` bracket still open just before command
7033/// `index` of `commands` — its command index — or `None` when no retained
7034/// bracket is open there.
7035///
7036/// Pops are matched the way [`project_command_range`] matches them: a
7037/// `PopRetained` closes the most recently opened retained bracket, whatever
7038/// clips or layers were opened in between, and one with no retained bracket
7039/// open is ignored. An `index` past the end asks about the end of the list,
7040/// so a bracket never closed is open there. A range starting at `index`
7041/// splits the bracket answered here: [`project_command_range`] demotes it to
7042/// inline paint on both sides of the cut. Read-only and one pass over
7043/// `commands[..index]`.
7044pub fn open_retained_bracket(commands: &[frust_scene::Command], index: usize) -> Option<usize> {
7045 let mut open: Vec<usize> = Vec::new();
7046 for (at, command) in commands.iter().enumerate().take(index) {
7047 match command {
7048 frust_scene::Command::PushRetained { .. } => open.push(at),
7049 frust_scene::Command::PopRetained => {
7050 open.pop();
7051 }
7052 _ => {}
7053 }
7054 }
7055 open.first().copied()
7056}
7057
7058#[cfg(test)]
7059mod tests {
7060 use super::*;
7061 use frust_gpu::DownlevelProfile;
7062 use kurbo::Rect;
7063 use peniko::color::palette::css::{BLUE, RED};
7064 use std::collections::BTreeMap;
7065 use vello_common::geometry::RectU16;
7066 use vello_common::paint::IndexedPaint;
7067
7068 /// The `frust-perf enc` line is a contract: a capture is graded by
7069 /// grepping its fields, so the field order and the names are pinned here
7070 /// rather than only by the formatter. `perf-trace`-only, with the line —
7071 /// `cargo test -p frust-engine --features perf-trace` is where it runs.
7072 #[cfg(feature = "perf-trace")]
7073 #[test]
7074 fn an_encode_trace_line_reports_every_column_in_order() {
7075 let mut trace = EncodeTrace {
7076 frames: 7,
7077 ..EncodeTrace::default()
7078 };
7079 // One row per column, each carrying a distinct value in its own
7080 // column, so a transposed or dropped field shows up as a wrong number
7081 // rather than only as a wrong name.
7082 let mut row = [0_u32; ENCODE_TRACE_ROW];
7083 for (column, slot) in row.iter_mut().enumerate() {
7084 *slot = (column as u32 + 1) * 1_000;
7085 }
7086 trace.window.push(row);
7087
7088 let line = trace.line();
7089 let phases: Vec<String> = ENCODE_TRACE_COLUMNS
7090 .iter()
7091 .enumerate()
7092 .map(|(column, name)| format!("{name}_us={}.0", column + 1))
7093 .collect();
7094 // The counts carry no unit suffix and are reported as the raw values
7095 // the row holds, not scaled to microseconds like the phases above.
7096 let counts: Vec<String> = ENCODE_TRACE_COUNTS
7097 .iter()
7098 .enumerate()
7099 .map(|(offset, name)| {
7100 format!(
7101 "{name}={}",
7102 (ENCODE_TRACE_COLUMNS.len() + offset + 1) * 1_000
7103 )
7104 })
7105 .collect();
7106 assert_eq!(
7107 line,
7108 format!(
7109 "frust-perf enc n=7 w=1 {} total_p95_us={}.0 {}",
7110 phases.join(" "),
7111 ENCODE_TRACE_COLUMNS.len(),
7112 counts.join(" "),
7113 ),
7114 );
7115 }
7116
7117 /// The event counts are sums over the window, the level counts medians,
7118 /// and the pinned order puts them where the contract says.
7119 #[cfg(feature = "perf-trace")]
7120 #[test]
7121 fn an_encode_trace_line_sums_events_and_takes_the_median_of_levels() {
7122 let mut trace = EncodeTrace::default();
7123 let base = ENCODE_TRACE_COLUMNS.len();
7124 for frame in 0..3_u32 {
7125 let mut row = [0_u32; ENCODE_TRACE_ROW];
7126 row[base] = 10 + frame; // draws: median 11
7127 row[base + 5] = 1; // replays: sum 3
7128 row[base + 6] = 2; // dirty_pages: sum 6
7129 row[base + 8] = frame; // glyph_misses: sum 3
7130 row[base + 10] = 4 + frame; // run_keys: median 5
7131 trace.window.push(row);
7132 }
7133 let line = trace.line();
7134 let tail = line.split(" total_p95_us=").nth(1).unwrap();
7135 let counts = tail.split_once(' ').unwrap().1;
7136 assert_eq!(
7137 counts,
7138 "draws=11 strips=0 alphas=0 glyph_draws=0 atlas_glyphs=0 replays=3 \
7139 dirty_pages=6 atlas_refused=0 glyph_misses=3 glyph_evicted=0 run_keys=5"
7140 );
7141 }
7142
7143 /// A window's percentile is nearest-rank over the column it names, so the
7144 /// value reported is one the window really contains.
7145 #[cfg(feature = "perf-trace")]
7146 #[test]
7147 fn an_encode_trace_percentile_is_nearest_rank_per_column() {
7148 let mut trace = EncodeTrace::default();
7149 for value in [4_000_u32, 1_000, 3_000, 2_000] {
7150 let mut row = [0_u32; ENCODE_TRACE_ROW];
7151 row[0] = value;
7152 trace.window.push(row);
7153 }
7154 assert_eq!(trace.percentile(0, 50), 2_000);
7155 assert_eq!(trace.percentile(0, 95), 4_000);
7156 // An empty window reports zero rather than reaching past its end.
7157 trace.window.clear();
7158 assert_eq!(trace.percentile(0, 50), 0);
7159 }
7160
7161 /// Nanoseconds round up to a tenth of a microsecond, so a phase that cost
7162 /// anything at all never reports as free.
7163 #[cfg(feature = "perf-trace")]
7164 #[test]
7165 fn a_span_that_cost_anything_never_formats_as_zero() {
7166 assert_eq!(format_us(0), "0.0");
7167 assert_eq!(format_us(1), "0.1");
7168 assert_eq!(format_us(4_170_000), "4170.0");
7169 }
7170
7171 /// The compile phases partition the compile they sit inside, so their sum
7172 /// is the whole call minus the overhead the renderer measures around it.
7173 #[test]
7174 fn compile_phases_sum_without_overflowing() {
7175 let spans = CompileSpans {
7176 validate: Duration::from_micros(10),
7177 prepare: Duration::from_micros(20),
7178 classify: Duration::from_micros(30),
7179 admit: Duration::from_micros(40),
7180 walk: Duration::from_micros(50),
7181 // A subset of `walk`, so the sum must not grow by it.
7182 glyphs: Duration::from_micros(45),
7183 finish: Duration::from_micros(60),
7184 };
7185 assert_eq!(spans.total(), Duration::from_micros(210));
7186 assert_eq!(
7187 CompileSpans {
7188 validate: Duration::MAX,
7189 walk: Duration::MAX,
7190 ..CompileSpans::default()
7191 }
7192 .total(),
7193 Duration::MAX,
7194 );
7195 }
7196
7197 #[test]
7198 fn a_target_past_the_u16_grid_is_refused() {
7199 assert_eq!(
7200 grid_size(1920, 1080).expect("a normal surface"),
7201 (1920, 1080)
7202 );
7203 assert_eq!(
7204 grid_size(u32::from(u16::MAX), 16).expect("the grid ceiling itself"),
7205 (u16::MAX, 16)
7206 );
7207 assert!(matches!(
7208 grid_size(u32::from(u16::MAX) + 1, 16),
7209 Err(EngineError::TargetTooLarge)
7210 ));
7211 assert!(matches!(
7212 grid_size(16, u32::from(u16::MAX) + 1),
7213 Err(EngineError::TargetTooLarge)
7214 ));
7215 }
7216
7217 #[test]
7218 fn a_premultiplied_clear_folds_alpha_into_the_colour() {
7219 let half = RED.with_alpha(0.5);
7220 let premultiplied = clear_color(half, OutputAlpha::Premultiplied);
7221 let straight = clear_color(half, OutputAlpha::Straight);
7222
7223 assert!((premultiplied.r - 0.5).abs() < 1e-6);
7224 assert!((premultiplied.a - 0.5).abs() < 1e-6);
7225 assert!((straight.r - 1.0).abs() < 1e-6);
7226 assert!((straight.a - 0.5).abs() < 1e-6);
7227
7228 // An opaque colour is identical under both conventions.
7229 assert_eq!(
7230 clear_color(BLUE, OutputAlpha::Premultiplied),
7231 clear_color(BLUE, OutputAlpha::Straight)
7232 );
7233 }
7234
7235 #[test]
7236 fn a_solid_paint_travels_premultiplied_in_the_instance_payload() {
7237 let paint = pack_paint(&Paint::from(RED), 3, &[]).expect("a solid paint always resolves");
7238 let values = paint.values_at(40, 12);
7239 assert_eq!(values.paint, SOLID_PAINT);
7240 assert_eq!(values.depth_index, 3);
7241 assert_eq!(values.payload, RED.premultiply().to_rgba8().to_u32());
7242 assert!(paint.opaque);
7243 assert_eq!(
7244 paint.values_at(0, 0).payload,
7245 values.payload,
7246 "a solid colour is the same wherever it is stamped"
7247 );
7248
7249 let paint = pack_paint(&Paint::from(RED.with_alpha(0.5)), 0, &[])
7250 .expect("a translucent solid paint still resolves");
7251 assert!(
7252 !paint.opaque,
7253 "a translucent paint never reaches the opaque pass"
7254 );
7255 }
7256
7257 #[test]
7258 fn an_unresolvable_indexed_paint_skips_its_draw() {
7259 let indexed = Paint::Indexed(IndexedPaint::new(0));
7260 assert!(
7261 pack_paint(&indexed, 0, &[]).is_none(),
7262 "an index past the frame's slots resolves to nothing"
7263 );
7264 assert!(
7265 pack_paint(&indexed, 0, &[None]).is_none(),
7266 "so does a slot the lowering refused"
7267 );
7268 }
7269
7270 /// An image record is classed by the path it samples through — the one
7271 /// split an instance's flag cannot make — and every other record by its
7272 /// paint type.
7273 #[test]
7274 fn a_lowered_record_names_its_strip_program_class() {
7275 use crate::gpu::atlas::pack_image_params;
7276 use crate::gpu::paint_texture::GpuEncodedImage;
7277
7278 let image = |quality: u32, external: bool| {
7279 GpuEncodedPaint::Image(GpuEncodedImage {
7280 image_params: pack_image_params(quality, 0, 0, 3, external),
7281 image_size: 0,
7282 image_offset: 0,
7283 transform: [1.0, 0.0, 0.0, 1.0, 0.0, 0.0],
7284 tint: 0,
7285 tint_mode: 0,
7286 image_padding: 0,
7287 })
7288 };
7289 assert_eq!(class_of_record(&image(0, false)), PaintClass::Image);
7290 assert_eq!(class_of_record(&image(1, false)), PaintClass::Image);
7291 assert_eq!(class_of_record(&image(2, false)), PaintClass::ImageFiltered);
7292 assert_eq!(class_of_record(&image(1, true)), PaintClass::ImageFiltered);
7293 }
7294
7295 #[test]
7296 fn a_resolved_indexed_paint_samples_at_each_instances_own_position() {
7297 let slots = [Some(ResolvedPaint {
7298 paint_type: PaintType::LinearGradient,
7299 texel_offset: 6,
7300 opaque: true,
7301 external: None,
7302 class: PaintClass::Gradient,
7303 })];
7304 let paint = pack_paint(&Paint::Indexed(IndexedPaint::new(0)), 2, &slots)
7305 .expect("a lowered paint resolves");
7306
7307 assert_eq!(
7308 paint.paint,
7309 pack_paint_descriptor(PaintType::LinearGradient, 6)
7310 );
7311 assert_eq!(paint.depth_index, 2);
7312 // The payload is the instance's scene origin, not a colour, so two
7313 // instances of the same draw carry different payloads.
7314 assert_eq!(paint.payload_at(8, 4), 8 | (4 << 16));
7315 assert_eq!(paint.payload_at(40, 4), 40 | (4 << 16));
7316 }
7317
7318 #[test]
7319 fn a_u16_pair_packs_low_half_first() {
7320 assert_eq!(pack_u16_pair(0, 0), 0);
7321 assert_eq!(pack_u16_pair(1, 0), 1);
7322 assert_eq!(pack_u16_pair(0, 1), 1 << 16);
7323 assert_eq!(pack_u16_pair(u16::MAX, u16::MAX), u32::MAX);
7324 }
7325
7326 #[test]
7327 fn the_minimum_resource_dimension_keeps_every_upload_row_legal() {
7328 // The gradient LUT is the narrowest resource at four bytes per texel,
7329 // so it is the one that fixes the floor.
7330 assert_eq!(
7331 MIN_RESOURCE_TEXTURE_DIM * BYTES_PER_TEXEL,
7332 wgpu::COPY_BYTES_PER_ROW_ALIGNMENT
7333 );
7334 assert!(MIN_RESOURCE_TEXTURE_DIM.is_power_of_two());
7335 }
7336
7337 // -----------------------------------------------------------------
7338 // Where the hole punch lands in the pass plan
7339 //
7340 // What a frame's pass plan COSTS, and where the erase sits inside it,
7341 // is a decision over the recording's shape — no device, no pixels.
7342 // The pixel half of the same contract is
7343 // `frust-testing`'s `tests/aa_over_punch.rs`, against `vello_cpu`.
7344 // -----------------------------------------------------------------
7345
7346 /// Viewport every plan below is built against.
7347 const PLAN_VIEWPORT: (u16, u16) = (64, 48);
7348
7349 /// The slot, and a chip straddling its right edge — so a chip drawn over
7350 /// the punch is half inside it and half over the backdrop.
7351 const PLAN_SLOT: Rect = Rect::new(4.0, 4.0, 32.0, 44.0);
7352 const PLAN_CHIP: Rect = Rect::new(16.0, 12.0, 56.0, 32.0);
7353
7354 /// The plan `Scratch::build` produces for `scene`, alongside the rounds it
7355 /// was scheduled from, under a target that carries alpha.
7356 fn plan_of(scene: &Scene) -> (Scratch, Vec<Round>) {
7357 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
7358 .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
7359 .expect("an in-range scene compiles");
7360 let rounds = Schedule::build(
7361 &frame.recorder,
7362 &TierCaps::fake(DownlevelProfile::Full),
7363 &PageConfig::default(),
7364 )
7365 .expect("a scene of solid fills schedules");
7366
7367 let mut scratch = Scratch::default();
7368 // Depth on and punching on: the shape a translucent presentation
7369 // carrying a platform-view slot is planned under.
7370 scratch.build(&frame, &rounds, true, !frame.clears.is_empty(), &[], &[]);
7371 (scratch, rounds)
7372 }
7373
7374 /// A scene recorded through the public builder.
7375 fn plan_scene(record: impl FnOnce(&mut SceneBuilder<'_>)) -> Scene {
7376 let mut scene = Scene::new();
7377 let mut builder = SceneBuilder::new(&mut scene);
7378 record(&mut builder);
7379 scene
7380 }
7381
7382 fn plan_backdrop(builder: &mut SceneBuilder<'_>) {
7383 builder.fill_rect(
7384 Rect::new(
7385 0.0,
7386 0.0,
7387 f64::from(PLAN_VIEWPORT.0),
7388 f64::from(PLAN_VIEWPORT.1),
7389 ),
7390 Brush::Solid(RED),
7391 );
7392 }
7393
7394 #[test]
7395 fn a_frame_that_punches_nothing_is_planned_exactly_as_its_rounds() {
7396 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
7397 plan_backdrop(builder);
7398 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
7399 }));
7400
7401 assert_eq!(
7402 scratch.rounds.len(),
7403 rounds.len(),
7404 "a frame with no clear is cut nowhere, so it costs exactly its \
7405 scheduled rounds' passes"
7406 );
7407 assert!(!scratch.punches(), "and records no punch pass at all");
7408 }
7409
7410 #[test]
7411 fn a_clear_recorded_last_keeps_its_pass_after_every_round() {
7412 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
7413 plan_backdrop(builder);
7414 builder.clear_rect(PLAN_SLOT);
7415 }));
7416
7417 assert_eq!(
7418 scratch.rounds.len(),
7419 rounds.len(),
7420 "nothing is recorded after the clear, so nothing is cut"
7421 );
7422 assert!(
7423 scratch.rounds.iter().all(|plan| plan.punch.1 == 0),
7424 "no round carries the punch"
7425 );
7426 assert!(
7427 scratch.punch.1 > 0,
7428 "it is issued after the last round instead — the position an \
7429 unconditionally-trailing pass would have put it in, which is why \
7430 a frame shaped like this renders byte-identically"
7431 );
7432 }
7433
7434 #[test]
7435 fn a_draw_over_a_clear_cuts_the_surface_round_and_the_punch_goes_in_the_cut() {
7436 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
7437 plan_backdrop(builder);
7438 builder.clear_rect(PLAN_SLOT);
7439 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
7440 }));
7441
7442 assert_eq!(
7443 scratch.rounds.len(),
7444 rounds.len() + 1,
7445 "the surface round is cut in two — one extra pass, and one only"
7446 );
7447 assert_eq!(
7448 scratch.punch,
7449 (0, 0),
7450 "with nothing left over for a trailing pass"
7451 );
7452
7453 let cut = scratch
7454 .rounds
7455 .iter()
7456 .position(|plan| plan.punch.1 > 0)
7457 .expect("the cut plan carries the punch");
7458 assert_eq!(cut, scratch.rounds.len() - 2, "and it is the cut plan");
7459
7460 // The buffer says the same thing the plan does: the backdrop's
7461 // instances precede the punch's, and the chip's follow them.
7462 let before = scratch.rounds[cut].segments.clone();
7463 let after = scratch.rounds[cut + 1].segments.clone();
7464 let end_of = |range: Range<usize>| {
7465 scratch.segments[range]
7466 .iter()
7467 .map(|segment| match *segment {
7468 Segment::Strips(first, count, _)
7469 | Segment::External(first, count, _, _)
7470 | Segment::Backdrop(first, count, _) => first + count,
7471 Segment::Composite(first, _) => first + 1,
7472 })
7473 .max()
7474 .expect("a plan of this frame draws something")
7475 };
7476 let (punch_first, punch_count) = scratch.rounds[cut].punch;
7477 assert!(
7478 end_of(before) <= punch_first,
7479 "everything recorded before the clear is erased by the punch"
7480 );
7481 assert!(
7482 scratch.segments[after]
7483 .iter()
7484 .all(|segment| match *segment {
7485 Segment::Strips(first, _, _)
7486 | Segment::External(first, _, _, _)
7487 | Segment::Backdrop(first, _, _)
7488 | Segment::Composite(first, _) => first >= punch_first + punch_count,
7489 }),
7490 "and everything recorded after it lands on top of the erase"
7491 );
7492 }
7493
7494 // -----------------------------------------------------------------
7495 // The pass list: what a frame of each shape records, merged and apart
7496 //
7497 // Host-only: `Scratch::plan_passes` states the list over the plan, and
7498 // a debug build holds every frame `record_frame` really records to it,
7499 // so the device suites (`tests/encode_contract.rs` among them) check the
7500 // same list against a real encoder. The filter shape is pinned only here:
7501 // no `Scene` command records a filter layer, so no `encode` call reaches
7502 // a filter round.
7503 // -----------------------------------------------------------------
7504
7505 /// A rectangle off the pixel grid, so its edges are anti-aliased and the
7506 /// draw always reaches a surface round's blended strips — never only the
7507 /// opaque ones.
7508 const PLAN_SOFT: Rect = Rect::new(8.5, 8.5, 24.5, 40.5);
7509
7510 /// The passes `scratch` records with the opening merged, and apart.
7511 fn pass_lists(scratch: &Scratch) -> (Vec<FramePass>, Vec<FramePass>) {
7512 let shape = |merge| PassShape {
7513 merge,
7514 instances: true,
7515 opaque: !scratch.opaque.is_empty(),
7516 punch: scratch.punches(),
7517 };
7518 let mut merged = Vec::new();
7519 scratch.plan_passes(&shape(true), &mut merged);
7520 let mut apart = Vec::new();
7521 scratch.plan_passes(&shape(false), &mut apart);
7522 assert_opened_once(&merged);
7523 assert!(
7524 !apart.contains(&FramePass::Opening),
7525 "an unmerged frame records no opening: {apart:?}"
7526 );
7527 (merged, apart)
7528 }
7529
7530 /// A merged list opens its surface exactly once, before every other pass
7531 /// over it, and never records the clear or the opaque strips apart.
7532 fn assert_opened_once(passes: &[FramePass]) {
7533 let openings = passes
7534 .iter()
7535 .filter(|pass| **pass == FramePass::Opening)
7536 .count();
7537 assert_eq!(openings, 1, "one opening per frame: {passes:?}");
7538 let first_surface = passes.iter().position(|pass| pass.targets_surface());
7539 assert_eq!(
7540 first_surface,
7541 passes.iter().position(|pass| *pass == FramePass::Opening),
7542 "the opening is the first pass over the surface: {passes:?}"
7543 );
7544 assert!(
7545 !passes.contains(&FramePass::Clear) && !passes.contains(&FramePass::Opaque),
7546 "and the clear and opaque strips live inside it: {passes:?}"
7547 );
7548 }
7549
7550 #[test]
7551 fn an_empty_frame_is_the_opening_alone() {
7552 let (scratch, _) = plan_of(&plan_scene(|_| {}));
7553 let (merged, apart) = pass_lists(&scratch);
7554 assert_eq!(merged, [FramePass::Opening]);
7555 assert_eq!(apart, [FramePass::Clear]);
7556
7557 // And the same before any instance buffer exists at all.
7558 for merge in [true, false] {
7559 let mut passes = Vec::new();
7560 scratch.plan_passes(
7561 &PassShape {
7562 merge,
7563 instances: false,
7564 opaque: false,
7565 punch: false,
7566 },
7567 &mut passes,
7568 );
7569 assert_eq!(passes.len(), 1, "one clean clear: {passes:?}");
7570 }
7571 }
7572
7573 #[test]
7574 fn a_surface_only_frame_folds_its_clear_and_opaque_strips_into_one_pass() {
7575 let (scratch, _) = plan_of(&plan_scene(|builder| {
7576 plan_backdrop(builder);
7577 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
7578 }));
7579 assert!(!scratch.opaque.is_empty(), "the backdrop is opaque strips");
7580
7581 let (merged, apart) = pass_lists(&scratch);
7582 assert_eq!(merged, [FramePass::Opening]);
7583 assert_eq!(
7584 apart,
7585 [FramePass::Clear, FramePass::Opaque, FramePass::Surface]
7586 );
7587 }
7588
7589 #[test]
7590 fn without_depth_the_opening_is_the_clear_and_the_surface_round() {
7591 let scene = plan_scene(|builder| {
7592 plan_backdrop(builder);
7593 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
7594 });
7595 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
7596 .compile(&scene, Affine::IDENTITY, PLAN_VIEWPORT)
7597 .expect("an in-range scene compiles");
7598 let rounds = Schedule::build(
7599 &frame.recorder,
7600 &TierCaps::fake(DownlevelProfile::Full),
7601 &PageConfig::default(),
7602 )
7603 .expect("a scene of solid fills schedules");
7604 let mut scratch = Scratch::default();
7605 scratch.build(&frame, &rounds, false, false, &[], &[]);
7606 assert!(scratch.opaque.is_empty(), "no depth, no opaque strips");
7607
7608 let (merged, apart) = pass_lists(&scratch);
7609 assert_eq!(merged, [FramePass::Opening]);
7610 assert_eq!(apart, [FramePass::Clear, FramePass::Surface]);
7611 }
7612
7613 #[test]
7614 fn a_page_round_waits_for_the_opening_rather_than_splitting_it() {
7615 let (scratch, _) = plan_of(&plan_scene(|builder| {
7616 plan_backdrop(builder);
7617 builder.push_layer(PLAN_CHIP, 0.5);
7618 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
7619 builder.pop_layer();
7620 }));
7621 let (merged, apart) = pass_lists(&scratch);
7622 // The page is rendered first either way — a layer round writes a
7623 // pooled page and reads neither the surface nor its depth — and the
7624 // surface round that composites it carries the opening.
7625 assert_eq!(merged, [FramePass::Page, FramePass::Opening]);
7626 assert_eq!(
7627 apart,
7628 [
7629 FramePass::Clear,
7630 FramePass::Opaque,
7631 FramePass::Page,
7632 FramePass::Surface
7633 ]
7634 );
7635 }
7636
7637 #[test]
7638 fn a_punch_cut_ends_the_opening_where_the_first_surface_pass_ended() {
7639 let (scratch, _) = plan_of(&plan_scene(|builder| {
7640 plan_backdrop(builder);
7641 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
7642 builder.clear_rect(PLAN_SLOT);
7643 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
7644 }));
7645 let (merged, apart) = pass_lists(&scratch);
7646 assert_eq!(
7647 merged,
7648 [FramePass::Opening, FramePass::Punch, FramePass::Surface]
7649 );
7650 assert_eq!(
7651 apart,
7652 [
7653 FramePass::Clear,
7654 FramePass::Opaque,
7655 FramePass::Surface,
7656 FramePass::Punch,
7657 FramePass::Surface
7658 ]
7659 );
7660 }
7661
7662 #[test]
7663 fn a_punch_ahead_of_every_surface_draw_takes_the_opening_alone() {
7664 // Nothing is recorded before the clear, so the plan the punch cuts
7665 // draws nothing: the opening — the backdrop's opaque strips inside it —
7666 // is recorded ahead of the erase on its own, and everything blended
7667 // after the cut loads.
7668 let (scratch, _) = plan_of(&plan_scene(|builder| {
7669 builder.clear_rect(PLAN_SLOT);
7670 plan_backdrop(builder);
7671 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
7672 }));
7673 let (merged, apart) = pass_lists(&scratch);
7674 assert_eq!(
7675 merged,
7676 [FramePass::Opening, FramePass::Punch, FramePass::Surface]
7677 );
7678 assert_eq!(
7679 apart,
7680 [
7681 FramePass::Clear,
7682 FramePass::Opaque,
7683 FramePass::Punch,
7684 FramePass::Surface
7685 ]
7686 );
7687 }
7688
7689 #[test]
7690 fn a_punch_recorded_last_follows_the_opening() {
7691 let (scratch, _) = plan_of(&plan_scene(|builder| {
7692 plan_backdrop(builder);
7693 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
7694 builder.clear_rect(PLAN_SLOT);
7695 }));
7696 let (merged, apart) = pass_lists(&scratch);
7697 assert_eq!(merged, [FramePass::Opening, FramePass::Punch]);
7698 assert_eq!(
7699 apart,
7700 [
7701 FramePass::Clear,
7702 FramePass::Opaque,
7703 FramePass::Surface,
7704 FramePass::Punch
7705 ]
7706 );
7707 }
7708
7709 #[test]
7710 fn a_filter_frame_records_its_filter_passes_before_the_opening() {
7711 use crate::filters::{LayerFilter, push_filter_layer};
7712 use vello_common::peniko::BlendMode;
7713 use vello_common::record::{CommandRecorder, LayerProps};
7714
7715 // Compiled as a backdrop and a chip, then re-recorded with the chip
7716 // inside a blurred layer under the surface: no `Scene` command
7717 // records a filter layer, so this is the only way to plan one.
7718 let mut frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
7719 .compile(
7720 &plan_scene(|builder| {
7721 plan_backdrop(builder);
7722 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
7723 }),
7724 Affine::IDENTITY,
7725 PLAN_VIEWPORT,
7726 )
7727 .expect("an in-range scene compiles");
7728 let draws = frame.recorder.draws.clone();
7729 assert_eq!(draws.len(), 2, "the backdrop and the chip");
7730 let mut recorder = CommandRecorder::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1);
7731 let strips_of = |draw: &crate::EngineDraw| &frame.strip_buf()[draw.strip_range.clone()];
7732 recorder.push_draw(draws[0].clone(), strips_of(&draws[0]));
7733 push_filter_layer(
7734 &mut recorder,
7735 LayerProps {
7736 blend_mode: BlendMode::default(),
7737 opacity: 1.0,
7738 mask: None,
7739 clip_path: None,
7740 },
7741 LayerFilter::Blur { sigma: 2.0 },
7742 Affine::IDENTITY,
7743 )
7744 .expect("a finite σ under the ceiling records");
7745 recorder.push_draw(draws[1].clone(), strips_of(&draws[1]));
7746 recorder.pop_layer();
7747 frame.recorder = recorder;
7748
7749 let rounds = Schedule::build(
7750 &frame.recorder,
7751 &TierCaps::fake(DownlevelProfile::Full),
7752 &PageConfig::default(),
7753 )
7754 .expect("a blurred layer under the surface schedules");
7755 let mut scratch = Scratch::default();
7756 scratch.build(&frame, &rounds, true, false, &[], &[]);
7757 assert!(scratch.filter_passes > 0, "the layer is filtered");
7758
7759 let (merged, apart) = pass_lists(&scratch);
7760 let filters = |passes: &[FramePass]| {
7761 passes
7762 .iter()
7763 .filter(|pass| **pass == FramePass::Filter)
7764 .count()
7765 };
7766 assert_eq!(filters(&merged), scratch.filter_passes as usize);
7767 assert_eq!(filters(&apart), filters(&merged));
7768 assert_eq!(
7769 merged.len() + 2,
7770 apart.len(),
7771 "the clear and the opaque strips fold into the surface round: \
7772 {merged:?} against {apart:?}"
7773 );
7774 assert_eq!(merged.first(), Some(&FramePass::Page));
7775 assert_eq!(
7776 merged.last(),
7777 Some(&FramePass::Opening),
7778 "every filter pass runs before the surface opens: {merged:?}"
7779 );
7780 }
7781
7782 // -----------------------------------------------------------------
7783 // A banded layer's column pages hold their own column, and only it
7784 //
7785 // The scheduler hands every band of a layer the same op list, so the
7786 // instances a band emits are where a column split is made or lost. Every
7787 // case here is host-only: the plan is built with no device, and the pixel
7788 // half of the same contract is `tests/desktop_stress.rs`'s 5K cases.
7789 // -----------------------------------------------------------------
7790
7791 /// Viewport the band plans below are built against: wide enough that one
7792 /// layer over all of it needs several column pages under [`BAND_PAGES`],
7793 /// and one tile row tall, so a draw costs one strip row per column.
7794 const BAND_VIEWPORT: (u16, u16) = (200, 8);
7795
7796 /// A page ceiling small enough to band a test-sized layer, so no case here
7797 /// has to allocate — or even name — a 5K one.
7798 const BAND_PAGES: PageConfig = PageConfig {
7799 min_page_size: 64,
7800 max_page_size: 64,
7801 };
7802
7803 /// The plan `scene` produces under `config`, alongside the rounds it was
7804 /// scheduled into.
7805 ///
7806 /// Depth is off, so every instance of the frame lands in the one blended
7807 /// buffer in painter order and a case can read the whole plan out of it;
7808 /// the split into the depth-writing pass is a surface-round decision and a
7809 /// page round never takes it.
7810 fn band_plan_of(scene: &Scene, config: &PageConfig) -> (Scratch, Vec<Round>) {
7811 let frame = SceneCompiler::new(BAND_VIEWPORT.0, BAND_VIEWPORT.1)
7812 .compile(scene, Affine::IDENTITY, BAND_VIEWPORT)
7813 .expect("an in-range scene compiles");
7814 let rounds = Schedule::build(
7815 &frame.recorder,
7816 &TierCaps::fake(DownlevelProfile::Full),
7817 config,
7818 )
7819 .expect("a layer wider than the ceiling bands rather than refusing");
7820
7821 let mut scratch = Scratch::default();
7822 scratch.build(&frame, &rounds, false, false, &[], &[]);
7823 assert_eq!(
7824 scratch.rounds.len(),
7825 rounds.len(),
7826 "a frame with no clear is cut nowhere, so the plans and the rounds \
7827 line up one for one"
7828 );
7829 (scratch, rounds)
7830 }
7831
7832 /// A layer at half opacity — so it cannot be inlined and has to take a page
7833 /// — holding one solid rectangle per entry of `rects`.
7834 fn band_scene(rects: &[(Rect, Color)]) -> Scene {
7835 let mut scene = Scene::new();
7836 let mut builder = SceneBuilder::new(&mut scene);
7837 builder.push_layer(
7838 Rect::new(
7839 0.0,
7840 0.0,
7841 f64::from(BAND_VIEWPORT.0),
7842 f64::from(BAND_VIEWPORT.1),
7843 ),
7844 0.5,
7845 );
7846 for (rect, color) in rects {
7847 builder.fill_rect(*rect, Brush::Solid(*color));
7848 }
7849 builder.pop_layer();
7850 scene
7851 }
7852
7853 /// The bounds of every page round, in order — one band's column each.
7854 fn band_bounds(rounds: &[Round]) -> Vec<RectU16> {
7855 rounds
7856 .iter()
7857 .filter_map(|round| round.page().map(|page| page.bounds))
7858 .collect()
7859 }
7860
7861 /// The strip instances one round plan issues, in execution order.
7862 fn plan_strips(scratch: &Scratch, plan: &RoundPlan) -> Vec<GpuStrip> {
7863 scratch.segments[plan.segments.clone()]
7864 .iter()
7865 .filter_map(|segment| match *segment {
7866 Segment::Strips(first, count, _)
7867 | Segment::External(first, count, _, _)
7868 | Segment::Backdrop(first, count, _) => Some((first as usize, count as usize)),
7869 Segment::Composite(..) => None,
7870 })
7871 .flat_map(|(first, count)| scratch.alpha[first..first + count].iter().copied())
7872 .collect()
7873 }
7874
7875 /// Every pixel column the frame's *page* rounds paint, mapped back out of
7876 /// the pages they were shifted into, and valued by what the shader reads
7877 /// there: the instance's paint payload, plus the exact alpha column that
7878 /// pixel samples (`None` for a sparse instance, which samples none).
7879 ///
7880 /// Keyed by the draw's own depth as well as the position, so a page painted
7881 /// by two different draws is not conflated — and so a *second* instance of
7882 /// one draw covering a pixel it already covered, which is precisely what a
7883 /// clamped band replay produces, is caught here rather than silently
7884 /// overwriting the first.
7885 fn painted_pixels(
7886 scratch: &Scratch,
7887 rounds: &[Round],
7888 ) -> BTreeMap<(u32, u16, u16), (u32, Option<u32>)> {
7889 let mut painted = BTreeMap::new();
7890
7891 for (plan, round) in scratch.rounds.iter().zip(rounds) {
7892 let Some(page) = round.page() else {
7893 continue;
7894 };
7895 for span in plan_strips(scratch, plan) {
7896 for offset in 0..span.width {
7897 let x = page.bounds.x0 + span.x + offset;
7898 let y = page.bounds.y0 + span.y;
7899 let column = (offset < span.dense_width_or_rect_height)
7900 .then(|| span.col_idx_or_rect_frac + u32::from(offset));
7901 assert!(
7902 painted
7903 .insert((span.depth_index, y, x), (span.payload, column))
7904 .is_none(),
7905 "one draw covers a device pixel at most once, however the \
7906 layer holding it was split"
7907 );
7908 }
7909 }
7910 }
7911
7912 painted
7913 }
7914
7915 #[test]
7916 fn a_banded_layer_paints_exactly_what_one_page_would_have() {
7917 // The whole point of the split, asserted as an equality rather than as
7918 // a rectangle property: the same scene planned onto one page and onto
7919 // column bands has to paint the same device pixels, from the same
7920 // paints, sampling the same alpha columns. A band replay that clamped
7921 // a strip left of its own column onto the page's edge fails here twice
7922 // over — once on the ghost pixel, once on the column it would sample.
7923 let scene = band_scene(&[
7924 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
7925 // Straddles a band edge on a tile that is only partly covered, so
7926 // the case exercises an alpha-sampled instance cut in two, not
7927 // just a solid one.
7928 (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
7929 (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
7930 ]);
7931
7932 let (one_page, one_page_rounds) = band_plan_of(&scene, &PageConfig::default());
7933 let (banded, banded_rounds) = band_plan_of(&scene, &BAND_PAGES);
7934
7935 assert_eq!(
7936 band_bounds(&one_page_rounds).len(),
7937 1,
7938 "the reference plan really does hold the layer on one page"
7939 );
7940 assert!(
7941 band_bounds(&banded_rounds).len() > 1,
7942 "and the case only means anything while the other one is banded"
7943 );
7944
7945 assert_eq!(
7946 painted_pixels(&banded, &banded_rounds),
7947 painted_pixels(&one_page, &one_page_rounds),
7948 "a banded layer paints what one whole-layer page would have"
7949 );
7950 }
7951
7952 #[test]
7953 fn a_band_holds_no_instance_of_a_strip_outside_its_own_column() {
7954 // The counterexample the equality above generalizes: content confined
7955 // to the outer columns, and nothing at all in the middle. A band whose
7956 // column the scene never drew in must render nothing — under a
7957 // saturating shift it would render the leftmost content clamped onto
7958 // its own edge instead.
7959 let scene = band_scene(&[
7960 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
7961 (Rect::new(160.0, 0.0, 200.0, 8.0), BLUE),
7962 ]);
7963 let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
7964 let bands = band_bounds(&rounds);
7965 assert!(bands.len() > 2, "the layer bands: {bands:?}");
7966
7967 let mut empty = 0_usize;
7968 for (plan, band) in scratch
7969 .rounds
7970 .iter()
7971 .zip(&rounds)
7972 .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
7973 {
7974 let strips = plan_strips(&scratch, plan);
7975 if strips.is_empty() {
7976 empty += 1;
7977 }
7978 for span in strips {
7979 let end = span.x + span.width;
7980 assert!(
7981 end <= band.width(),
7982 "an instance of {span:?} reaches past the {band:?} band's own \
7983 column, which its composite never samples"
7984 );
7985 // Mapped back to the scene, every instance lands where one of
7986 // the two rectangles was actually drawn.
7987 let x = band.x0 + span.x;
7988 assert!(
7989 x < 44 || band.x0 + end > 160,
7990 "an instance covers device x {x}..{}, which neither \
7991 rectangle reaches",
7992 band.x0 + end
7993 );
7994 }
7995 }
7996
7997 assert!(
7998 empty > 0,
7999 "a band whose column holds nothing renders nothing: {bands:?}"
8000 );
8001 }
8002
8003 #[test]
8004 fn a_strip_straddling_a_band_edge_advances_its_alpha_column_with_its_geometry() {
8005 // A pixel-aligned rectangle's left edge lands on a tile of its own, so
8006 // a rectangle starting at 49 puts an alpha-sampled instance across
8007 // 48..52 — and a band edge falls inside it. The two pieces together
8008 // have to read the same coverage the whole instance would: neighbouring
8009 // pixels of one strip row sampling neighbouring alpha columns, with no
8010 // column repeated. The rectangles either side of it are what carries
8011 // the layer past the ceiling, so the middle one is banded at all.
8012 let scene = band_scene(&[
8013 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8014 (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
8015 (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
8016 ]);
8017 let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
8018 let edges: Vec<u16> = band_bounds(&rounds)
8019 .iter()
8020 .map(|bounds| bounds.x0)
8021 .collect();
8022
8023 // Every alpha-sampled pixel of every band, as (strip row, device x,
8024 // the alpha column it samples) — a row at a time, because two rows of
8025 // one draw sample different columns at the same x by construction.
8026 let mut dense: Vec<(u16, u16, u32)> = Vec::new();
8027 for (plan, band) in scratch
8028 .rounds
8029 .iter()
8030 .zip(&rounds)
8031 .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
8032 {
8033 for span in plan_strips(&scratch, plan) {
8034 for offset in 0..span.dense_width_or_rect_height {
8035 dense.push((
8036 band.y0 + span.y,
8037 band.x0 + span.x + offset,
8038 span.col_idx_or_rect_frac + u32::from(offset),
8039 ));
8040 }
8041 }
8042 }
8043 dense.sort_unstable();
8044
8045 let neighbours = || {
8046 dense
8047 .windows(2)
8048 .map(|pair| (pair[0], pair[1]))
8049 .filter(|((row, x, _), (next_row, next_x, _))| row == next_row && x + 1 == *next_x)
8050 };
8051 assert!(
8052 neighbours().any(|(_, (_, x, _))| edges.contains(&x)),
8053 "the case only means anything while an alpha-sampled instance \
8054 really is cut by a band edge {edges:?}: {dense:?}"
8055 );
8056
8057 for ((_, _, column), (_, x, next_column)) in neighbours() {
8058 assert_eq!(
8059 next_column,
8060 column + 1,
8061 "neighbouring pixels of one strip row sample neighbouring alpha \
8062 columns, band edge at {x} or not: {dense:?}"
8063 );
8064 }
8065 }
8066
8067 #[test]
8068 fn a_translucent_layer_over_a_clear_is_composited_after_the_punch() {
8069 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8070 plan_backdrop(builder);
8071 builder.clear_rect(PLAN_SLOT);
8072 builder.push_layer(PLAN_CHIP, 0.5);
8073 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8074 builder.pop_layer();
8075 }));
8076
8077 assert_eq!(
8078 scratch.rounds.len(),
8079 rounds.len() + 1,
8080 "the layer's page round, then the surface round cut in two"
8081 );
8082 let cut = scratch
8083 .rounds
8084 .iter()
8085 .position(|plan| plan.punch.1 > 0)
8086 .expect("the cut plan carries the punch");
8087 // The composite is what the cut has to fall before: a layer recorded
8088 // after the clear carries a deeper index than the punch, and the
8089 // composite writes no depth for the test to save it by.
8090 let composited = scratch.segments[scratch.rounds[cut + 1].segments.clone()]
8091 .iter()
8092 .any(|segment| matches!(segment, Segment::Composite(..)));
8093 assert!(
8094 composited,
8095 "the layer composites in the plan AFTER the punch, not before it"
8096 );
8097 assert!(
8098 !scratch.segments[scratch.rounds[cut].segments.clone()]
8099 .iter()
8100 .any(|segment| matches!(segment, Segment::Composite(..))),
8101 "and nothing composites into the plan the punch closes"
8102 );
8103 }
8104
8105 // -----------------------------------------------------------------
8106 // Backdrop blurs: the copy round, the sampled page, the opaque limit
8107 //
8108 // Host-only, over the same plan the frame path records from: a
8109 // backdrop's region copy is the one round besides a punch that must
8110 // follow the opening, and its rectangle the one strip run that samples
8111 // a page rather than a paint.
8112 // -----------------------------------------------------------------
8113
8114 /// The rectangle every backdrop case below blurs behind.
8115 const PLAN_PANEL: Rect = Rect::new(8.0, 8.0, 40.0, 36.0);
8116
8117 /// [`plan_of`] for a frame whose backdrops are served — scheduled through
8118 /// [`Schedule::build_frame`] and planned with the frame's own backdrop
8119 /// records, as `encode` does on a readable target.
8120 fn backdrop_plan_of(scene: &Scene, punches: bool) -> (Scratch, Vec<Round>, CompiledFrame) {
8121 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8122 .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
8123 .expect("an in-range scene compiles");
8124 let rounds = Schedule::build_frame(
8125 &frame.recorder,
8126 &TierCaps::fake(DownlevelProfile::Full),
8127 &PageConfig::default(),
8128 &RetainedLayers::new(),
8129 &frame.backdrops,
8130 )
8131 .expect("a backdrop under the surface schedules");
8132 let mut scratch = Scratch::default();
8133 scratch.build(&frame, &rounds, true, punches, &[], &frame.backdrops);
8134 (scratch, rounds, frame)
8135 }
8136
8137 /// An opaque field, the backdrop, and a chip painted over the panel.
8138 fn backdrop_scene() -> Scene {
8139 plan_scene(|builder| {
8140 builder.fill_rect(
8141 Rect::new(
8142 0.0,
8143 0.0,
8144 f64::from(PLAN_VIEWPORT.0),
8145 f64::from(PLAN_VIEWPORT.1),
8146 ),
8147 Brush::Solid(BLUE),
8148 );
8149 builder.backdrop_blur(PLAN_PANEL, 2.0);
8150 builder.fill_rect(Rect::new(12.0, 12.0, 20.0, 20.0), Brush::Solid(RED));
8151 })
8152 }
8153
8154 #[test]
8155 fn a_backdrop_ends_the_opening_before_its_copy_and_draws_after_its_passes() {
8156 let (scratch, _, frame) = backdrop_plan_of(&backdrop_scene(), false);
8157 assert_eq!(frame.backdrops.len(), 1);
8158 let (merged, apart) = pass_lists(&scratch);
8159 // σ 2 convolves directly: one horizontal and one vertical pass.
8160 assert_eq!(
8161 merged,
8162 [
8163 FramePass::Opening,
8164 FramePass::Copy,
8165 FramePass::Filter,
8166 FramePass::Filter,
8167 FramePass::Surface,
8168 ],
8169 "the opening — clear, opaque strips and the field — lands before the copy reads \
8170 the surface back, and the panel and chip draw after the blur"
8171 );
8172 assert_eq!(
8173 apart,
8174 [
8175 FramePass::Clear,
8176 FramePass::Opaque,
8177 FramePass::Surface,
8178 FramePass::Copy,
8179 FramePass::Filter,
8180 FramePass::Filter,
8181 FramePass::Surface,
8182 ],
8183 "unmerged, the same order with the clear and the opaque strips apart"
8184 );
8185 }
8186
8187 #[test]
8188 fn a_backdrops_rectangle_samples_the_page_its_round_releases() {
8189 let (scratch, rounds, frame) = backdrop_plan_of(&backdrop_scene(), false);
8190 let backdrop = frame.backdrops[0];
8191 let sampling = rounds
8192 .iter()
8193 .position(|round| !round.samples.is_empty())
8194 .expect("one surface round samples the blurred page");
8195 let sample = rounds[sampling].samples[0];
8196 assert_eq!(sample.layer, backdrop.layer);
8197 assert!(
8198 rounds[sampling].released.contains(&sample.parity),
8199 "the round that samples the page hands it back"
8200 );
8201
8202 let plan = &scratch.rounds[sampling];
8203 let segments = &scratch.segments[plan.segments.clone()];
8204 let (first, count, parity) = segments
8205 .iter()
8206 .find_map(|segment| match *segment {
8207 Segment::Backdrop(first, count, parity) => Some((first, count, parity)),
8208 _ => None,
8209 })
8210 .expect("the panel is drawn as a backdrop segment");
8211 assert_eq!(parity, sample.parity);
8212 assert!(count > 0);
8213 for instance in &scratch.alpha[first as usize..(first + count) as usize] {
8214 assert_eq!(
8215 instance.paint_and_rect_flag & !crate::gpu::strips::RECT_STRIP_FLAG,
8216 LAYER_PAINT_SOURCE | u32::from(u8::MAX),
8217 "every panel instance reads the layer input at full opacity"
8218 );
8219 let (page_x, page_y) = (instance.payload & 0xffff, instance.payload >> 16);
8220 assert_eq!(
8221 (page_x, page_y),
8222 (
8223 u32::from(instance.x - backdrop.region.x0),
8224 u32::from(instance.y - backdrop.region.y0)
8225 ),
8226 "and samples the page texel under its own device origin"
8227 );
8228 }
8229 }
8230
8231 #[test]
8232 fn draws_after_a_backdrop_never_take_the_depth_writing_pass() {
8233 // The field recorded *after* the backdrop: its opaque interior must
8234 // not be drawn in the opening, ahead of the copy that reads the
8235 // surface back.
8236 let after = plan_scene(|builder| {
8237 builder.backdrop_blur(PLAN_PANEL, 2.0);
8238 builder.fill_rect(
8239 Rect::new(
8240 0.0,
8241 0.0,
8242 f64::from(PLAN_VIEWPORT.0),
8243 f64::from(PLAN_VIEWPORT.1),
8244 ),
8245 Brush::Solid(BLUE),
8246 );
8247 });
8248 let (scratch, _, _) = backdrop_plan_of(&after, false);
8249 assert!(
8250 scratch.opaque.is_empty(),
8251 "nothing at or past the backdrop's depth is routed opaque"
8252 );
8253
8254 // The same field before the backdrop keeps its opaque spans.
8255 let (before, _, _) = backdrop_plan_of(&backdrop_scene(), false);
8256 assert!(!before.opaque.is_empty());
8257 }
8258
8259 #[test]
8260 fn a_punch_ahead_of_a_backdrop_is_erased_before_the_copy() {
8261 let scene = plan_scene(|builder| {
8262 builder.fill_rect(PLAN_SLOT, Brush::Solid(BLUE));
8263 builder.clear_rect(PLAN_SLOT);
8264 builder.backdrop_blur(PLAN_PANEL, 2.0);
8265 });
8266 let (scratch, _, _) = backdrop_plan_of(&scene, true);
8267 let (merged, _) = pass_lists(&scratch);
8268 let punch = merged
8269 .iter()
8270 .position(|pass| *pass == FramePass::Punch)
8271 .expect("the punch is recorded");
8272 let copy = merged
8273 .iter()
8274 .position(|pass| *pass == FramePass::Copy)
8275 .expect("the copy is recorded");
8276 assert!(
8277 punch < copy,
8278 "the backdrop reads the erased surface, not the field the punch erases: {merged:?}"
8279 );
8280 }
8281
8282 #[test]
8283 fn a_target_that_cannot_be_read_back_serves_no_backdrop() {
8284 // `encode` hands the plan no backdrop records on such a target: the
8285 // panel is the transparent draw it was recorded as, scheduled as the
8286 // ordinary layer it is — no copy, no filter pass, no sampled page.
8287 let (scratch, rounds) = plan_of(&backdrop_scene());
8288 assert!(
8289 rounds
8290 .iter()
8291 .all(|round| round.copy.is_none() && round.samples.is_empty())
8292 );
8293 let (merged, _) = pass_lists(&scratch);
8294 assert_eq!(merged, [FramePass::Opening]);
8295 assert_eq!(scratch.copies, 0);
8296 }
8297
8298 #[test]
8299 fn the_region_copy_mode_prefers_sampling_and_needs_a_single_sample() {
8300 use wgpu::TextureFormat::{Bgra8Unorm, Rgba8Unorm, Rgba8UnormSrgb};
8301 use wgpu::TextureUsages as U;
8302 let attach = U::RENDER_ATTACHMENT;
8303 assert_eq!(
8304 RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8Unorm, 1),
8305 RegionCopyMode::Texture,
8306 "a headless readback target in the pages' own format is copied"
8307 );
8308 assert_eq!(
8309 RegionCopyMode::for_usage(attach | U::COPY_SRC | U::TEXTURE_BINDING, Rgba8Unorm, 1),
8310 RegionCopyMode::Fragment,
8311 "sampling costs the one pass alone, so it wins whenever it is allowed"
8312 );
8313 assert_eq!(
8314 RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING, Bgra8Unorm, 1),
8315 RegionCopyMode::Fragment,
8316 "any format can be sampled across into a page"
8317 );
8318 assert_eq!(
8319 RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8UnormSrgb, 1),
8320 RegionCopyMode::Unavailable,
8321 "a copy would move encoded bytes into a linear page"
8322 );
8323 assert_eq!(
8324 RegionCopyMode::for_usage(attach, Bgra8Unorm, 1),
8325 RegionCopyMode::Unavailable,
8326 "a swapchain is RENDER_ATTACHMENT only"
8327 );
8328 assert_eq!(
8329 RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING | U::COPY_SRC, Rgba8Unorm, 4),
8330 RegionCopyMode::Unavailable,
8331 "a multisampled target is neither"
8332 );
8333 }
8334}