frust_engine/renderer.rs
1//! [`EngineRenderer`]: the public seam a host drives one surface's 2D frames
2//! through.
3//!
4//! # The encode contract
5//!
6//! [`EngineRenderer::encode`] **records into a `wgpu::CommandEncoder` the
7//! caller owns and never submits it.** That is the whole point of the seam: a
8//! host compositing 2D over its own 3D content records its passes before and
9//! after the engine's into one encoder and submits once, and a submit hidden
10//! inside the engine would split that into two command buffers with a pipeline
11//! flush between them. Two obligations follow, and both are contract rather
12//! than preference:
13//!
14//! - every pass the engine begins is ended before `encode` returns, so the
15//! caller's next `begin_render_pass` on the same encoder is legal; and
16//! - the engine issues no `queue.submit` for scene work. Its per-frame
17//! uploads — coverage, encoded paints, instances, the viewport uniform and
18//! each page round's — are written into a mapped staging slot and moved into
19//! place by copy commands recorded into the caller's encoder, ahead of the
20//! passes that read them (see [`frust_gpu::staging`]). The slot's re-mapping
21//! rides the same encoder (`map_buffer_on_submit`), so the caller's one
22//! submit is also what returns the slot to the engine; an encoder dropped
23//! unsubmitted just costs the engine that slot. Its on-change uploads
24//! (colour ramps, atlas regions) — and every upload under
25//! `FRUST_ENGINE_NO_STAGING_RING`, or on a frame the ring has no free slot
26//! for — are `queue.write_texture`/`write_buffer` calls, which are ordered
27//! ahead of the command buffers submitted after them and so land before the
28//! passes that read them.
29//!
30//! One thing the engine *does* submit, and it is worth naming precisely because
31//! the rule above is otherwise absolute: growing the image atlas array submits a
32//! command buffer of its own, holding one texture copy and nothing else, on the
33//! rare frame that grows it. That is **maintenance, not scene work** — it never
34//! touches the caller's encoder, records no pass and no draw, and exists because
35//! the queued writes it has to precede would otherwise be flushed ahead of it
36//! (see [`crate::gpu::atlas`]'s *Why growth submits a command buffer of its
37//! own*). The caller's encoder is still never submitted by the engine, and the
38//! frame's own passes still reach the queue only when the caller submits it.
39//!
40//! # Sharing the depth attachment
41//!
42//! A caller recording its own depth-writing passes into that encoder hands the
43//! same attachment in as `EngineTarget::depth`, and the two renderers then
44//! occlude each other correctly in either order. Three rules make that work,
45//! and [`crate::gpu::depth`] is where they are stated in full: the shared
46//! comparison and which end of the range is near
47//! ([`DEPTH_COMPARE`](crate::gpu::depth::DEPTH_COMPARE) over a buffer whose far
48//! plane is [`DEPTH_CLEAR`](crate::gpu::depth::DEPTH_CLEAR)); the depth
49//! attachment's extent matching the colour target's; and who owns the clear —
50//! whichever pass runs first in the encoder, which the caller states through
51//! [`EngineRenderer::set_depth_pre_cleared`].
52//!
53//! What that contract does *not* extend to is colour. The frame's opening
54//! below clears the frame's colour target unconditionally, so content painted
55//! into that target before `encode` keeps its depth and loses its pixels: a
56//! host compositing over its own content records that content after the frame,
57//! or into a target of its own the frame composites.
58//!
59//! # The frame's passes
60//!
61//! A frame records into that encoder, in this order.
62//!
63//! 1. **The opening** ([`FramePass::Opening`]), the first pass over the
64//! surface, which does three things in one render pass:
65//! - **clears** the colour target to the frame's base colour, and the depth
66//! attachment to the far plane unless the caller stated it is already
67//! populated (see [`crate::gpu::depth`]) — the pass's own load
68//! operations, so a frame with no draws at all still resolves to a clean
69//! surface;
70//! - draws the **opaque strips**, depth-tested and depth-writing,
71//! unblended: only the fully-covered interior spans of opaque draws
72//! *targeting the surface* — an anti-aliased edge is by definition not
73//! opaque, and a page carries no depth attachment for the split to be
74//! sound against. The depth they establish is what lets every blended
75//! strip after them reject fragments an opaque draw in front of it
76//! already covered;
77//! - then draws the **first surface round's own strips** (pass 2 below),
78//! in order, the pipeline switching between the two inside the pass.
79//!
80//! It ends exactly where that round's own pass would have ended — at a
81//! punch cut, at a page or filter round, or at the end of the frame. Page
82//! and filter rounds scheduled ahead of the first surface round run ahead
83//! of the opening: they write pooled pages and read neither the surface nor
84//! its depth, so the opening waits past them. A punch issued before any
85//! surface round draws takes the opening as a pass of its own, ahead of the
86//! erase, and so does the end of a frame whose surface rounds drew nothing
87//! blended.
88//!
89//! The opaque strips are drawn once, not once per surface round, and that
90//! is a correctness requirement: the surface can take several rounds (a
91//! [cut](crate::schedule::cut_at) round is how a wide sibling fan is
92//! served), and re-drawing them ahead of each would re-draw opaque coverage
93//! at equal stored depth over composites the round before it had already
94//! blended.
95//!
96//! One pass rather than three is what a tile-based GPU is paid for: every
97//! pass over the surface is a whole-framebuffer load into tile memory and a
98//! store back out, and the clear and the opaque strips were two such passes
99//! that drew little or nothing of their own. The pixels are identical —
100//! the same draws in the same order against the same attachments, only the
101//! pass boundaries gone. `FRUST_ENGINE_NO_PASS_MERGE`
102//! ([`EngineRenderer::set_pass_merge`]) records them apart again: a
103//! [`FramePass::Clear`] pass drawing nothing and a [`FramePass::Opaque`]
104//! pass loading it, ahead of every round, and then every surface round in
105//! a pass of its own.
106//! 2. **One pass per [round](crate::schedule)**, the first surface round that
107//! draws excepted — it is drawn inside the opening. A page round
108//! ([`FramePass::Page`]) renders one isolated layer into a pooled
109//! intermediate [page](crate::schedule::pages) it clears to transparent, at
110//! the page's own origin — so every instance is shifted by the page's
111//! tile-aligned bounds, and clipped to them, which is what makes a
112//! [banded](crate::schedule::pages::page_bands) layer's column pages tile
113//! their layer instead of each holding a clamped copy of it (see
114//! [`PageWindow`]) — and through the page's own viewport uniform; a round
115//! continuing a page an earlier round of the same layer opened loads it
116//! instead. A surface round ([`FramePass::Surface`]) draws **alpha
117//! strips**, premultiplied-blended, in painter order, loading the surface:
118//! depth-tested but not depth-writing when a depth attachment is in play, a
119//! plain painter's-algorithm pass when it is not. A surface round with
120//! nothing blended to draw records no pass. A round's ops run in the order
121//! [`Schedule::build`] listed them, so a finished child page composites into
122//! its parent exactly where the recording entered it.
123//!
124//! A **filter round** is the one round that draws no strip at all: it runs
125//! one pass of a [filter](crate::filters)'s sequence, one instanced quad
126//! through [`EnginePipeline::Filter`], reading the layer's other pooled page
127//! through the engine's only sampler and clearing the page it writes (see
128//! [`FilterResources`]). It is an ordinary round of this walk in every other
129//! respect — recorded into the caller's own encoder, in the order the
130//! scheduler listed it, with its pages handed back the moment its pass ends.
131//! It is *not* an own-encoder exception; the atlas replay remains the only
132//! one of those.
133//!
134//! A **region-copy round** ([`FramePass::Copy`]) is a backdrop blur's
135//! first: it reads a region of the frame's own target back into a pooled
136//! page ([`RegionCopyMode`]) for the blur's filter rounds to run over. It is
137//! the one page round that *follows* the opening rather than waiting past
138//! it — it reads the surface, so the clear, the opaque strips and every
139//! surface round ahead of it in painter order must have landed (the
140//! scheduler cuts the surface round there; a punch recorded ahead of it is
141//! erased first) — and, for the same reason, no draw at or past the first
142//! backdrop's depth is routed into the opaque strips. The backdrop's own
143//! rectangle is then drawn in the next surface round as strip instances
144//! reading the blurred page through the composite's layer input, so a clip
145//! open around it masks it like any fill.
146//! 3. **The hole punch** ([`FramePass::Punch`]), destination-out, when the
147//! frame recorded a `ClearRect` — see [`crate::compile::clear`] for the
148//! whole contract this pass implements. It is issued at the punch's own painter-order position,
149//! not at the end of the frame: a surface round is *cut* where the punch
150//! was recorded, the punch pass goes into that cut, and the round's
151//! remaining ops resume in a pass of their own after it. Everything drawn
152//! over the slot is therefore recorded after the erase and survives it,
153//! whether or not it wrote depth. A punch past every op of the frame — the
154//! ordinary case, a `ClearRect` recorded last — cuts nothing and lands after
155//! the last round exactly as it always did.
156//!
157//! With depth unavailable — no attachment, or `FRUST_ENGINE_NO_DEPTH` set —
158//! the opening draws no opaque strips (and, unmerged, the opaque pass
159//! disappears) and every instance travels through the surface rounds, blended
160//! in painter order. That is a correctness requirement rather than a
161//! fallback detail: routing the opaque spans into a separate, earlier pass is
162//! only sound because the depth buffer re-establishes their ordering against the
163//! blended ones.
164//!
165//! # Compositing a layer
166//!
167//! A finished page reaches its parent as ONE instanced quad through the strip
168//! program every draw goes through (its [`PaintClass::LayerComposite`]
169//! specialisation — see *Paint-class runs*), flagged as a whole rectangle and
170//! naming the layer colour source: the fragment stage then reads the page
171//! bound as `layer_input_texture` at the quad's own texel and scales it by the
172//! opacity packed into the instance's low byte. The page is bound through a
173//! bind group of its own rather than the frame's shared one, because group 0
174//! carries both the pass's viewport uniform and that layer input, and a page
175//! round's viewport is its own.
176//!
177//! A retained page ([`crate::cache::retained`]) is composited by exactly the
178//! same quad, bound the same way; only the texture differs — the store's own,
179//! named by the retained bracket rather than by a page group. A round
180//! rendering a body into one is a page round in every respect above (cleared on
181//! its first round, loaded on a continued one, through its own viewport
182//! uniform, never taking the surface opening) and is listed as a
183//! [`FramePass::Page`]. A `DrawRetained`'s composite carries the painter-order
184//! index the command itself was recorded at.
185//!
186//! A composite carries the deepest painter's-order index of everything inside
187//! the layer it composites, nested layers included. That is what keeps a
188//! translucent layer correctly ordered against the root round's own draws
189//! without giving the scheduler a depth model: every draw recorded before the
190//! layer sits behind that index and every draw recorded after it sits in front.
191//!
192//! # Paint-class runs
193//!
194//! Every strip pass is drawn in **runs**, each through the pipeline of one
195//! [`PaintClass`] — the strip program specialised to the paints the run holds
196//! (see [`crate::gpu::pipelines`]'s *The paint-class axis*). The runs are cut
197//! when a segment is planned (`Scratch::push_run`, by
198//! [`merge_class_runs`]): contiguous, in painter order, covering the segment
199//! exactly, so a pass draws the same instances in the same order it always did
200//! and only the program shading each run changes. A run shorter than the
201//! renderer's threshold ([`EngineRenderer::run_split_min`],
202//! [`crate::gpu::strips::RUN_SPLIT_MIN`] by default) is merged and drawn with
203//! [`PaintClass::Uber`] rather than paying a pipeline switch of its own; a
204//! composite always takes [`PaintClass::LayerComposite`], the opaque pass its
205//! own runs, and a punch [`PaintClass::SolidCoverage`]. A pass sets a pipeline
206//! only where the class changes between runs.
207//!
208//! `FRUST_ENGINE_UBER_PROGRAM` ([`EngineRenderer::set_uber_program`]) turns all
209//! of it off: no segment is split, every pass draws through its state's one
210//! Uber pipeline, and no class program is warmed or built.
211//!
212//! `FRUST_ENGINE_IMAGE_SAMPLER` ([`EngineRenderer::with_image_sampler`]) picks
213//! the program *family* the classes are compiled from, not a class: in the
214//! sampler family — the default — every program that reads atlas images (the
215//! image classes and Uber) filters a bilinear image through one hardware
216//! sampler, which every strip bind group then carries beside the atlas array;
217//! its kill switch (`0`) compiles the manual four-tap family instead. The
218//! runs, and the pipelines a frame resolves, are the same in both families.
219//!
220//! # Paint resolution
221//!
222//! A solid colour travels inside the strip instance itself. Anything else the
223//! compiler encoded — a gradient, an image, a blurred rounded rectangle — is
224//! resolved once per frame before a single instance is built, in three steps
225//! that have to happen in this order:
226//!
227//! 1. residency is settled for the whole frame: the frame's LUT requests are
228//! serviced through the [`GradientCache`] so every gradient's colour ramp
229//! has an offset into the packed LUT buffer, and every image paint's atlas
230//! rectangle is looked up (or learned, the first time it is drawn) in the
231//! renderer's own image registry — see [`FrameResources::resolve_paints`];
232//! 2. each encoded paint is lowered into the [`GpuEncodedPaint`] record the
233//! fragment shader samples, carrying that residency; and
234//! 3. the records are serialized back to back, which fixes the texel each one
235//! starts at — the index a strip instance names its paint by.
236//!
237//! Ramp offsets are only valid within the frame that took them: the cache
238//! compacts and rewrites them in [`EngineRenderer::end_frame`], which is why
239//! residency is decided here rather than at compile time. An image's atlas
240//! rectangle, by contrast, is stable for as long as the image stays resident
241//! (the compiler's [`crate::cache::images::ImageResidency`] does not move a
242//! live image), so the renderer's own registry only ever forgets an entry
243//! when the frame that compiled it reports the entry's region evicted.
244//!
245//! A paint that still cannot be resolved — an image the atlas has no room
246//! for, a gradient whose ramp could not be baked, an external texture (nothing
247//! binds one yet) — leaves its draw skipped rather than stamped in a wrong
248//! colour: the same "a frame draws less, never wrong" rule the compiler
249//! follows for the commands it does not lower.
250//!
251//! An image the atlas holds a *minified* copy of is the one paint whose lowered
252//! record needs a correction here. The compiler composed its natural-to-device
253//! transform against the source's declared extent, before residency was
254//! consulted and so before the fit was known; the shader samples the resident
255//! rectangle. [`ResidentImage::minify_scale`] is the ratio between the two, and
256//! folding it into the lowered record's transform is what keeps a downsampled
257//! image landing on the destination rectangle the display list asked for.
258
259use core::ops::Range;
260use std::collections::{HashMap, HashSet};
261use std::sync::{Arc, Once};
262use std::time::{Duration, Instant};
263
264use frust_gpu::staging::{
265 BUFFER_COPY_ALIGNMENT, StagingFrame, StagingRing, TEXTURE_COPY_ALIGNMENT,
266};
267use frust_gpu::{PipelineCache, PooledTexture, SceneTextureId, ShaderId, ShaderLibrary, TierCaps};
268use frust_scene::{Scene, SceneBuilder};
269use glifo::{AtlasCommand, AtlasCommandRecorder, AtlasPaint};
270use kurbo::Affine;
271use peniko::{Brush, Color};
272use vello_common::encode::{EncodedImage, EncodedPaint};
273use vello_common::fearless_simd::Level;
274use vello_common::paint::{ImageId, ImageSource, Paint};
275use vello_common::strip::Strip;
276
277use crate::cache::images::{ATLAS_PADDING, AtlasRegion};
278use crate::cache::retained::{
279 RetainedBudget, RetainedDecision, RetainedEntry, RetainedPages, RetainedPlan,
280};
281use crate::cache::{
282 AtlasBudget, BYTES_PER_TEXEL, CachedRamp, GradientCache, GradientTextureLayout, ResidentImage,
283};
284#[cfg(any(test, feature = "perf-trace"))]
285use crate::compile::CompileSpans;
286use crate::compile::paint::resolve_lut_request;
287use crate::compile::{
288 ClearPunch, CompiledFrame, GlyphQuad, PhaseClock, RetainedKind, SceneCompiler,
289};
290use crate::config;
291use crate::diag::{EngineSpan, FrameTimestamps};
292use crate::error::EngineError;
293use crate::filters::blur::{FilterInstanceData, GpuFilterData, GpuGaussianBlur};
294use crate::filters::drop_shadow::GpuDropShadow;
295use crate::filters::{Backdrop, FilterStep, ServedFilter, served_filter};
296use crate::gpu::atlas::{
297 AtlasPageBuffers, AtlasRenderReport, AtlasRenderer, lower_encoded_image, push_solid_strips,
298};
299use crate::gpu::bindings::{
300 ATLAS_SAMPLER_BINDING, ExternalRuns, ExternalTextures, GroupKind, GroupSource,
301 SourceGroupCache, SourceGroupKey, lower_encoded_external,
302};
303use crate::gpu::config::resource_widths_agree;
304use crate::gpu::depth::DepthAttachment;
305use crate::gpu::paint_texture::lower_encoded_paint;
306use crate::gpu::pipelines::{
307 EnginePipeline, EngineShaders, INTERMEDIATE_FORMAT, atlas_strip_desc, warm_up_plan,
308};
309use crate::gpu::strips::{
310 ClassRun, ClassSet, PaintClass, PaintType, merge_class_runs, pack_paint_descriptor,
311};
312use crate::gpu::targets::{
313 IntermediateTargets, IntermediateTexture, filter_data_texture_descriptor,
314 filter_data_texture_height, filter_sampler,
315};
316use crate::gpu::{self, AtlasArray, GpuConfig, GpuEncodedPaint, GpuStrip, StripDraw};
317use crate::schedule::pages::{PageConfig, PageSize};
318use crate::schedule::{
319 Composite, MAX_LIVE_PAGES, PageParity, PageTarget, RetainedLayers, RetainedRole,
320 RetainedTarget, Round, RoundOp, Schedule,
321};
322use crate::{EngineTarget, OutputAlpha};
323use vello_common::geometry::RectU16;
324use vello_common::geometry::SizeU16;
325use vello_common::record::RecordedLayerKind;
326
327/// The packed paint descriptor of an inline premultiplied solid colour.
328///
329/// A solid paint indexes no encoded-paint record, so its descriptor carries
330/// only the colour source and paint type — both of which are zero, which is
331/// why this is named rather than written as a bare `0` at its call site: the
332/// zero is a coincidence of the layout, not an absence of information.
333const SOLID_PAINT: u32 = pack_paint_descriptor(PaintType::Solid, 0);
334
335/// The smallest resource-texture dimension the engine can address.
336///
337/// A resource texture's row stride is its width times its texel size, and
338/// `wgpu::COPY_BYTES_PER_ROW_ALIGNMENT` is 256; the narrowest of the three
339/// resources is the gradient LUT at 4 bytes per texel, so 64 texels is the
340/// point below which an upload's `bytes_per_row` stops being legal.
341const MIN_RESOURCE_TEXTURE_DIM: u32 = 64;
342
343/// The smallest strip instance buffer the engine allocates, in instances.
344///
345/// A first frame with a handful of strips should not force a second allocation
346/// on the second frame.
347const MIN_INSTANCE_CAPACITY: u64 = 4096;
348
349/// The colour source a composite instance names its finished page by:
350/// `COLOR_SOURCE_LAYER` in bits 29-30 of the packed paint descriptor.
351///
352/// Deliberately not built through
353/// [`pack_paint_descriptor`](crate::gpu::strips::pack_paint_descriptor): that
354/// helper packs a paint type and a paint-record index into the low bits, and a
355/// composite spends the same bits on a constant opacity instead. Two readings
356/// of one word, so each is written where its own reading is obvious.
357const LAYER_PAINT_SOURCE: u32 = 1 << 29;
358
359/// The premultiplied source colour a hole-punch instance erases with.
360///
361/// Opaque white. Destination-out weights the erase by the source's own *alpha*
362/// and multiplies its colour by zero, so the colour channels never reach the
363/// target and only full alpha matters — it is what makes a fully covered pixel
364/// read exactly `(0, 0, 0, 0)`.
365const PUNCH_SOURCE: u32 = u32::MAX;
366
367/// The label every intermediate page is acquired from the pool under.
368const PAGE_LABEL: &str = "frust-engine layer page";
369
370/// The label every filter round's pass is recorded under.
371const FILTER_LABEL: &str = "frust-engine filter pass";
372
373/// Vertices one filter pass's quad is built from, the same four-vertex
374/// triangle strip every engine program expands an instance into.
375const FILTER_QUAD_VERTICES: u32 = 4;
376
377/// The smallest filter instance buffer the engine allocates, in instances.
378///
379/// A σ-32 blur is ten passes, so this is roughly "one deep blur costs no second
380/// allocation"; the buffer is 32 bytes an instance and grows from here.
381const MIN_FILTER_INSTANCE_CAPACITY: u64 = 16;
382
383static INDEXED_PAINT_WARNING: Once = Once::new();
384
385/// Raised the first time a frame's backdrop blurs are drawn as nothing because
386/// its target can be neither copied nor sampled (see [`RegionCopyMode`]), on
387/// the same once-per-process terms as the compiler's skip warnings.
388static BACKDROP_UNREADABLE_WARNING: Once = Once::new();
389
390/// Vertices the region-copy program's one quad draws (`copy.wgsl`'s
391/// `quad_corner`, a triangle strip).
392const COPY_QUAD_VERTICES: u32 = 4;
393
394/// The bytes of one region-copy instance: four `u16` pairs
395/// ([`crate::gpu::pipelines::copy_vertex_layout`]).
396const COPY_INSTANCE_BYTES: u64 = 16;
397
398/// Raised the first time an atlas region is declined, so a renderer whose
399/// budget and array have gone out of agreement says so once rather than every
400/// frame.
401static ATLAS_REFUSAL_WARNING: Once = Once::new();
402
403/// The CPU phases one [`EngineRenderer::encode_traced`] call splits into.
404///
405/// The GPU half of a frame is [`EngineSpan`]'s, measured by the timestamp ring
406/// at pass boundaries; this is the CPU half, measured by lapping a
407/// [`PhaseClock`] between the encode's own steps. The two answer different
408/// questions and neither substitutes for the other — a text-heavy scene can
409/// cost milliseconds here while its passes cost a fraction of one there.
410///
411/// The phases partition the call in the order they run, so [`Self::total`] is
412/// the whole encode. [`counts`](Self::counts) trails them and is not one of
413/// them.
414///
415/// - [`compile`](Self::compile) — [`SceneCompiler::compile`] end to end,
416/// itself split six ways by [`CompileSpans`].
417/// - [`schedule`](Self::schedule) — building the frame's pass plan and
418/// settling its depth attachment.
419/// - [`paints`](Self::paints) — resolving every draw's paint to a texel of the
420/// encoded-paint texture.
421/// - [`instances`](Self::instances) — building the per-round GPU instance
422/// arrays from the recording.
423/// - [`resize`](Self::resize) — the fallible capacity checks, the image
424/// atlas's own growth, the glyph-rectangle clears an earlier frame's
425/// eviction owes, and the three resource-texture resizes.
426/// - [`upload`](Self::upload) — the frame's queue writes: alpha coverage,
427/// encoded paints, gradient ramps, image texels and the instance arrays.
428/// - [`replay`](Self::replay) — the render-to-atlas pass that draws newly
429/// cached glyphs into their array layers (zero on a steady frame, which
430/// caches none).
431/// - [`pipelines`](Self::pipelines) — taking (or building) this frame's
432/// pipelines and bind groups, and preparing the filter blocks.
433/// - [`record`](Self::record) — recording the frame's passes into the caller's
434/// encoder.
435///
436/// Compiled only under `perf-trace`, with everything that reads it: the whole
437/// `frust-perf enc` route is a `#[cfg]` island rather than a runtime branch the
438/// optimizer is trusted to fold, so a release-lean binary carries none of its
439/// field names, format strings or window (see [`EncodeTrace`]).
440#[cfg(feature = "perf-trace")]
441#[derive(Debug, Clone, Copy, Default)]
442struct EncodeSpans {
443 /// Scene compilation, split further by its own six phases.
444 compile: CompileSpans,
445 /// Whole-call cost of [`SceneCompiler::compile`], call overhead included.
446 ///
447 /// Deliberately measured from outside rather than summed from
448 /// [`Self::compile`]: the difference between the two is the part of the
449 /// call the six inner phases do not cover, and a breakdown that could only
450 /// report its own sum could never show that gap.
451 compile_total: Duration,
452 /// Pass planning and depth settlement.
453 schedule: Duration,
454 /// Paint resolution.
455 paints: Duration,
456 /// Instance-array building.
457 instances: Duration,
458 /// Capacity checks, atlas growth, glyph-rectangle clears, resource resizes.
459 resize: Duration,
460 /// The frame's queue writes.
461 upload: Duration,
462 /// The render-to-atlas pass for newly cached glyphs.
463 replay: Duration,
464 /// Pipeline/bind-group acquisition and filter preparation.
465 pipelines: Duration,
466 /// Recording the frame's passes into the caller's encoder.
467 record: Duration,
468 /// This frame's own draw, strip, alpha and glyph counts.
469 ///
470 /// Last because it is not a phase: it is what the phases above were spent
471 /// on. Carried here because a phase's cost is only readable against the
472 /// work it did — "the walk cost two milliseconds" says nothing on its own,
473 /// while "two milliseconds over three thousand strips and three hundred
474 /// glyphs" says where a lever would have to bite.
475 counts: EncodeCounts,
476}
477
478/// The columns one `frust-perf enc` line reports, in row order.
479///
480/// The first seven are [`CompileSpans`]' own — six phases plus the `glyphs`
481/// subset of the walk — and `compile` after them is the whole compile they sit
482/// inside (so the six sum to at most it, never past it). The next eight are
483/// [`EncodeSpans`]' remaining phases, and `total` closes the line. Pinned as
484/// one list because the line's field order is what a capture is graded by —
485/// the same contract the `frust-perf img` line keeps.
486#[cfg(feature = "perf-trace")]
487const ENCODE_TRACE_COLUMNS: [&str; 17] = [
488 "validate",
489 "prepare",
490 "classify",
491 "admit",
492 "walk",
493 "glyphs",
494 "finish",
495 "compile",
496 "schedule",
497 "paints",
498 "instances",
499 "resize",
500 "upload",
501 "replay",
502 "pipelines",
503 "record",
504 "total",
505];
506
507/// The unitless per-frame counts the line reports after its phases, in row
508/// order — what the phases above were spent on.
509///
510/// The field order is a capture contract: append only. The first five are
511/// per-frame medians over the window. The next five are **sums over the
512/// window** (frames' worth of events, not a typical frame):
513///
514/// - `replays` — frames in which the render-to-atlas replay pass ran.
515/// - `dirty_pages` — atlas pages those replays rendered.
516/// - `atlas_refused` — pages, clears and uploads the replays refused.
517/// - `glyph_misses` — glyphs that took the cache-miss path.
518/// - `glyph_evicted` — atlas rectangles the eviction pass freed.
519///
520/// `run_keys` is a per-frame median again: the distinct run identities (font,
521/// size, hinting) the policy classified in a frame. The replay figures read
522/// zero on a frame whose replay did not run.
523///
524/// The last five cover the staging ring and the bind-group cache. `staged_bytes`
525/// is the frame's staging-slot bytes, a median (zero on the queue-write route).
526/// The other four are **sums over the window** of this frame's change in the
527/// ring's counters and in the bind-group count: `staging_busy` and
528/// `staging_too_large` are frames the ring refused, `staging_buffers` is slot
529/// buffers it created, and `bind_groups` is bind groups the frame path created.
530/// A ring that is off reports zero for all three staging counts.
531#[cfg(feature = "perf-trace")]
532const ENCODE_TRACE_COUNTS: [&str; 16] = [
533 "draws",
534 "strips",
535 "alphas",
536 "glyph_draws",
537 "atlas_glyphs",
538 "replays",
539 "dirty_pages",
540 "atlas_refused",
541 "glyph_misses",
542 "glyph_evicted",
543 "run_keys",
544 "staged_bytes",
545 "staging_busy",
546 "staging_too_large",
547 "staging_buffers",
548 "bind_groups",
549];
550
551/// Index into [`ENCODE_TRACE_COUNTS`] of the first count reported as a sum over
552/// the window rather than its median, up to and excluding
553/// [`ENCODE_TRACE_MEDIAN_AGAIN`]: `replays`, `dirty_pages`, `atlas_refused`,
554/// `glyph_misses` and `glyph_evicted` are events, mostly zero on a frame, so a
555/// median would hide exactly the frames the line exists to count.
556#[cfg(feature = "perf-trace")]
557const ENCODE_TRACE_FIRST_SUMMED: usize = 5;
558
559/// Index into [`ENCODE_TRACE_COUNTS`] of the first count after the summed run
560/// (`run_keys`, a per-frame level, reported as its median).
561#[cfg(feature = "perf-trace")]
562const ENCODE_TRACE_MEDIAN_AGAIN: usize = 10;
563
564/// Index into [`ENCODE_TRACE_COUNTS`] of the first count summed over the window
565/// again, through the end of the line: the staging refusals, buffer creations
566/// and bind-group creations. `staged_bytes`, just before it, is a median.
567#[cfg(feature = "perf-trace")]
568const ENCODE_TRACE_SUMMED_AGAIN: usize = 12;
569
570/// Width of one window row: every phase column followed by every count.
571#[cfg(feature = "perf-trace")]
572const ENCODE_TRACE_ROW: usize = ENCODE_TRACE_COLUMNS.len() + ENCODE_TRACE_COUNTS.len();
573
574/// The staging ring's and the bind-group cache's counters, read at one point so
575/// a frame's row can carry only the change since the previous read.
576#[cfg(feature = "perf-trace")]
577#[derive(Debug, Clone, Copy, Default)]
578struct TraceCounters {
579 busy: u64,
580 too_large: u64,
581 buffers_created: u64,
582 bind_groups: u64,
583}
584
585/// `now - before` as one frame's count, saturating at both ends.
586#[cfg(feature = "perf-trace")]
587fn counter_delta(now: u64, before: u64) -> u32 {
588 u32::try_from(now.saturating_sub(before)).unwrap_or(u32::MAX)
589}
590
591/// What one frame drew, carried beside its phases.
592///
593/// Counts rather than durations, and so reported without a unit suffix.
594/// `perf-trace`-only, with [`EncodeSpans`], which is all that carries it.
595#[cfg(feature = "perf-trace")]
596#[derive(Debug, Clone, Copy, Default)]
597struct EncodeCounts {
598 /// Recorded draws.
599 draws: u32,
600 /// Strips generated.
601 strips: u32,
602 /// Bytes of alpha coverage those strips index — the frame's largest
603 /// single queue write.
604 alphas: u32,
605 /// Draws that painted one glyph outline.
606 glyph_draws: u32,
607 /// How many of those sampled the glyph atlas rather than rasterizing.
608 atlas_glyphs: u32,
609 /// 1 when the render-to-atlas replay ran this frame, else 0.
610 replays: u32,
611 /// Atlas pages the replay rendered this frame.
612 dirty_pages: u32,
613 /// Atlas pages, clears and uploads the replay refused this frame.
614 atlas_refused: u32,
615 /// Glyphs that took the cache-miss path while compiling this frame.
616 glyph_misses: u32,
617 /// Atlas rectangles this frame's eviction freed.
618 glyph_evicted: u32,
619 /// Distinct run identities classified this frame.
620 run_keys: u32,
621 /// Bytes the frame placed in its staging slot, alignment padding included;
622 /// zero on the queue-write route.
623 staged_bytes: u32,
624 /// Frames the staging ring refused this frame because every slot was in
625 /// flight. Filled by [`EncodeTrace::record`], which holds the previous
626 /// reading.
627 staging_busy: u32,
628 /// Frames the staging ring refused this frame because they needed more than
629 /// a slot may hold. Filled by [`EncodeTrace::record`].
630 staging_too_large: u32,
631 /// Slot buffers the staging ring created this frame. Filled by
632 /// [`EncodeTrace::record`].
633 staging_buffers: u32,
634 /// Bind groups the frame path created this frame. Filled by
635 /// [`EncodeTrace::record`].
636 bind_groups: u32,
637}
638
639#[cfg(feature = "perf-trace")]
640impl EncodeCounts {
641 /// What `frame` drew, with the atlas figures the frame's replay and policy
642 /// reported and the bytes it staged. The staging-counter and bind-group
643 /// deltas are left at zero for [`EncodeTrace::record`] to fill.
644 fn of(
645 frame: &CompiledFrame,
646 replayed: bool,
647 report: &AtlasRenderReport,
648 policy: &crate::text::atlas_policy::AtlasTrace,
649 staged: u64,
650 ) -> Self {
651 fn count(len: usize) -> u32 {
652 u32::try_from(len).unwrap_or(u32::MAX)
653 }
654 Self {
655 draws: count(frame.draws().len()),
656 strips: count(frame.strip_buf().len()),
657 alphas: count(frame.alphas().len()),
658 glyph_draws: frame.glyph_draws,
659 atlas_glyphs: frame.atlas_glyph_draws,
660 replays: u32::from(replayed),
661 dirty_pages: report.pages,
662 atlas_refused: report.refused,
663 glyph_misses: policy.glyph_misses,
664 glyph_evicted: policy.glyph_evicted,
665 run_keys: policy.run_keys(),
666 staged_bytes: u32::try_from(staged).unwrap_or(u32::MAX),
667 staging_busy: 0,
668 staging_too_large: 0,
669 staging_buffers: 0,
670 bind_groups: 0,
671 }
672 }
673}
674
675/// Frames one `frust-perf enc` line summarises — one line a second at 60 Hz.
676///
677/// A window rather than a line per frame: the encode is the very thing being
678/// measured, so formatting and logging inside it once per frame would charge
679/// the measurement to its own subject. One line per sixty frames keeps that
680/// charge under a microsecond a frame while still resolving a scenario's
681/// phases (a thirty-second run reports thirty times), and it matches the
682/// cadence `frust-shell-common`'s own rate-limited frame summary already
683/// emits at.
684#[cfg(feature = "perf-trace")]
685const ENCODE_TRACE_WINDOW: usize = 60;
686
687#[cfg(feature = "perf-trace")]
688impl EncodeSpans {
689 /// The whole encode.
690 ///
691 /// Saturating throughout: a diagnostic sum must not take the frame with it
692 /// on overflow (E17).
693 fn total(&self) -> Duration {
694 // `compile.glyphs` is deliberately absent: it is a subset of the walk
695 // inside `compile_total`, and adding it would count text twice.
696 [
697 self.schedule,
698 self.paints,
699 self.instances,
700 self.resize,
701 self.upload,
702 self.replay,
703 self.pipelines,
704 self.record,
705 ]
706 .iter()
707 .fold(self.compile_total, |acc, span| acc.saturating_add(*span))
708 }
709
710 /// This frame's row, in [`ENCODE_TRACE_COLUMNS`] order, in nanoseconds.
711 ///
712 /// Nanoseconds in a `u32` rather than a `Duration` per column: a window of
713 /// sixty rows is then three kilobytes of plain integers to sort, and no
714 /// single phase of one frame reaches the four-second ceiling that would
715 /// saturate one.
716 fn row(&self) -> [u32; ENCODE_TRACE_ROW] {
717 fn ns(span: Duration) -> u32 {
718 u32::try_from(span.as_nanos()).unwrap_or(u32::MAX)
719 }
720 [
721 ns(self.compile.validate),
722 ns(self.compile.prepare),
723 ns(self.compile.classify),
724 ns(self.compile.admit),
725 ns(self.compile.walk),
726 ns(self.compile.glyphs),
727 ns(self.compile.finish),
728 ns(self.compile_total),
729 ns(self.schedule),
730 ns(self.paints),
731 ns(self.instances),
732 ns(self.resize),
733 ns(self.upload),
734 ns(self.replay),
735 ns(self.pipelines),
736 ns(self.record),
737 ns(self.total()),
738 self.counts.draws,
739 self.counts.strips,
740 self.counts.alphas,
741 self.counts.glyph_draws,
742 self.counts.atlas_glyphs,
743 self.counts.replays,
744 self.counts.dirty_pages,
745 self.counts.atlas_refused,
746 self.counts.glyph_misses,
747 self.counts.glyph_evicted,
748 self.counts.run_keys,
749 self.counts.staged_bytes,
750 self.counts.staging_busy,
751 self.counts.staging_too_large,
752 self.counts.staging_buffers,
753 self.counts.bind_groups,
754 ]
755 }
756}
757
758/// The rolling window of per-frame [`EncodeSpans`] one `frust-perf enc` line
759/// is computed over.
760///
761/// Absent altogether from a build without `perf-trace`, rather than inert in
762/// one. The type, its window, its column names and the `frust-perf enc` literal
763/// are all inside a `#[cfg]` island — including the renderer's own field — so
764/// "no frame pays a push, a sort or a format" is what the compiler emitted and
765/// not what the optimizer was expected to prove about a constant branch. That
766/// distinction is the one the release-lean gate measures: it greps a shipping
767/// binary for `frust-perf` and expects to find nothing.
768#[cfg(feature = "perf-trace")]
769#[derive(Debug, Default)]
770struct EncodeTrace {
771 /// One row per frame in the window, in [`ENCODE_TRACE_COLUMNS`] order.
772 window: Vec<[u32; ENCODE_TRACE_ROW]>,
773 /// The column being ranked, kept across windows so a steady stream of
774 /// them allocates once.
775 ranked: Vec<u32>,
776 /// Frames recorded since this renderer was built — the line's own `n`,
777 /// which is the engine's count and deliberately not the host's raw-line
778 /// `n` (a different emitter counting different frames).
779 frames: u64,
780 /// The counters as the previous frame read them, so each row reports only
781 /// its own frame's change.
782 previous: TraceCounters,
783}
784
785#[cfg(feature = "perf-trace")]
786impl EncodeTrace {
787 /// Adds one frame's phases to the window, emitting the window's line when
788 /// it fills.
789 ///
790 /// Called at the very end of the encode, so the formatting a full window
791 /// costs falls outside every phase the line reports — the numbers describe
792 /// the encode, not the encode plus its own accounting.
793 ///
794 /// `now` is the counters read at the same point in the encode; the row
795 /// carries the change since the previous frame's reading.
796 fn record(&mut self, spans: &EncodeSpans, now: TraceCounters) {
797 // Saturating like every other counter on this path: a renderer that
798 // outlived `u64::MAX` frames would report a wrapped `n`, and a wrong
799 // number in a capture is worse than a stuck one.
800 self.frames = self.frames.saturating_add(1);
801 let mut spans = *spans;
802 spans.counts.staging_busy = counter_delta(now.busy, self.previous.busy);
803 spans.counts.staging_too_large = counter_delta(now.too_large, self.previous.too_large);
804 spans.counts.staging_buffers =
805 counter_delta(now.buffers_created, self.previous.buffers_created);
806 spans.counts.bind_groups = counter_delta(now.bind_groups, self.previous.bind_groups);
807 self.previous = now;
808 self.window.push(spans.row());
809 if self.window.len() < ENCODE_TRACE_WINDOW {
810 return;
811 }
812 let line = self.line();
813 self.window.clear();
814 log::info!("{line}");
815 }
816
817 /// The `frust-perf enc` line the current window reports.
818 ///
819 /// The counts `replays` through `glyph_evicted` and `staging_busy` through
820 /// `bind_groups` are window sums; every other column (including `run_keys`
821 /// and `staged_bytes`) is its own median over the window.
822 /// The trailing field is the whole encode's 95th percentile, so a column
823 /// answers "what does this phase usually cost" while the tail answers "how
824 /// bad is a bad frame". Medians are taken per column and so do not sum to
825 /// the `total_us`
826 /// median exactly — they are close on a steady scene, and the gap is
827 /// itself the signal that the phases are not moving together.
828 fn line(&mut self) -> String {
829 // Sized for the longest line the loops below write: a
830 // `<name>_us=<7 digits>.<1>` field per phase, a `<name>=<10 digits>`
831 // field per count, the two `n`/`w` fields and the p95 tail, with room
832 // to spare rather than a byte-exact fit.
833 let mut line = String::with_capacity(896);
834 line.push_str("frust-perf enc n=");
835 line.push_str(&self.frames.to_string());
836 line.push_str(" w=");
837 line.push_str(&self.window.len().to_string());
838 for (column, name) in ENCODE_TRACE_COLUMNS.iter().enumerate() {
839 let p50 = self.percentile(column, 50);
840 line.push(' ');
841 line.push_str(name);
842 line.push_str("_us=");
843 line.push_str(&format_us(p50));
844 }
845 let tail = self.percentile(ENCODE_TRACE_COLUMNS.len() - 1, 95);
846 line.push_str(" total_p95_us=");
847 line.push_str(&format_us(tail));
848 for (offset, name) in ENCODE_TRACE_COUNTS.iter().enumerate() {
849 let column = ENCODE_TRACE_COLUMNS.len() + offset;
850 let summed = (ENCODE_TRACE_FIRST_SUMMED..ENCODE_TRACE_MEDIAN_AGAIN).contains(&offset)
851 || offset >= ENCODE_TRACE_SUMMED_AGAIN;
852 let value = if summed {
853 self.window
854 .iter()
855 .filter_map(|row| row.get(column))
856 .fold(0_u64, |acc, v| acc.saturating_add(u64::from(*v)))
857 } else {
858 u64::from(self.percentile(column, 50))
859 };
860 line.push(' ');
861 line.push_str(name);
862 line.push('=');
863 line.push_str(&value.to_string());
864 }
865 line
866 }
867
868 /// The `pct`-th percentile of `column` over the window, in nanoseconds.
869 ///
870 /// Nearest-rank on the sorted column, which needs no interpolation and so
871 /// reports a value the window really contains — the same choice
872 /// `frust-shell-common`'s frame summary makes, and the one that keeps a
873 /// sixty-sample window honest.
874 fn percentile(&mut self, column: usize, pct: usize) -> u32 {
875 self.ranked.clear();
876 self.ranked
877 .extend(self.window.iter().filter_map(|row| row.get(column)));
878 self.ranked.sort_unstable();
879 let len = self.ranked.len();
880 if len == 0 {
881 return 0;
882 }
883 let rank = (len * pct).div_ceil(100).clamp(1, len);
884 self.ranked.get(rank - 1).copied().unwrap_or(0)
885 }
886}
887
888/// Nanoseconds as microseconds with one decimal — the unit every
889/// `frust-perf enc` field is reported in.
890///
891/// Fixed-point rather than a float format: a phase can be tens of nanoseconds
892/// (an empty punch pass) or milliseconds (a ten-thousand-row table's walk), and
893/// one decimal microsecond reads the same either way without a float's
894/// locale-dependent formatting reaching a capture the harness greps.
895#[cfg(feature = "perf-trace")]
896fn format_us(nanos: u32) -> String {
897 let tenths = u64::from(nanos).div_ceil(100);
898 format!("{}.{}", tenths / 10, tenths % 10)
899}
900
901/// Where one [`EngineRenderer`] construction spent its wall time, measured with
902/// `Instant` reads inside [`EngineRenderer::with_image_sampler`]. Evidence for
903/// the startup sub-step stamps only; no behaviour depends on it.
904#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
905pub struct ConstructionTimings {
906 /// `ShaderLibrary::new` plus `EngineShaders::register_family`: registering
907 /// the shader sources. No module is created here any more — each is created
908 /// on first use, the first frame's on the shader warm-up pool — so this
909 /// reads the registration cost alone.
910 pub shader_modules: Duration,
911 /// `PipelineCache::new`, `warm_up_shaders` (the first-frame module pool's
912 /// spawn) and `warm_up` (enqueue and background-worker spawn).
913 pub warm_up_start: Duration,
914 /// Everything else in construction (routing, gradients, atlas, resources).
915 pub rest: Duration,
916}
917
918/// One surface's 2D render engine: a scene in, recorded passes out.
919///
920/// Create one per surface and keep it across frames — the retained scene
921/// compiler, gradient cache, pipeline cache, resource textures and intermediate
922/// pool are the reason a steady-state frame allocates nothing.
923#[derive(Debug)]
924pub struct EngineRenderer {
925 /// Where [`Self::with_image_sampler`] spent its time (startup evidence only).
926 construction: ConstructionTimings,
927 caps: TierCaps,
928 format: wgpu::TextureFormat,
929 shaders: EngineShaders,
930 pipelines: PipelineCache,
931 compiler: SceneCompiler,
932 gradients: GradientCache,
933 depth: DepthAttachment,
934 targets: IntermediateTargets,
935 /// The bounds an intermediate page is sized between — the policy half of
936 /// page sizing, kept beside the pool the extents are requested from.
937 pages: PageConfig,
938 /// The caller-owned textures a `Command::SceneTexture` resolves against
939 /// (see [`crate::gpu::bindings`]). Its extent half lives on the compiler,
940 /// written by the same two calls that write this.
941 textures: ExternalTextures,
942 resources: FrameResources,
943 scratch: Scratch,
944 /// How each pass's instances are split into paint-class runs — the kill
945 /// switch and the run threshold, settled at construction from
946 /// [`crate::config`] and overridable per renderer.
947 routing: ClassRouting,
948 /// Whether the surface opening is one pass — settled at construction from
949 /// `FRUST_ENGINE_NO_PASS_MERGE` and overridable per renderer (see
950 /// [`Self::set_pass_merge`]).
951 pass_merge: bool,
952 /// The passes the last encoded frame recorded into the caller's encoder,
953 /// in order; refilled every frame, so its allocation is reused.
954 frame_passes: Vec<FramePass>,
955 /// The strip pipelines resolved so far, by state and class, so a frame
956 /// takes each one it draws with by a two-enum lookup rather than by
957 /// rebuilding and re-keying its description through the cache.
958 resolved: ResolvedPipelines,
959 /// The render-to-atlas pass, created by the first frame that caches a
960 /// glyph.
961 ///
962 /// Lazy because it owns a coverage texture, an instance buffer and five
963 /// stand-in bindings of its own, and a renderer that never draws text —
964 /// or one running with `FRUST_ENGINE_NO_ATLAS` — should pay for none of
965 /// them.
966 atlas_glyphs: Option<AtlasRenderer>,
967 /// The compiler the atlas replay lowers a page's recorded commands
968 /// through, sized to the atlas page rather than to the surface.
969 ///
970 /// A second compiler rather than this renderer's own: the frame's compiler
971 /// is mid-frame (its glyph entry map is exactly what the replay is
972 /// draining) and its viewport is the surface's, while a page's commands are
973 /// in page space. Created on the first replay and kept, so a steady stream
974 /// of first-seen glyphs allocates a strip generator once.
975 atlas_lowering: Option<SceneCompiler>,
976 /// What the last frame's replay serviced, for
977 /// [`Self::atlas_render_report`].
978 atlas_report: AtlasRenderReport,
979 /// The filter-data texture, sampler and instance buffer a filter round is
980 /// executed with.
981 ///
982 /// Lazy for the same reason [`Self::atlas_glyphs`] is: a renderer that
983 /// never blurs should own neither a sampler nor a resource texture it will
984 /// not read. Created by the first frame that schedules a filter round.
985 filters: Option<FilterResources>,
986 /// The instance buffer a frame's fragment region copies are drawn from
987 /// ([`RegionCopyMode::Fragment`]). Lazy for [`Self::filters`]' reason: a
988 /// renderer whose frames never copy a region owns none.
989 region_copies: Option<RegionCopyResources>,
990 /// The composite and backdrop groups of the round being recorded, kept
991 /// here so the buffer is reused; emptied at the end of every round.
992 composite_groups: Vec<wgpu::BindGroup>,
993 /// Bind groups the region copy passes built outside the cache.
994 region_copy_groups: u64,
995 /// The retained pages `PushRetained` bodies render into and
996 /// `DrawRetained`s composite from, kept across frames (see
997 /// [`crate::cache::retained`]).
998 retained: RetainedPages,
999 /// The roles the store gave this frame's retained layers, handed to the
1000 /// scheduler — retained across frames for its allocation alone.
1001 retained_layers: RetainedLayers,
1002 /// The scene [`Self::record_range`] projects a range into, kept for its
1003 /// allocation alone; empty until the first range is recorded.
1004 range_scene: Scene,
1005 /// The mapped staging slots a frame's per-frame uploads are written into
1006 /// and copied out of inside the caller's encoder (see
1007 /// [`frust_gpu::staging`]); `None` under `FRUST_ENGINE_NO_STAGING_RING`,
1008 /// which sends every one of them through queue writes instead.
1009 staging: Option<StagingRing>,
1010 /// Bytes the last encoded frame placed in its staging slot — zero when it
1011 /// took the queue-write route.
1012 last_staged: u64,
1013 /// The rolling CPU-phase window `frust-perf enc` lines are emitted from.
1014 ///
1015 /// Absent entirely from a build without `perf-trace`, along with the type
1016 /// itself — see [`EncodeTrace`].
1017 #[cfg(feature = "perf-trace")]
1018 encode_trace: EncodeTrace,
1019}
1020
1021/// Every pipeline a renderer warms up, in the order its background worker
1022/// builds them.
1023///
1024/// [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for)'s list, reordered and extended for the frames that
1025/// follow an install: the worker opens with what an ordinary first frame draws
1026/// (the atlas page pass, the depth-tested surface pass's glyph and solid
1027/// programs, the opaque pass's solid one, then the surface pass's Uber
1028/// program) because that frame waits behind whatever the worker is building,
1029/// and it ends with the surface's layer composite, which only a later
1030/// transition needs.
1031///
1032/// A variant listed twice is queued once, so the reordering needs no
1033/// de-duplication. With the Uber program forced the list is the Uber states
1034/// alone, exactly as [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for) gives them.
1035fn warm_up_list(
1036 shaders: &EngineShaders,
1037 format: wgpu::TextureFormat,
1038 uber_only: bool,
1039) -> Vec<frust_gpu::RenderPipelineDesc> {
1040 ordered_warm_up(
1041 shaders.image_sampler,
1042 uber_only,
1043 |state, class| state.class_desc(shaders, format, class),
1044 || atlas_strip_desc(shaders),
1045 )
1046}
1047
1048/// [`warm_up_list`]'s order and membership over any description type, so it
1049/// can be checked without a device (see [`warm_up_plan`]).
1050fn ordered_warm_up<D>(
1051 image_sampler: bool,
1052 uber_only: bool,
1053 describe: impl Fn(EnginePipeline, PaintClass) -> D,
1054 atlas: impl Fn() -> D,
1055) -> Vec<D> {
1056 let full = warm_up_plan(image_sampler, uber_only, &describe, &atlas);
1057 if uber_only {
1058 return full;
1059 }
1060 let mut descs = Vec::with_capacity(full.len() + 6);
1061 descs.push(atlas());
1062 descs.push(describe(EnginePipeline::StripDepthAlpha, PaintClass::Glyph));
1063 descs.push(describe(
1064 EnginePipeline::StripDepthAlpha,
1065 PaintClass::SolidCoverage,
1066 ));
1067 descs.push(describe(
1068 EnginePipeline::StripOpaque,
1069 PaintClass::SolidCoverage,
1070 ));
1071 descs.push(describe(EnginePipeline::StripDepthAlpha, PaintClass::Uber));
1072 descs.extend(full);
1073 descs.push(describe(
1074 EnginePipeline::StripDepthAlpha,
1075 PaintClass::LayerComposite,
1076 ));
1077 descs
1078}
1079
1080/// The shader modules the first frame after an install draws with, in the
1081/// order it first needs them: the distinct programs of [`warm_up_list`]'s
1082/// opening — the manual family's Uber program the atlas page pass draws with,
1083/// the glyph and solid class programs, and the family's own Uber program —
1084/// or, with the Uber program forced, the Uber module(s) alone.
1085///
1086/// [`EngineRenderer::with_image_sampler`] hands them to
1087/// `PipelineCache::warm_up_shaders`, so they are created side by side at
1088/// install instead of one after another on the worker the frame waits behind.
1089fn first_frame_shaders(shaders: &EngineShaders, uber_only: bool) -> Vec<ShaderId> {
1090 first_frame_programs(
1091 uber_only,
1092 |class| shaders.strip_class(class),
1093 || shaders.atlas_strip,
1094 )
1095}
1096
1097/// [`first_frame_shaders`]' order and membership over any program type, so it
1098/// can be checked against [`ordered_warm_up`]'s opening without a device.
1099/// `program(class)` is the family's program for `class`, `atlas()` the atlas
1100/// page pass's.
1101fn first_frame_programs<P: PartialEq>(
1102 uber_only: bool,
1103 program: impl Fn(PaintClass) -> P,
1104 atlas: impl Fn() -> P,
1105) -> Vec<P> {
1106 let classes: &[PaintClass] = if uber_only {
1107 &[PaintClass::Uber]
1108 } else {
1109 &[
1110 PaintClass::Glyph,
1111 PaintClass::SolidCoverage,
1112 PaintClass::Uber,
1113 ]
1114 };
1115 let mut programs = vec![atlas()];
1116 for class in classes {
1117 let next = program(*class);
1118 if !programs.contains(&next) {
1119 programs.push(next);
1120 }
1121 }
1122 programs
1123}
1124
1125impl EngineRenderer {
1126 /// Builds a renderer for `format` targets on `caps`' adapter, warming
1127 /// every engine pipeline in the background.
1128 ///
1129 /// `pipeline_cache` is the host's persisted driver cache when it has one —
1130 /// `None` on every backend but Vulkan.
1131 ///
1132 /// # Errors
1133 ///
1134 /// [`EngineError::AtlasError`] when the adapter's `resource_texture_dim`
1135 /// is not a power of two of at least [`MIN_RESOURCE_TEXTURE_DIM`]. The
1136 /// strip shader reconstructs a resource texture's width as `1 << bits` and
1137 /// an upload's row stride must satisfy wgpu's copy alignment, so neither
1138 /// the shader's addressing nor the uploads would be valid otherwise.
1139 pub fn new(
1140 device: &wgpu::Device,
1141 caps: &TierCaps,
1142 format: wgpu::TextureFormat,
1143 pipeline_cache: Option<&wgpu::PipelineCache>,
1144 ) -> Result<Self, EngineError> {
1145 Self::with_image_sampler(
1146 device,
1147 caps,
1148 format,
1149 pipeline_cache,
1150 config::image_sampler_enabled(),
1151 )
1152 }
1153
1154 /// [`Self::new`] with the strip program family stated rather than read
1155 /// from `FRUST_ENGINE_IMAGE_SAMPLER` — the programmatic counterpart of
1156 /// that switch, which only sets the family [`Self::new`] picks.
1157 ///
1158 /// With `image_sampler` set, every strip program that reads atlas images
1159 /// filters a bilinear one through a hardware sampler instead of four
1160 /// `textureLoad` taps (see [`crate::gpu::shader_src`]'s *The two program
1161 /// families*), and every strip bind group carries the one linear,
1162 /// clamp-to-edge sampler this renderer creates for it. A family is a
1163 /// construction-time choice rather than a per-frame setter because it
1164 /// decides which programs are compiled and warmed: a caller comparing the
1165 /// two builds one renderer of each.
1166 ///
1167 /// # Errors
1168 ///
1169 /// As [`Self::new`].
1170 pub fn with_image_sampler(
1171 device: &wgpu::Device,
1172 caps: &TierCaps,
1173 format: wgpu::TextureFormat,
1174 pipeline_cache: Option<&wgpu::PipelineCache>,
1175 image_sampler: bool,
1176 ) -> Result<Self, EngineError> {
1177 let dim = caps.resource_texture_dim;
1178 if !dim.is_power_of_two() || dim < MIN_RESOURCE_TEXTURE_DIM {
1179 return Err(EngineError::AtlasError);
1180 }
1181
1182 let started = Instant::now();
1183 let routing = ClassRouting::from_config();
1184 let shaders_started = Instant::now();
1185 let mut library = ShaderLibrary::new();
1186 let shaders = EngineShaders::register_family(&mut library, device, image_sampler);
1187 let shader_modules = shaders_started.elapsed();
1188 let warm_started = Instant::now();
1189 let mut pipelines = PipelineCache::new(Arc::new(library), pipeline_cache.cloned());
1190 // The first frame's modules are created side by side on a short-lived
1191 // pool, started ahead of the worker: the worker's first jobs and the
1192 // frame's own requests then find them built or in flight. Every other
1193 // module is created by whichever build first needs it.
1194 pipelines.warm_up_shaders(device, &first_frame_shaders(&shaders, routing.uber));
1195 pipelines.warm_up(device, &warm_up_list(&shaders, format, routing.uber));
1196 let warm_up_start = warm_started.elapsed();
1197
1198 let level = Level::try_detect().unwrap_or(Level::baseline());
1199 let gradients = GradientCache::for_texture(GradientTextureLayout::square(dim), level);
1200
1201 let mut renderer = Self {
1202 construction: ConstructionTimings::default(),
1203 caps: caps.clone(),
1204 format,
1205 shaders,
1206 pipelines,
1207 // The viewport is re-asserted on every compile, so the extent is
1208 // only an initial allocation hint. `caps` is not: it is what fixes
1209 // the image atlas budget for this renderer's whole life (mobile or
1210 // desktop tier, the adapter's own ceilings, and any
1211 // `FRUST_ENGINE_ATLAS_SIZE` override), and the adapter is known
1212 // exactly here — a compiler built without it would silently keep
1213 // the mobile budget on every device.
1214 compiler: {
1215 let mut compiler = SceneCompiler::for_caps(1, 1, caps);
1216 // A renderer has a store behind its compiler, so retained
1217 // brackets are recorded for it — unless the engine-side kill
1218 // switch says otherwise.
1219 compiler.set_retained_layers(!config::layers_disabled());
1220 compiler
1221 },
1222 gradients,
1223 depth: DepthAttachment::new(),
1224 targets: IntermediateTargets::new(caps),
1225 pages: PageConfig::default(),
1226 textures: ExternalTextures::new(),
1227 resources: FrameResources::new(device, dim, image_sampler),
1228 scratch: Scratch::default(),
1229 routing,
1230 pass_merge: !config::pass_merge_disabled(),
1231 frame_passes: Vec::new(),
1232 resolved: ResolvedPipelines::default(),
1233 atlas_glyphs: None,
1234 atlas_lowering: None,
1235 atlas_report: AtlasRenderReport::default(),
1236 filters: None,
1237 region_copies: None,
1238 composite_groups: Vec::new(),
1239 region_copy_groups: 0,
1240 retained: RetainedPages::for_caps(caps),
1241 retained_layers: RetainedLayers::new(),
1242 range_scene: Scene::new(),
1243 staging: (!config::staging_ring_disabled()).then(StagingRing::new),
1244 last_staged: 0,
1245 #[cfg(feature = "perf-trace")]
1246 encode_trace: EncodeTrace::default(),
1247 };
1248 let rest = started
1249 .elapsed()
1250 .saturating_sub(shader_modules)
1251 .saturating_sub(warm_up_start);
1252 renderer.construction = ConstructionTimings {
1253 shader_modules,
1254 warm_up_start,
1255 rest,
1256 };
1257 Ok(renderer)
1258 }
1259
1260 /// Where this renderer's construction spent its time.
1261 #[must_use]
1262 pub fn construction_timings(&self) -> ConstructionTimings {
1263 self.construction
1264 }
1265
1266 /// The target format this renderer warmed its pipelines for.
1267 #[must_use]
1268 pub fn format(&self) -> wgpu::TextureFormat {
1269 self.format
1270 }
1271
1272 /// Whether this renderer's strip programs filter bilinear atlas images
1273 /// through the hardware sampler (see [`Self::with_image_sampler`]).
1274 #[must_use]
1275 pub fn image_sampler(&self) -> bool {
1276 self.shaders.image_sampler
1277 }
1278
1279 /// The adapter capabilities every sizing decision is made against.
1280 #[must_use]
1281 pub fn caps(&self) -> &TierCaps {
1282 &self.caps
1283 }
1284
1285 /// The pool the engine's off-screen intermediates come from.
1286 #[must_use]
1287 pub fn targets(&self) -> &IntermediateTargets {
1288 &self.targets
1289 }
1290
1291 /// The bounds this renderer sizes intermediate layer pages between.
1292 #[must_use]
1293 pub fn page_config(&self) -> PageConfig {
1294 self.pages
1295 }
1296
1297 /// The atlas geometry image residency allocates within.
1298 ///
1299 /// Derived from the adapter in [`Self::new`], so this is the tier's budget
1300 /// narrowed to what the adapter can create — not a constant.
1301 #[must_use]
1302 pub fn atlas_budget(&self) -> AtlasBudget {
1303 self.compiler.images().budget()
1304 }
1305
1306 /// Re-budget image residency, dropping every image currently resident and
1307 /// the atlas array holding them.
1308 ///
1309 /// An atlas rectangle only means anything against the geometry it was
1310 /// allocated in, so a new budget invalidates every one already handed out —
1311 /// which is why the array, the registry of where each image lives and the
1312 /// bind groups naming that array all go in the same step, and each image
1313 /// re-uploads on the next frame that draws it. A start-up or adapter-change
1314 /// operation, never a per-frame one.
1315 pub fn set_atlas_budget(&mut self, budget: AtlasBudget) {
1316 self.compiler.set_atlas_budget(budget);
1317 self.resources.reset_atlas();
1318 }
1319
1320 /// Replace image residency wholesale, on the same invalidation terms as
1321 /// [`Self::set_atlas_budget`].
1322 ///
1323 /// The programmatic counterpart to `FRUST_ENGINE_NO_ATLAS`: an
1324 /// [`ImageResidency::disabled`] residency takes both atlas classes out of
1325 /// the frame — images are skipped and every glyph is drawn as outline
1326 /// strips — without a process-global environment variable, which is what a
1327 /// caller comparing the two paths on one device needs.
1328 pub fn set_image_residency(&mut self, images: crate::cache::images::ImageResidency) {
1329 self.compiler.set_image_residency(images);
1330 self.resources.reset_atlas();
1331 }
1332
1333 /// How many atlas regions this renderer has declined to write or clear.
1334 ///
1335 /// Zero on every sound frame: residency allocates inside the budget the
1336 /// array is created at, and the array is grown to the depth the frame
1337 /// reports before its regions are written, so a refusal means those two
1338 /// went out of agreement. The count exists so that disagreement is
1339 /// measurable rather than silent — a refused write is a region the frame
1340 /// believed it had filled.
1341 #[must_use]
1342 pub fn refused_atlas_regions(&self) -> u64 {
1343 self.resources.refused_regions
1344 }
1345
1346 /// Finishes pipeline warm-up on the calling thread, returning only once
1347 /// every engine pipeline exists.
1348 ///
1349 /// Warm-up is started in the background by [`Self::new`] and normally
1350 /// needs no attention. Two callers want it forced: a host that must not
1351 /// let the *first* frame pay for a compile, and anything about to drop the
1352 /// `wgpu::Device` shortly after building a renderer — the warm-up worker
1353 /// holds its own handle on that device, and tearing it down while the
1354 /// worker is mid-compile is a driver-level hazard rather than a clean
1355 /// cancellation.
1356 ///
1357 /// Each variant is requested through the cache, which builds a queued one
1358 /// inline and waits for one the worker has already started, so nothing is
1359 /// compiled twice and nothing is left for the worker to claim afterwards.
1360 ///
1361 /// "Every" is the warm-up list: the nine Uber states, plus — unless the
1362 /// Uber program is forced — the warmed paint classes and the surface's
1363 /// layer-composite program (see [`warm_up_list`]). A class variant outside
1364 /// that list is created on the first frame that draws a run of it.
1365 ///
1366 /// Idempotent: once the list is built every request is a cache hit, so a
1367 /// second call compiles nothing. It encodes and submits nothing either —
1368 /// the first frame afterwards is still the first frame.
1369 pub fn finish_warm_up(&mut self, device: &wgpu::Device) {
1370 for desc in warm_up_list(&self.shaders, self.format, self.routing.uber) {
1371 let _ = self.pipelines.get_or_create(device, &desc);
1372 }
1373 }
1374
1375 /// Waits up to `timeout` for the background warm-up [`Self::new`] started
1376 /// to build every pipeline on its list, and returns whether it did.
1377 ///
1378 /// The counterpart of [`Self::finish_warm_up`] that compiles nothing on
1379 /// the calling thread: the worker keeps building at its own pace and this
1380 /// only parks until it is done, so a thread that must not spend its own
1381 /// time compiling can still keep the first frame from being the one that
1382 /// waits on a pipeline. Encodes and submits nothing. Returns at once when
1383 /// the warm-up is already done ([`Self::warm_up_done`]).
1384 pub fn wait_warm_up(&mut self, timeout: Duration) -> bool {
1385 self.pipelines.wait_warm_up(timeout)
1386 }
1387
1388 /// Whether every pipeline on the warm-up list has been built (or failed
1389 /// and been left to its first request), by whichever thread built it.
1390 #[must_use]
1391 pub fn warm_up_done(&self) -> bool {
1392 self.pipelines.warm_up_done()
1393 }
1394
1395 /// Forces every strip run through the whole (Uber) strip program, or
1396 /// lets passes split into paint-class runs again — the programmatic
1397 /// counterpart of `FRUST_ENGINE_UBER_PROGRAM`, which only sets the value a
1398 /// renderer starts with.
1399 ///
1400 /// Takes effect from the next frame. With it set a frame draws exactly
1401 /// what it drew before paint classes existed — one pipeline per pass, the
1402 /// whole `strip.wgsl` program of this renderer's family — which is what a
1403 /// caller comparing the two needs without a process-global environment
1404 /// variable.
1405 pub fn set_uber_program(&mut self, uber: bool) {
1406 self.routing.uber = uber;
1407 }
1408
1409 /// Whether every strip run is drawn with the Uber program.
1410 #[must_use]
1411 pub fn uber_program(&self) -> bool {
1412 self.routing.uber
1413 }
1414
1415 /// Draws a glyph run resolved entirely out of the glyph atlas as glyph
1416 /// quads, or the general way — the programmatic counterpart of
1417 /// `FRUST_ENGINE_NO_GLYPH_QUADS`, which only sets the value a renderer
1418 /// starts with (see [`SceneCompiler::set_glyph_quads`]).
1419 ///
1420 /// Takes effect from the next frame. Both arms draw the same pixels, which
1421 /// is what a caller comparing them in one process checks.
1422 pub fn set_glyph_quads(&mut self, enabled: bool) {
1423 self.compiler.set_glyph_quads(enabled);
1424 }
1425
1426 /// Whether resident glyph runs are drawn as glyph quads.
1427 #[must_use]
1428 pub fn glyph_quads(&self) -> bool {
1429 self.compiler.glyph_quads_enabled()
1430 }
1431
1432 /// Sets the shortest paint-class run worth a pipeline switch of its own
1433 /// (see [`merge_class_runs`]); shorter runs are merged and drawn with the
1434 /// Uber program. `FRUST_ENGINE_RUN_SPLIT_MIN` only sets the value a
1435 /// renderer starts with.
1436 pub fn set_run_split_min(&mut self, min_run: u32) {
1437 self.routing.min_run = min_run;
1438 }
1439
1440 /// The run threshold this renderer splits passes at.
1441 #[must_use]
1442 pub fn run_split_min(&self) -> u32 {
1443 self.routing.min_run
1444 }
1445
1446 /// Records the surface opening as one pass, or as the clear and the
1447 /// opaque strips apart from the first surface round — the programmatic
1448 /// counterpart of `FRUST_ENGINE_NO_PASS_MERGE`, which only sets the value a
1449 /// renderer starts with.
1450 ///
1451 /// Takes effect from the next frame. Unmerged, a frame records the clear
1452 /// and the opaque strips as passes of their own ahead of every round (see
1453 /// the module doc's *The frame's passes*) and draws the same pixels, which
1454 /// is what a caller comparing the two arms in one process needs.
1455 pub fn set_pass_merge(&mut self, merge: bool) {
1456 self.pass_merge = merge;
1457 }
1458
1459 /// Whether the surface opening is recorded as one pass.
1460 #[must_use]
1461 pub fn pass_merge(&self) -> bool {
1462 self.pass_merge
1463 }
1464
1465 /// Routes the frame's per-frame uploads through the staging ring, or
1466 /// through queue writes — the programmatic counterpart of
1467 /// `FRUST_ENGINE_NO_STAGING_RING`, which only sets the value a renderer
1468 /// starts with.
1469 ///
1470 /// Takes effect from the next frame. Both routes upload the same bytes to
1471 /// the same resources, so both draw the same pixels, which is what a caller
1472 /// comparing the two arms in one process checks. Turning the ring off
1473 /// drops its slots; a slot still in flight is kept alive by `wgpu` until
1474 /// its submission retires.
1475 pub fn set_staging_ring(&mut self, enabled: bool) {
1476 match (enabled, self.staging.is_some()) {
1477 (true, false) => self.staging = Some(StagingRing::new()),
1478 (false, true) => self.staging = None,
1479 _ => {}
1480 }
1481 }
1482
1483 /// Whether the frame's per-frame uploads go through the staging ring.
1484 #[must_use]
1485 pub fn staging_ring(&self) -> bool {
1486 self.staging.is_some()
1487 }
1488
1489 /// Bytes the last [`Self::encode`] placed in its staging slot, alignment
1490 /// padding included — zero when the frame's uploads took the queue-write
1491 /// route (the ring off, every slot still in flight, or a frame larger than
1492 /// a slot may hold). A diagnostic: what a test pins to tell the two routes
1493 /// apart.
1494 #[must_use]
1495 pub fn last_frame_staged_bytes(&self) -> u64 {
1496 self.last_staged
1497 }
1498
1499 /// The staging and bind-group counters as they stand now, for the trace's
1500 /// per-frame deltas. A ring that is off reads as zero.
1501 #[cfg(feature = "perf-trace")]
1502 fn trace_counters(&self) -> TraceCounters {
1503 let staging = self.staging_stats().unwrap_or_default();
1504 TraceCounters {
1505 busy: staging.busy,
1506 too_large: staging.too_large,
1507 buffers_created: staging.buffers_created,
1508 bind_groups: self.bind_groups_created(),
1509 }
1510 }
1511
1512 /// What the staging ring has done so far, or `None` with the ring off.
1513 #[must_use]
1514 pub fn staging_stats(&self) -> Option<frust_gpu::staging::StagingStats> {
1515 self.staging.as_ref().map(StagingRing::stats)
1516 }
1517
1518 /// How many bind groups this renderer's frame path has created so far —
1519 /// the strip, external, page-round, composite, backdrop, filter and
1520 /// region-copy groups. A steady frame over unchanged pages creates none, so
1521 /// a test diffs this across two frames to pin that. The glyph-atlas
1522 /// replay's own groups are not counted.
1523 #[must_use]
1524 pub fn bind_groups_created(&self) -> u64 {
1525 self.resources
1526 .groups_created
1527 .saturating_add(self.resources.source_groups.created())
1528 .saturating_add(self.region_copy_groups)
1529 .saturating_add(
1530 self.filters
1531 .as_ref()
1532 .map_or(0, |filters| filters.groups_created),
1533 )
1534 }
1535
1536 /// The passes the last [`Self::encode`] recorded into the caller's
1537 /// encoder, in recording order.
1538 ///
1539 /// Empty after a refused frame, which records no pass at all. A
1540 /// diagnostic: how many passes a frame costs is the number a tile-based
1541 /// GPU pays a whole-framebuffer load and store for, so it is what a test
1542 /// pins and a capture is read against.
1543 #[must_use]
1544 pub fn last_frame_passes(&self) -> &[FramePass] {
1545 &self.frame_passes
1546 }
1547
1548 /// How many render pipelines this renderer has compiled so far.
1549 ///
1550 /// Counts *distinct* pipelines, which is why it is a diagnostic rather
1551 /// than something to wait on: two entries of [`EnginePipeline::ALL`] that
1552 /// differ only in their colour format describe the same pipeline whenever
1553 /// the frame's target format happens to equal
1554 /// [`crate::gpu::pipelines::INTERMEDIATE_FORMAT`], and the cache compiles
1555 /// that one variant once. Use [`Self::finish_warm_up`] to wait.
1556 #[must_use]
1557 pub fn compiled_pipelines(&self) -> u64 {
1558 self.pipelines.compiled_variants()
1559 }
1560
1561 /// States whether a caller-supplied depth attachment already holds the
1562 /// depth this frame should test against — see
1563 /// [`DepthAttachment::set_pre_cleared`].
1564 pub fn set_depth_pre_cleared(&mut self, pre_cleared: bool) {
1565 self.depth.set_pre_cleared(pre_cleared);
1566 }
1567
1568 /// Whether a caller-supplied depth attachment is treated as already
1569 /// populated.
1570 ///
1571 /// The statement is sticky and set once, so a host driving several
1572 /// surfaces (or re-establishing one after a device loss) can read back
1573 /// what this renderer is on rather than tracking it a second time.
1574 #[must_use]
1575 pub fn depth_pre_cleared(&self) -> bool {
1576 self.depth.is_pre_cleared()
1577 }
1578
1579 /// Registers a caller-owned texture so a
1580 /// [`Command::SceneTexture`](frust_scene::Command::SceneTexture) naming
1581 /// `id` draws it, returning whatever was registered under `id` before.
1582 ///
1583 /// `id` is the [`SceneTextureId`] the texture minted for itself
1584 /// (`frust_gpu::Texture::as_scene_texture`), and `size` is its extent in
1585 /// texels — the rectangle a display list's destination is mapped onto.
1586 /// `view` must be a non-array 2D view of a float-sampleable texture
1587 /// carrying `wgpu::TextureUsages::TEXTURE_BINDING`; `wgpu` rejects
1588 /// anything else when the frame's bind group is built.
1589 ///
1590 /// Both halves of the registration land here: the view a pass samples and
1591 /// the extent the compiler composes a paint transform against. Registering
1592 /// is idempotent — re-registering the same id replaces the view and drops
1593 /// the bind groups naming the old one.
1594 ///
1595 /// An extent past `u16::MAX` on either axis, or a zero one, registers
1596 /// nothing and answers `None`: the record the shader reads packs the
1597 /// source region into `u16` halves, so there is no honest rectangle to
1598 /// name. Scenes drawing that id go on drawing nothing.
1599 pub fn bind_texture(
1600 &mut self,
1601 id: SceneTextureId,
1602 size: (u32, u32),
1603 view: wgpu::TextureView,
1604 ) -> Option<wgpu::TextureView> {
1605 self.resources.forget_external(id.get());
1606 if !self.compiler.bind_external_texture(id.get(), size) {
1607 return self.textures.unbind(id);
1608 }
1609 self.textures.bind(id, view)
1610 }
1611
1612 /// Removes the texture registered under `id`, returning its view.
1613 pub fn unbind_texture(&mut self, id: SceneTextureId) -> Option<wgpu::TextureView> {
1614 self.compiler.unbind_external_texture(id.get());
1615 self.resources.forget_external(id.get());
1616 self.textures.unbind(id)
1617 }
1618
1619 /// The view registered under `id`, if any.
1620 #[must_use]
1621 pub fn bound_texture(&self, id: SceneTextureId) -> Option<&wgpu::TextureView> {
1622 self.textures.get(id)
1623 }
1624
1625 /// How many external textures are currently bound.
1626 #[must_use]
1627 pub fn bound_texture_count(&self) -> usize {
1628 self.textures.len()
1629 }
1630
1631 /// Releases everything sized against the old surface extent and
1632 /// re-establishes what the next frame needs at the new one.
1633 ///
1634 /// Pipelines, shader modules, the gradient cache and the resource textures
1635 /// are all extent-independent and deliberately survive: a resize must not
1636 /// cost a pipeline rebuild or a ramp re-bake. What goes is the intermediate
1637 /// pool's parked entries — every one keyed on an extent nothing will ask
1638 /// for again — and the engine-owned depth attachment, which has to match
1639 /// its colour attachment exactly. Reallocating the depth buffer here rather
1640 /// than on the next frame keeps it off the frame path.
1641 ///
1642 /// Every retained page goes too: a page holds its body as it was laid out
1643 /// for the old extent, and the resident-key ledger published after the
1644 /// next frame reports the empty store, so every body is painted (and
1645 /// re-recorded) at the new one.
1646 pub fn resize(&mut self, device: &wgpu::Device, width: u32, height: u32) {
1647 self.targets.drop_parked();
1648 // A cached group would otherwise keep a dropped page's texture alive.
1649 self.resources.source_groups.clear();
1650 self.depth.resize(device, width, height);
1651 self.retained.clear();
1652 }
1653
1654 /// The retained pages this renderer holds resident after its last frame —
1655 /// the renderer's half of the resident-key ledger (see
1656 /// [`crate::cache::retained`]), in no particular order.
1657 pub fn retained_ledger(&self) -> impl Iterator<Item = RetainedEntry> + '_ {
1658 self.retained.entries()
1659 }
1660
1661 /// The retained store's frame clock — how many frames it has planned —
1662 /// for a ledger's diagnostic frame stamp.
1663 #[must_use]
1664 pub fn retained_frame(&self) -> u64 {
1665 self.retained.frame()
1666 }
1667
1668 /// A counter advanced whenever what [`Self::retained_ledger`] reports
1669 /// changes; a host publishing the ledger only when it moved publishes
1670 /// every change and nothing else.
1671 #[must_use]
1672 pub fn retained_generation(&self) -> u64 {
1673 self.retained.generation()
1674 }
1675
1676 /// Whether a frame was refused with [`EngineError::RetainedKeyMissing`]
1677 /// (and the store cleared) since the last call, clearing the flag. A host
1678 /// that sees `true` owes the widget tree one more paint against the
1679 /// now-empty ledger: the refused frame presented nothing.
1680 pub fn take_retained_refusal(&mut self) -> bool {
1681 self.retained.take_refusal()
1682 }
1683
1684 /// Drop every retained page — what a surface recreate or a device-scale
1685 /// change calls for. The next ledger read reports the empty store.
1686 pub fn clear_retained(&mut self) {
1687 // A cached group would otherwise keep a dropped page's texture alive.
1688 self.resources.source_groups.clear();
1689 self.retained.clear();
1690 }
1691
1692 /// The byte budget the retained store is held to.
1693 #[must_use]
1694 pub fn retained_budget(&self) -> RetainedBudget {
1695 self.retained.budget()
1696 }
1697
1698 /// Re-budget the retained store, evicting least recently used pages until
1699 /// what it holds fits.
1700 pub fn set_retained_budget(&mut self, budget: RetainedBudget) {
1701 // A cached group would otherwise keep an evicted page's texture alive.
1702 self.resources.source_groups.clear();
1703 self.retained.set_budget(budget);
1704 }
1705
1706 /// Set whether this renderer records retained brackets at all — the
1707 /// programmatic counterpart of `FRUST_ENGINE_NO_LAYERS` (see
1708 /// [`SceneCompiler::set_retained_layers`]). Turning it off clears the
1709 /// store, since nothing would ever draw from it again.
1710 pub fn set_retained_layers(&mut self, enabled: bool) {
1711 self.compiler.set_retained_layers(enabled);
1712 if !enabled {
1713 self.retained.clear();
1714 }
1715 }
1716
1717 /// Whether this renderer records retained brackets.
1718 #[must_use]
1719 pub fn retained_layers(&self) -> bool {
1720 self.compiler.retained_layers_enabled()
1721 }
1722
1723 /// Takes the time this renderer's pipeline cache has spent resolving
1724 /// variants it did not already hold (waiting on the warm-up worker's
1725 /// in-flight build, or building one inline) since the last call.
1726 ///
1727 /// `perf-trace`-only: it feeds the first frame's
1728 /// `first_submit_pipeline_wait` startup span. Time spent in
1729 /// [`Self::wait_warm_up`] counts too.
1730 #[cfg(feature = "perf-trace")]
1731 pub fn take_pipeline_wait(&mut self) -> Duration {
1732 self.pipelines.take_resolve_wait()
1733 }
1734
1735 /// How many pipelines the warm-up worker has finished building.
1736 #[must_use]
1737 pub fn warm_up_finished(&self) -> u64 {
1738 self.pipelines.warm_up_finished()
1739 }
1740
1741 /// Closes the frame out: ages the intermediate pool by one frame and
1742 /// evicts the gradient cache down to its capacity.
1743 ///
1744 /// Call once per frame, after the frame's commands have been submitted and
1745 /// including frames that drew nothing — those are the frames a parked
1746 /// intermediate ages on. Gradient eviction compacts the packed LUT buffer
1747 /// and rewrites the offsets of the survivors, which is why it belongs at
1748 /// the frame boundary rather than mid-frame, where it would invalidate
1749 /// offsets the frame's own encoded paints already carry.
1750 ///
1751 /// `_queue` is part of the signature because the end-of-frame maintenance
1752 /// this method owns grows queue writes as the engine does (an atlas region
1753 /// cleared after the frame that consumed it, in the reference renderer);
1754 /// it has none of them yet.
1755 pub fn end_frame(&mut self, _queue: &wgpu::Queue) {
1756 self.targets.end_frame();
1757 self.gradients.maintain();
1758 self.resources.source_groups.sweep();
1759 }
1760
1761 /// Compiles `scene` and records the frame's passes into `encoder`.
1762 ///
1763 /// `root` is applied ahead of every command's own transform and
1764 /// `base_color` is what the target is cleared to before anything is drawn.
1765 /// Neither the encoder nor the queue is submitted — see the module header.
1766 ///
1767 /// # Errors
1768 ///
1769 /// [`EngineError::TargetTooLarge`] for a target outside the `u16` device
1770 /// grid the strip pipeline addresses, [`EngineError::InvalidTransform`] for
1771 /// a non-finite transform, [`EngineError::InvalidGeometry`] for non-finite
1772 /// command geometry (rect extents, radii, path points, stroke or dash
1773 /// values), [`EngineError::SchedulerEscalation`] for a layer shape the
1774 /// engine's scheduler does not serve,
1775 /// [`EngineError::IntermediateTextureTooLarge`] for a layer no page can be
1776 /// sized to, [`EngineError::AlphaCapacity`] when a frame's coverage
1777 /// outgrows the alpha texture, and [`EngineError::PaintCapacity`] when its
1778 /// encoded paints or colour ramps outgrow theirs. Every one of them is
1779 /// returned before anything is recorded, uploaded, allocated or submitted,
1780 /// so a refused frame leaves `encoder` exactly as it was found and the
1781 /// renderer's own resources — the atlas array included — exactly as they
1782 /// were. That is what lets the caller skip the frame cleanly (nothing is
1783 /// presented and the previously presented content persists) rather than
1784 /// present it half-drawn, and what keeps a refused frame's image uploads
1785 /// alive for the next frame that is not refused.
1786 #[expect(
1787 clippy::too_many_arguments,
1788 reason = "the seam frust-render drives: device, queue, encoder, scene, \
1789 target, base colour and root transform are each supplied by a \
1790 different owner, so bundling them would only move the \
1791 assembly to every call site"
1792 )]
1793 pub fn encode(
1794 &mut self,
1795 device: &wgpu::Device,
1796 queue: &wgpu::Queue,
1797 encoder: &mut wgpu::CommandEncoder,
1798 scene: &Scene,
1799 target: EngineTarget<'_>,
1800 base_color: Color,
1801 root: Affine,
1802 ) -> Result<(), EngineError> {
1803 self.encode_traced(
1804 device,
1805 queue,
1806 encoder,
1807 scene,
1808 target,
1809 base_color,
1810 root,
1811 FrameTimestamps::inert(),
1812 )
1813 }
1814
1815 /// [`Self::encode`], with each pass's GPU time stamped into `timestamps`.
1816 ///
1817 /// The one difference is the sink: every pass this records asks
1818 /// `timestamps` for its own `timestamp_writes` and takes `None` for an
1819 /// answer, so a frame encoded with [`FrameTimestamps::inert`] — which is
1820 /// exactly what [`Self::encode`] passes — records byte-identical work.
1821 /// Which pass is charged to which span is [`EngineSpan`]'s own
1822 /// documentation; the host owns the ring behind the sink and reads the
1823 /// frame's spans back out of it some frames later (see
1824 /// [`frust_gpu::diag::TimestampRing`]).
1825 ///
1826 /// # Errors
1827 ///
1828 /// Exactly [`Self::encode`]'s, on exactly its terms — a refused frame has
1829 /// recorded no pass, so it has taken no timestamp either and the host
1830 /// abandons the ring's slot rather than mapping it.
1831 #[expect(
1832 clippy::too_many_arguments,
1833 reason = "[`Self::encode`]'s argument list plus the timestamp sink, \
1834 each still supplied by a different owner"
1835 )]
1836 pub fn encode_traced(
1837 &mut self,
1838 device: &wgpu::Device,
1839 queue: &wgpu::Queue,
1840 encoder: &mut wgpu::CommandEncoder,
1841 scene: &Scene,
1842 target: EngineTarget<'_>,
1843 base_color: Color,
1844 root: Affine,
1845 timestamps: FrameTimestamps<'_>,
1846 ) -> Result<(), EngineError> {
1847 // The CPU half of the frame's accounting, lapped phase by phase
1848 // through to the end of the call (see [`EncodeSpans`]). Free without
1849 // `perf-trace` — no clock is read at all.
1850 let mut clock = PhaseClock::start();
1851 // Emptied first, so a frame refused below reports the nothing it
1852 // recorded rather than the last frame's passes. The atlas report is
1853 // the same kind of record: a frame that errors before its replay
1854 // would otherwise carry the last replay's numbers forward.
1855 self.frame_passes.clear();
1856 self.atlas_report = AtlasRenderReport::default();
1857 let size = grid_size(target.width, target.height)?;
1858 let mut frame = self.compiler.compile(scene, root, size)?;
1859 let compile_total = clock.lap();
1860
1861 // The frame's pass plan, settled before anything is allocated or
1862 // recorded: a layer shape this scheduler does not serve, or one larger
1863 // than a page can be sized to, refuses the whole frame here so the
1864 // caller can skip it cleanly rather than present it half-drawn.
1865 //
1866 // The retained store decides first, because its answers are roles the
1867 // scheduler serves: which bodies render into a kept page, where each
1868 // `DrawRetained` composites — or that one cannot be served, which
1869 // refuses the frame here and empties the store (see
1870 // `crate::cache::retained`). Nothing it decides is applied until the
1871 // frame is past its last fallible step.
1872 let retained_plan = self
1873 .retained
1874 .plan(&frame.retained, &mut frame.recorder.layers)?;
1875 retained_roles(&frame, &retained_plan, &mut self.retained_layers);
1876 // Whether the frame's backdrop blurs can be served at all: each one
1877 // reads its region of this very target back (see `RegionCopyMode`).
1878 // A target that can be neither copied nor sampled serves none — every
1879 // backdrop's rectangle is then the transparent draw the compiler
1880 // recorded, drawing nothing, and is scheduled as the ordinary layer it
1881 // is — rather than refusing a frame it can otherwise draw whole.
1882 let copy_mode = RegionCopyMode::of(target.view.texture());
1883 let serve_backdrops = copy_mode != RegionCopyMode::Unavailable;
1884 if !frame.backdrops.is_empty() && !serve_backdrops {
1885 note_backdrop_unreadable(target.view.texture());
1886 }
1887 let backdrops: &[Backdrop] = if serve_backdrops {
1888 &frame.backdrops
1889 } else {
1890 &[]
1891 };
1892 // The largest region a texture-mode frame transfers, which the
1893 // staging texture is grown to before the first pass is recorded.
1894 let staging_extent = backdrops.iter().fold((0_u32, 0_u32), |(w, h), backdrop| {
1895 (
1896 w.max(u32::from(backdrop.region.width())),
1897 h.max(u32::from(backdrop.region.height())),
1898 )
1899 });
1900 let rounds = Schedule::build_frame(
1901 &frame.recorder,
1902 &self.caps,
1903 &self.pages,
1904 &self.retained_layers,
1905 backdrops,
1906 )?;
1907 let ceiling = self.targets.max_texture_size();
1908 for round in &rounds {
1909 if let Some(page) = round.page()
1910 && (page.size.width > ceiling || page.size.height > ceiling)
1911 {
1912 return Err(EngineError::IntermediateTextureTooLarge);
1913 }
1914 }
1915
1916 // Depth is available only when there is an attachment to use and the
1917 // kill switch is off. Settling that before a single instance is built
1918 // is what keeps the opaque/alpha split and the pass shape in agreement.
1919 let depth_enabled = !config::depth_disabled();
1920 if depth_enabled && target.depth.is_none() {
1921 self.depth.ensure(device, target.width, target.height);
1922 }
1923 // Cloned out of the renderer rather than borrowed from it: an engine-
1924 // owned attachment lives in `self.depth`, and recording the frame needs
1925 // `self` mutably (the page pool, the pipeline cache). A `wgpu`
1926 // texture view is a reference-counted handle, so the clone is a
1927 // refcount bump once per frame rather than an allocation.
1928 let depth_view = depth_enabled
1929 .then(|| target.depth.or_else(|| self.depth.owned_view()))
1930 .flatten()
1931 .cloned();
1932 let depth_view = depth_view.as_ref();
1933
1934 // Destination-out erases colour as well as alpha, so a target whose
1935 // alpha is disregarded would take a black rectangle where the display
1936 // list says nothing changes. A frame cleared to an opaque base colour
1937 // is exactly that target — every pixel of it presents opaquely — and is
1938 // the only such statement `encode` is handed, so it is what the skip
1939 // `compile::clear`'s contract calls for is decided on.
1940 let punches = !frame.clears.is_empty() && !is_opaque(base_color);
1941 let schedule_span = clock.lap();
1942
1943 // Paints are resolved before instances are built: an instance names
1944 // its paint by the texel its record starts at, which only exists once
1945 // the frame's ramps are resident and its records are laid out.
1946 let atlas_budget = self.compiler.images().budget();
1947 self.resources
1948 .resolve_paints(&frame, &mut self.gradients, atlas_budget, &self.textures);
1949 let paints_span = clock.lap();
1950 self.scratch.routing = self.routing;
1951 self.scratch.build(
1952 &frame,
1953 &rounds,
1954 depth_view.is_some(),
1955 punches,
1956 &self.resources.paint_slots,
1957 if serve_backdrops {
1958 &frame.backdrops
1959 } else {
1960 &[]
1961 },
1962 );
1963 let instances_span = clock.lap();
1964
1965 // Everything that can fail does so here, ahead of the first
1966 // `begin_render_pass` AND ahead of the first thing this frame changes
1967 // about the renderer's frame-visible state: apart from the engine's
1968 // own depth attachment (re-sized above, an internal resource no pass
1969 // has read yet), a frame refused below has allocated nothing, replaced
1970 // no texture and submitted nothing, which is what lets the caller
1971 // skip it cleanly.
1972 let dim = self.caps.resource_texture_dim;
1973 let alphas_grown = gpu::grow_alpha_texture_height(
1974 self.resources.alphas.height,
1975 frame.alphas().len(),
1976 dim,
1977 )?;
1978 let paints_grown = gpu::paint_texture::grow_encoded_paints_texture_height(
1979 self.resources.paints.height,
1980 self.resources.paint_texels(),
1981 dim,
1982 )?;
1983 let gradients_grown = self.resources.grown_gradient_height(&self.gradients)?;
1984
1985 // Past the last fallible step. The retained store applies its plan
1986 // first — evictions, then a page for every body a round really renders
1987 // into — so every retained page the frame's rounds name exists before
1988 // they are recorded. The evictions include every key the frame
1989 // re-recorded whose page will not hold that recording (refused,
1990 // nested or unrendered), so the ledger published after this frame
1991 // never names a stale page.
1992 let rendered = retained_rendered(&frame, &rounds);
1993 self.retained.commit(
1994 device,
1995 &frame.retained,
1996 &frame.retained_superseded,
1997 &retained_plan,
1998 &rendered,
1999 );
2000
2001 // The atlas is created or grown first, so
2002 // the growth copy's own submit precedes the frame's atlas writes below
2003 // (see `gpu::atlas`) and the array is deep enough for every region they
2004 // name.
2005 self.resources
2006 .ensure_atlas(device, queue, atlas_budget, frame.atlas_layers);
2007
2008 // The glyph atlas, in the order [`crate::gpu::atlas`] documents: the
2009 // rectangles last frame's eviction freed are zeroed first, ahead of
2010 // every write this frame issues, so a rectangle handed straight back
2011 // out cannot be erased after its new occupant landed in it.
2012 //
2013 // Acknowledged only once the writes were really issued — the same
2014 // re-offer contract the image plan keeps below. A frame refused before
2015 // this point leaves every rectangle pending, so the next frame that
2016 // gets here still zeroes it.
2017 if self.clear_glyph_rects(device, queue, &frame) {
2018 self.compiler.acknowledge_glyph_clears();
2019 }
2020
2021 self.resources.resize_alphas(device, alphas_grown);
2022 self.resources.resize_paints(device, paints_grown);
2023 self.resources.resize_gradients(device, gradients_grown);
2024 let resize_span = clock.lap();
2025 // The frame's staging slot, opened only now that nothing can refuse
2026 // the frame: the per-frame uploads below — and each page round's
2027 // viewport uniform in `record_frame` — are written into it and copied
2028 // out of it into `encoder`, ahead of every pass this frame records
2029 // that reads them. `None` sends them through queue writes instead.
2030 let mut stage = self.begin_staging(device);
2031 let atlas_serviced = self.resources.upload(
2032 queue,
2033 encoder,
2034 stage.as_mut(),
2035 &mut frame,
2036 &mut self.gradients,
2037 size,
2038 dim,
2039 );
2040 self.resources
2041 .upload_instances(device, queue, encoder, stage.as_mut(), &self.scratch);
2042
2043 // Residency is committed exactly here: the frame passed every fallible
2044 // step and its evictions and uploads have reached the array, so the
2045 // compiler may stop re-offering them. A frame that returned early above
2046 // never gets here, and its plan is re-offered on the next frame that
2047 // does (see `cache::images`).
2048 if atlas_serviced {
2049 self.compiler.acknowledge_image_plan();
2050 }
2051 let upload_span = clock.lap();
2052
2053 // The glyph pixels themselves, last of the atlas work and strictly
2054 // before the scene pass: every page `glifo` dirtied this frame is
2055 // lowered to strips and drawn into its own array layer, on an encoder
2056 // this call owns and submits (the sanctioned exception to the encode
2057 // contract — see this module's header and `gpu::atlas`). The queue
2058 // writes issued above are flushed ahead of that submit, so the pass
2059 // composites onto a layer whose clears and image uploads have landed.
2060 //
2061 // Driven by the atlas's own pending work, never by this frame's
2062 // surviving draws: `glifo` dirties a page when it *inserts* an entry,
2063 // so a run culled away behind a clip records fills while drawing
2064 // nothing, and a draw-gated replay would leave those commands recorded
2065 // until some later frame happened to run one — by which time eviction
2066 // may have re-let the rectangles they name.
2067 //
2068 // Acknowledged only when the pass really ran, the same way the clears
2069 // above are: acknowledging is what lifts the eviction deferral, so an
2070 // acknowledgement for a replay that returned early would let `glifo`
2071 // free and re-let the very rectangles those commands still name.
2072 #[cfg(feature = "perf-trace")]
2073 let mut replayed = false;
2074 if self.compiler.glyph_replay_pending()
2075 && self.replay_glyph_pages(device, queue, timestamps)
2076 {
2077 self.compiler.acknowledge_glyph_replay();
2078 #[cfg(feature = "perf-trace")]
2079 {
2080 replayed = true;
2081 }
2082 }
2083
2084 let replay_span = clock.lap();
2085
2086 let format = target.format;
2087 let pipelines = self.frame_pipelines(device, format, depth_view.is_some(), copy_mode);
2088 if pipelines.copy.is_some() {
2089 let copies = self
2090 .region_copies
2091 .get_or_insert_with(|| RegionCopyResources::new(device));
2092 if copy_mode == RegionCopyMode::Texture {
2093 copies.ensure_staging(device, staging_extent.0, staging_extent.1);
2094 }
2095 }
2096
2097 // The filter rounds' own resources, created by the first frame that
2098 // schedules one. Only the parameter blocks are uploaded here: a pass's
2099 // instance names the extent the *pool* quantized its destination page
2100 // up to, which only the round that acquires it knows, so the instances
2101 // are written round by round in `record_frame`.
2102 if let Some(pipeline) = pipelines.filter.as_ref() {
2103 let scratch = &self.scratch;
2104 self.filters
2105 .get_or_insert_with(|| FilterResources::new(device))
2106 .prepare(
2107 device,
2108 queue,
2109 pipeline,
2110 &scratch.filter_blocks,
2111 scratch.filter_passes,
2112 );
2113 }
2114
2115 let pipelines_span = clock.lap();
2116
2117 self.record_frame(
2118 device,
2119 queue,
2120 encoder,
2121 stage.as_mut(),
2122 &target,
2123 depth_view,
2124 base_color,
2125 &pipelines,
2126 copy_mode,
2127 timestamps,
2128 );
2129 // Unmapped and fenced on `encoder`: the slot is handed out again only
2130 // once the submission carrying this frame has finished on the GPU.
2131 self.last_staged = stage.as_ref().map_or(0, StagingFrame::used);
2132 if let Some(stage) = stage {
2133 stage.finish(encoder);
2134 }
2135
2136 // The laps above still run without `perf-trace` — `PhaseClock` reads
2137 // no clock there, so each one is `Duration::ZERO` — but nothing records
2138 // them, and the trace they would have fed is not compiled at all.
2139 #[cfg(not(feature = "perf-trace"))]
2140 let _ = (
2141 compile_total,
2142 schedule_span,
2143 paints_span,
2144 instances_span,
2145 resize_span,
2146 upload_span,
2147 replay_span,
2148 pipelines_span,
2149 clock.lap(),
2150 );
2151
2152 // Last, so the window's own formatting is charged to no phase it
2153 // reports. A frame refused above records nothing: its phases are a
2154 // partial encode and would drag every percentile toward a frame that
2155 // was never presented.
2156 #[cfg(feature = "perf-trace")]
2157 let now = self.trace_counters();
2158 #[cfg(feature = "perf-trace")]
2159 self.encode_trace.record(
2160 &EncodeSpans {
2161 compile: frame.compile_spans,
2162 compile_total,
2163 schedule: schedule_span,
2164 paints: paints_span,
2165 instances: instances_span,
2166 resize: resize_span,
2167 upload: upload_span,
2168 replay: replay_span,
2169 pipelines: pipelines_span,
2170 record: clock.lap(),
2171 counts: EncodeCounts::of(
2172 &frame,
2173 replayed,
2174 &self.atlas_report,
2175 self.compiler.glyph_atlas_trace(),
2176 self.last_staged,
2177 ),
2178 },
2179 now,
2180 );
2181
2182 Ok(())
2183 }
2184
2185 /// [`Self::encode_traced`] over the commands of `scene` in `range` alone,
2186 /// cut out by [`project_command_range`] — the overlay spike's way of
2187 /// drawing one segment of a frame into a target of its own (render-model
2188 /// Phase 4, decision D10).
2189 ///
2190 /// The range is projected into a scene this renderer keeps for the
2191 /// purpose (its allocation reused from call to call), carrying
2192 /// `DrawRetained`s only when this renderer retains
2193 /// ([`Self::retained_layers`]) — a renderer with retention off refuses a
2194 /// frame carrying one. The bracket rule is [`project_command_range`]'s.
2195 /// `target`, `base_color` (an overlay passes transparent), `root` and
2196 /// `timestamps` mean exactly what they mean to [`Self::encode_traced`].
2197 ///
2198 /// A range covering the whole scene is still projected (and re-recorded),
2199 /// so a caller that draws the whole frame calls [`Self::encode_traced`]
2200 /// instead; this method is never on a path that does.
2201 ///
2202 /// # Errors
2203 ///
2204 /// Exactly [`Self::encode_traced`]'s, over the projected scene.
2205 #[expect(
2206 clippy::too_many_arguments,
2207 reason = "[`Self::encode_traced`]'s argument list plus the range, each \
2208 still supplied by a different owner"
2209 )]
2210 pub fn record_range(
2211 &mut self,
2212 device: &wgpu::Device,
2213 queue: &wgpu::Queue,
2214 encoder: &mut wgpu::CommandEncoder,
2215 scene: &Scene,
2216 range: Range<usize>,
2217 target: EngineTarget<'_>,
2218 base_color: Color,
2219 root: Affine,
2220 timestamps: FrameTimestamps<'_>,
2221 ) -> Result<RangeCut, EngineError> {
2222 // Taken out for the call (and always put back): `encode_traced`
2223 // borrows the renderer mutably while it reads the scene.
2224 let mut projected = std::mem::take(&mut self.range_scene);
2225 let options = RangeOptions {
2226 keep_draw_retained: self.retained_layers(),
2227 };
2228 let cut = project_command_range(scene, range, options, &mut projected);
2229 let result = self.encode_traced(
2230 device, queue, encoder, &projected, target, base_color, root, timestamps,
2231 );
2232 self.range_scene = projected;
2233 result.map(|()| cut)
2234 }
2235
2236 /// Opens this frame's staging slot, sized for every upload the frame routes
2237 /// through it, or `None` when the ring is off or refuses the frame (every
2238 /// slot still in flight, or more bytes than a slot may hold) — in which
2239 /// case each of those uploads is a queue write, as without the ring.
2240 fn begin_staging(&mut self, device: &wgpu::Device) -> Option<StagingFrame> {
2241 let bytes = self.staging_bytes();
2242 self.staging.as_mut()?.begin(device, bytes)
2243 }
2244
2245 /// An upper bound on the bytes this frame stages, alignment padding
2246 /// included: the whole alpha texture, the whole encoded-paint texture
2247 /// when the frame has paints, the viewport uniform, the instances, and
2248 /// one viewport uniform per scheduled round (more than the page rounds
2249 /// that take one, which is all an upper bound needs).
2250 fn staging_bytes(&self) -> u64 {
2251 let resources = &self.resources;
2252 let texture = |texture: &ResourceTexture| {
2253 gpu::resource_texture_bytes(texture.width, texture.height)
2254 .saturating_add(TEXTURE_COPY_ALIGNMENT)
2255 };
2256 let buffer = |len: u64| len.saturating_add(BUFFER_COPY_ALIGNMENT);
2257 let config = buffer(size_of::<GpuConfig>() as u64);
2258 let (opaque, alpha) = self.scratch.instance_bytes();
2259 let paints = if resources.paints_data.is_empty() {
2260 0
2261 } else {
2262 texture(&resources.paints)
2263 };
2264 let rounds = self.scratch.rounds.len() as u64;
2265 texture(&resources.alphas)
2266 .saturating_add(paints)
2267 .saturating_add(config)
2268 .saturating_add(buffer((opaque.len() + alpha.len()) as u64))
2269 .saturating_add(config.saturating_mul(rounds))
2270 }
2271
2272 /// Creates the render-to-atlas pass on first use.
2273 fn ensure_atlas_renderer(&mut self, device: &wgpu::Device) {
2274 if self.atlas_glyphs.is_none() {
2275 self.atlas_glyphs = Some(AtlasRenderer::new(device, &self.caps));
2276 }
2277 }
2278
2279 /// Zero every atlas rectangle an earlier frame's glyph eviction freed,
2280 /// answering whether they were serviced.
2281 ///
2282 /// Queue writes, issued before this frame's image uploads and before the
2283 /// replay pass's submit — the first of the three orderings
2284 /// [`crate::gpu::atlas`] states. A rectangle the array will not take is
2285 /// counted rather than dropped silently, on the same terms an image region
2286 /// it refuses is.
2287 ///
2288 /// `true` means every rectangle was *offered* to the array — including one
2289 /// it refused, which no later frame could place either — so the caller may
2290 /// stop re-offering them. `false` means there was no array to write to at
2291 /// all, which is the one case where trying again later can succeed. An
2292 /// empty list is serviced trivially.
2293 fn clear_glyph_rects(
2294 &mut self,
2295 device: &wgpu::Device,
2296 queue: &wgpu::Queue,
2297 frame: &CompiledFrame,
2298 ) -> bool {
2299 if frame.glyph_clears.is_empty() {
2300 return true;
2301 }
2302 if self.resources.atlas.is_none() {
2303 return false;
2304 }
2305 self.ensure_atlas_renderer(device);
2306
2307 let refused = {
2308 let Self {
2309 atlas_glyphs,
2310 resources,
2311 ..
2312 } = self;
2313 let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_ref(), resources.atlas.as_ref())
2314 else {
2315 return false;
2316 };
2317 frame
2318 .glyph_clears
2319 .iter()
2320 .filter(|rect| !glyphs.clear_rect(queue, atlas, **rect))
2321 .count()
2322 };
2323
2324 if refused > 0 {
2325 self.resources.note_refused_regions(refused as u64);
2326 }
2327 true
2328 }
2329
2330 /// Draw every atlas page `glifo` dirtied this frame into its own array
2331 /// layer.
2332 ///
2333 /// The pixels of a newly cached glyph, and the last atlas work before the
2334 /// scene pass. Each page's recorded commands are lowered to strips by
2335 /// [`lower_atlas_page`] and drawn through the pipeline
2336 /// [`atlas_strip_desc`] describes; a page the lowering declines is left
2337 /// undrawn and counted, so a glyph whose shape this tier cannot express
2338 /// goes *missing* rather than landing half-painted.
2339 ///
2340 /// Answers whether the pass really ran, on the same terms
2341 /// [`clear_glyph_rects`](Self::clear_glyph_rects) does and for the same
2342 /// reason: `false` means there was no atlas array to draw into at all, so
2343 /// the recorded commands are still recorded and the caller must go on
2344 /// offering them. A page the lowering *declined* is not a `false` — it was
2345 /// offered to the array and counted refused, and no later frame could lower
2346 /// it either.
2347 ///
2348 /// `timestamps` is passed straight through to
2349 /// [`gpu::atlas::AtlasRenderer::render_pending`], which charges each dirty
2350 /// page's own pass to [`EngineSpan::Prepass`] — this is the frame's own
2351 /// Prepass recording site.
2352 #[must_use]
2353 fn replay_glyph_pages(
2354 &mut self,
2355 device: &wgpu::Device,
2356 queue: &wgpu::Queue,
2357 timestamps: FrameTimestamps<'_>,
2358 ) -> bool {
2359 if self.resources.atlas.is_none() {
2360 return false;
2361 }
2362 self.ensure_atlas_renderer(device);
2363 let pipeline = self
2364 .pipelines
2365 .get_or_create(device, &atlas_strip_desc(&self.shaders))
2366 .clone();
2367
2368 let report = {
2369 let Self {
2370 atlas_glyphs,
2371 atlas_lowering,
2372 resources,
2373 compiler,
2374 ..
2375 } = self;
2376 let (Some(glyphs), Some(atlas)) = (atlas_glyphs.as_mut(), resources.atlas.as_ref())
2377 else {
2378 return false;
2379 };
2380
2381 let (width, height) = atlas.size();
2382 let page = (
2383 u16::try_from(width).unwrap_or(u16::MAX),
2384 u16::try_from(height).unwrap_or(u16::MAX),
2385 );
2386 let lowering = atlas_lowering.get_or_insert_with(|| SceneCompiler::new(page.0, page.1));
2387
2388 glyphs.render_pending(
2389 device,
2390 queue,
2391 &pipeline,
2392 atlas,
2393 compiler.glyph_atlas_mut(),
2394 timestamps,
2395 |recorder, buffers| lower_atlas_page(recorder, buffers, lowering, page),
2396 )
2397 };
2398
2399 if report.refused > 0 {
2400 self.resources
2401 .note_refused_regions(u64::from(report.refused));
2402 }
2403 self.atlas_report = report;
2404 true
2405 }
2406
2407 /// What the last frame's render-to-atlas pass serviced.
2408 ///
2409 /// Zero across the board on a steady-state frame: text that hit the cache
2410 /// on every glyph frees no rectangle, queues no pixmap and dirties no page.
2411 #[must_use]
2412 pub fn atlas_render_report(&self) -> AtlasRenderReport {
2413 self.atlas_report
2414 }
2415
2416 /// Builds (or takes from the cache) every pipeline this frame's passes
2417 /// need, and the bind groups each of them will be bound through.
2418 ///
2419 /// All of it happens before the first `begin_render_pass`: a pipeline
2420 /// compiled mid-recording would be the very stall the warm-up exists to
2421 /// avoid. Each pass resolves exactly the paint classes its runs name
2422 /// ([`Scratch`]'s class sets), folded to what its state serves; the bind
2423 /// groups are built once per state, against any one of its class
2424 /// pipelines — every strip pipeline shares one explicit layout, so a group
2425 /// built against one binds under all of them.
2426 fn frame_pipelines(
2427 &mut self,
2428 device: &wgpu::Device,
2429 format: wgpu::TextureFormat,
2430 depth: bool,
2431 copy_mode: RegionCopyMode,
2432 ) -> FramePipelines {
2433 let alpha_variant = if depth {
2434 EnginePipeline::StripDepthAlpha
2435 } else {
2436 EnginePipeline::StripAlpha
2437 };
2438 let punch_variant = if depth {
2439 EnginePipeline::StripDepthDestOut
2440 } else {
2441 EnginePipeline::StripDestOut
2442 };
2443 let uber = ClassSet::only(PaintClass::Uber);
2444 if self.resolved.format != Some(format) {
2445 self.resolved = ResolvedPipelines {
2446 format: Some(format),
2447 ..ResolvedPipelines::default()
2448 };
2449 self.resources.source_groups.clear();
2450 // Derived from the copy pipeline built for the old format.
2451 if let Some(copies) = self.region_copies.as_mut() {
2452 copies.staging_group = None;
2453 }
2454 }
2455
2456 // The surface's own pass is unconditional — it is what the frame's
2457 // bind groups are built against even when it draws nothing — so a frame
2458 // with no surface run still resolves one pipeline for it: the Uber one,
2459 // which the warm-up already built.
2460 let surface_classes = if self.scratch.surface_classes.is_empty() {
2461 uber
2462 } else {
2463 self.scratch.surface_classes
2464 };
2465 let mut frame = FramePipelines {
2466 alpha: (
2467 alpha_variant,
2468 self.class_pipelines(device, format, alpha_variant, surface_classes),
2469 ),
2470 opaque: None,
2471 page: None,
2472 punch: None,
2473 filter: None,
2474 copy: None,
2475 };
2476 if depth && !self.scratch.opaque.is_empty() {
2477 frame.opaque = Some(self.class_pipelines(
2478 device,
2479 format,
2480 EnginePipeline::StripOpaque,
2481 self.scratch.opaque_classes,
2482 ));
2483 }
2484 if self.scratch.page_rounds() > 0 {
2485 let page_classes = if self.scratch.page_classes.is_empty() {
2486 uber
2487 } else {
2488 self.scratch.page_classes
2489 };
2490 frame.page = Some(self.class_pipelines(
2491 device,
2492 format,
2493 EnginePipeline::StripIntermediate,
2494 page_classes,
2495 ));
2496 }
2497 if self.scratch.punches() {
2498 frame.punch = Some((
2499 punch_variant,
2500 self.class_pipelines(
2501 device,
2502 format,
2503 punch_variant,
2504 ClassSet::only(self.routing.punch_class()),
2505 ),
2506 ));
2507 }
2508 if self.scratch.filter_passes > 0 {
2509 // Takes the frame's format like every other variant and ignores
2510 // it: a filter pass only ever writes a pooled page, so its own
2511 // description is pinned to `INTERMEDIATE_FORMAT`. One filter
2512 // pipeline therefore serves a renderer for its whole life,
2513 // whatever its surface is reconfigured to.
2514 if self.resolved.filter.is_none() {
2515 self.resolved.filter = Some(
2516 self.pipelines
2517 .get_or_create(device, &EnginePipeline::Filter.desc(&self.shaders, format))
2518 .clone(),
2519 );
2520 }
2521 frame.filter.clone_from(&self.resolved.filter);
2522 }
2523
2524 if self.scratch.copies > 0 && copy_mode != RegionCopyMode::Unavailable {
2525 // Pinned to `INTERMEDIATE_FORMAT` like the filter pipeline — a copy
2526 // only ever writes a pooled page — so one serves the renderer for
2527 // its whole life whatever the frame's own format.
2528 if self.resolved.copy.is_none() {
2529 self.resolved.copy = Some(
2530 self.pipelines
2531 .get_or_create(device, &EnginePipeline::Copy.desc(&self.shaders, format))
2532 .clone(),
2533 );
2534 }
2535 frame.copy.clone_from(&self.resolved.copy);
2536 }
2537
2538 if let Some(pipeline) = frame.alpha.1.any() {
2539 self.resources
2540 .ensure_bind_groups(device, frame.alpha.0, pipeline, format);
2541 self.resources
2542 .ensure_external_groups(device, frame.alpha.0, pipeline, &self.textures);
2543 }
2544 if let Some(pipeline) = frame.opaque.as_ref().and_then(ClassPipelines::any) {
2545 self.resources.ensure_bind_groups(
2546 device,
2547 EnginePipeline::StripOpaque,
2548 pipeline,
2549 format,
2550 );
2551 }
2552 if let Some(pipeline) = frame.page.as_ref().and_then(ClassPipelines::any) {
2553 self.resources.ensure_bind_groups(
2554 device,
2555 EnginePipeline::StripIntermediate,
2556 pipeline,
2557 format,
2558 );
2559 // A layer's own round draws through this variant, so an external
2560 // texture inside an isolated layer needs its group here too.
2561 self.resources.ensure_external_groups(
2562 device,
2563 EnginePipeline::StripIntermediate,
2564 pipeline,
2565 &self.textures,
2566 );
2567 }
2568 if let Some((variant, pipelines)) = frame.punch.as_ref()
2569 && let Some(pipeline) = pipelines.any()
2570 {
2571 self.resources
2572 .ensure_bind_groups(device, *variant, pipeline, format);
2573 }
2574 self.resources
2575 .ensure_page_configs(device, self.scratch.page_rounds());
2576
2577 frame
2578 }
2579
2580 /// The pipelines `state` draws `classes` with — each class folded to what
2581 /// the state serves — resolved through this renderer's memo first and the
2582 /// pipeline cache (warm-up, steal, or an inline compile on a class's first
2583 /// use) only on a miss.
2584 fn class_pipelines(
2585 &mut self,
2586 device: &wgpu::Device,
2587 format: wgpu::TextureFormat,
2588 state: EnginePipeline,
2589 classes: ClassSet,
2590 ) -> ClassPipelines {
2591 let mut out = ClassPipelines::default();
2592 for class in classes.iter() {
2593 let class = state.fold_class(class);
2594 if out.get(class).is_some() {
2595 continue;
2596 }
2597 let slot = ResolvedPipelines::slot(state, class);
2598 let pipeline = match self.resolved.entries.get(slot).and_then(Option::as_ref) {
2599 Some(pipeline) => pipeline.clone(),
2600 None => {
2601 let pipeline = self
2602 .pipelines
2603 .get_or_create(device, &state.class_desc(&self.shaders, format, class))
2604 .clone();
2605 if let Some(entry) = self.resolved.entries.get_mut(slot) {
2606 *entry = Some(pipeline.clone());
2607 }
2608 pipeline
2609 }
2610 };
2611 out.set(class, pipeline);
2612 }
2613 out
2614 }
2615
2616 /// Records the frame's surface opening, every round's own pass, and the
2617 /// hole punch — the pass list the module doc's *The frame's passes*
2618 /// states — logging each pass it begins into [`Self::frame_passes`].
2619 ///
2620 /// Every pass opened here is ended before the method returns, which is the
2621 /// half of the encode contract a caller cannot check for itself.
2622 ///
2623 /// Each pass names the [`EngineSpan`] it is charged to: the frame's own
2624 /// surface passes are [`EngineSpan::Main`], a layer page round and a
2625 /// filter pass are [`EngineSpan::Composite`]. Several passes per span is
2626 /// the ordinary case and they sum. A pass takes its query pair only when
2627 /// it is really begun, so no pair is ever taken for a pass that was not
2628 /// recorded.
2629 #[expect(
2630 clippy::too_many_arguments,
2631 reason = "one frame's full recording state, each piece owned by a \
2632 different part of the renderer; bundling them would move the \
2633 same assembly one call up"
2634 )]
2635 fn record_frame(
2636 &mut self,
2637 device: &wgpu::Device,
2638 queue: &wgpu::Queue,
2639 encoder: &mut wgpu::CommandEncoder,
2640 mut stage: Option<&mut StagingFrame>,
2641 target: &EngineTarget<'_>,
2642 depth_view: Option<&wgpu::TextureView>,
2643 base_color: Color,
2644 pipelines: &FramePipelines,
2645 copy_mode: RegionCopyMode,
2646 timestamps: FrameTimestamps<'_>,
2647 ) {
2648 let merge = self.pass_merge;
2649 let clear = clear_color(base_color, target.output);
2650 let depth_load = self.depth.load_op(target.depth.is_some());
2651
2652 // Under the kill switch the clear is a pass of its own, recorded first
2653 // and drawing nothing, with the opaque strips in a pass of their own
2654 // after it. It is timed alongside the frame's other surface
2655 // passes even though a backend that samples its counters at the
2656 // vertex/fragment stage boundaries may write nothing for it — an
2657 // untimed pass is a measurement gap, not a wrong measurement (see
2658 // `frust_gpu::diag`).
2659 if !merge {
2660 drop(encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
2661 label: Some(FramePass::Clear.label()),
2662 color_attachments: &[Some(color_attachment(
2663 target.view,
2664 wgpu::LoadOp::Clear(clear),
2665 ))],
2666 depth_stencil_attachment: depth_view.map(|view| depth_attachment(view, depth_load)),
2667 timestamp_writes: timestamps.writes(EngineSpan::Main),
2668 occlusion_query_set: None,
2669 multiview_mask: None,
2670 }));
2671 self.frame_passes.push(FramePass::Clear);
2672 }
2673
2674 // Every field below is reached through `self.<field>` rather than
2675 // through a method: the page pool is borrowed mutably for the whole
2676 // walk while the instance buffer, the bind groups and the plan are
2677 // borrowed immutably, and only disjoint field borrows let those
2678 // coexist.
2679 let Some(instances) = self.resources.instances.as_ref() else {
2680 // No instance has ever been uploaded, so the frame draws nothing
2681 // at all — and still resolves to a clean surface: the opening with
2682 // nothing to draw is exactly the clear.
2683 if merge {
2684 SurfaceOpening {
2685 view: target.view,
2686 depth: depth_view,
2687 clear,
2688 depth_load,
2689 opaque: None,
2690 }
2691 .record(
2692 encoder,
2693 None,
2694 timestamps.writes(EngineSpan::Main),
2695 &mut self.frame_passes,
2696 );
2697 }
2698 #[cfg(debug_assertions)]
2699 self.check_pass_list(merge, false, false, false);
2700 return;
2701 };
2702 let dim = self.caps.resource_texture_dim;
2703 let opaque_count = self.scratch.opaque.len() as u32;
2704 // The alpha region starts where the opaque one ends, so every segment's
2705 // own index is relative to that.
2706 let base = opaque_count;
2707
2708 // The opaque strips, once for the whole frame and ahead of every
2709 // surface draw: the depth they write is what every blended instance
2710 // the frame draws onto its own target, composites included, is then
2711 // tested against.
2712 //
2713 // Once rather than ahead of each surface round, because a frame can
2714 // take several of those (the scheduler cuts one short to hand a page
2715 // group back) and a second recording would re-draw opaque coverage at
2716 // equal stored depth over composites the round before it had already
2717 // blended. Ahead of the layer rounds or after them makes no difference
2718 // to those: a layer round writes a pooled page and reads neither this
2719 // target nor the depth attachment.
2720 let opaque = if opaque_count > 0
2721 && let Some(opaque) = pipelines.opaque.as_ref()
2722 && let Some(opaque_groups) =
2723 self.resources.bind_groups.get(&EnginePipeline::StripOpaque)
2724 {
2725 Some(PassPlan {
2726 pass: FramePass::Opaque,
2727 view: target.view,
2728 load: wgpu::LoadOp::Load,
2729 depth: depth_view,
2730 state: EnginePipeline::StripOpaque,
2731 pipelines: opaque,
2732 groups: opaque_groups,
2733 resources: &opaque_groups.resources,
2734 composites: &[],
2735 // An external paint is never claimed opaque, so the
2736 // depth-writing pass never holds a run.
2737 external_keys: &[],
2738 instances,
2739 base: 0,
2740 // The whole opaque region, `(0, opaque_count)`, cut into its
2741 // own class runs when the frame was planned.
2742 segments: &self.scratch.opaque_segments,
2743 composite_class: self.routing.composite_class(),
2744 })
2745 } else {
2746 None
2747 };
2748 let opaque_drawn = opaque.is_some();
2749
2750 // The surface opening: the clear, then the opaque strips, then — when
2751 // one reaches the surface before anything else does — the first
2752 // surface round's own strips, all in ONE pass. Owed until the first
2753 // pass that touches the surface takes it: the first surface round
2754 // that draws, or a punch issued ahead of any (which takes it as a
2755 // pass of its own, drawing no round). A page or filter round never
2756 // takes it — neither reads nor writes the surface or its depth — so
2757 // the opening simply waits past them. Whatever is still owed at the
2758 // end of the walk is recorded there, which is how a frame whose
2759 // surface draws nothing still resolves to a clean surface.
2760 //
2761 // Under the kill switch the clear was recorded above and the opaque
2762 // strips are recorded here, as passes of their own, and nothing is
2763 // owed.
2764 let mut opening = if merge {
2765 Some(SurfaceOpening {
2766 view: target.view,
2767 depth: depth_view,
2768 clear,
2769 depth_load,
2770 opaque,
2771 })
2772 } else {
2773 if let Some(opaque) = opaque.as_ref() {
2774 record_pass(
2775 encoder,
2776 opaque,
2777 timestamps.writes(EngineSpan::Main),
2778 &mut self.frame_passes,
2779 );
2780 }
2781 None
2782 };
2783
2784 // The destination-out pass's own recording state, resolved once for the
2785 // frame: a punch can now land at any of the walk's cuts, and every one
2786 // of them erases the same target through the same pipeline.
2787 let punch_pass = pipelines.punch.as_ref().and_then(|(variant, punch)| {
2788 Some(PunchPass {
2789 view: target.view,
2790 depth: depth_view,
2791 state: *variant,
2792 pipelines: punch,
2793 class: self.routing.punch_class(),
2794 groups: self.resources.bind_groups.get(variant)?,
2795 instances,
2796 base,
2797 })
2798 });
2799 let punch_drawn = punch_pass.is_some();
2800
2801 // The frame's live pages — the two ping-pong groups and the one spill
2802 // page beside them — each holding the finished page a later round
2803 // composites (see [`crate::schedule`]).
2804 let mut live: [Option<PooledTexture>; MAX_LIVE_PAGES] = [const { None }; MAX_LIVE_PAGES];
2805 let mut page_slot = 0_usize;
2806 // The fragment region copies' own instance slots, one per copy round.
2807 let mut copy_slot = 0_u32;
2808
2809 for plan in &self.scratch.rounds {
2810 let own = match plan.page {
2811 None => None,
2812 // A round continuing a page an earlier round of the same layer
2813 // opened takes that very texture back out of its group: a fresh
2814 // one from the pool would hold the previous holder's pixels
2815 // instead of the half already drawn.
2816 Some(page) if page.continued => match live[page.parity.index()].take() {
2817 Some(pooled) => Some((page.parity, pooled)),
2818 // Unreachable: the round that opened the page put it in
2819 // this group, and no round between the two releases it.
2820 None => continue,
2821 },
2822 Some(page) => match self.targets.acquire(
2823 device,
2824 page.size.width,
2825 page.size.height,
2826 PAGE_LABEL,
2827 ) {
2828 IntermediateTexture::Texture(pooled) => Some((page.parity, pooled)),
2829 // Unreachable: every page extent was checked against this
2830 // pool's own ceiling before the first pass was recorded.
2831 // Skipping the round draws less rather than taking a device
2832 // error mid-frame.
2833 IntermediateTexture::TooLarge { .. } => continue,
2834 },
2835 };
2836
2837 // A region-copy round seeds its page with a region of the frame's
2838 // own target, read back after everything painted before it in
2839 // painter order — so the opening, which clears the target and
2840 // draws the opaque strips, is recorded first if it is still owed
2841 // (the scheduler already cut the surface round ahead of it).
2842 if let Some(source) = plan.copy {
2843 if let Some((_, pooled)) = own.as_ref() {
2844 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
2845 // The staging texture's group is stable until the texture
2846 // grows, so a texture-mode copy builds it once.
2847 if copy_mode == RegionCopyMode::Texture
2848 && let (Some(copies), Some(pipeline)) =
2849 (self.region_copies.as_mut(), pipelines.copy.as_ref())
2850 && copies.staging_group.is_none()
2851 && let Some(staging) = copies.staging.as_ref()
2852 {
2853 let group = copy_source_group(device, pipeline, &staging.view);
2854 copies.staging_group = Some(group);
2855 self.region_copy_groups = self.region_copy_groups.saturating_add(1);
2856 }
2857 let built = RegionCopyPass {
2858 mode: copy_mode,
2859 target: target.view,
2860 page: pooled,
2861 source,
2862 pipeline: pipelines.copy.as_ref(),
2863 resources: self.region_copies.as_ref(),
2864 slot: copy_slot,
2865 }
2866 .record(
2867 device,
2868 queue,
2869 encoder,
2870 timestamps.writes(EngineSpan::Composite),
2871 );
2872 self.region_copy_groups = self.region_copy_groups.saturating_add(built);
2873 copy_slot = copy_slot.saturating_add(1);
2874 self.frame_passes.push(FramePass::Copy);
2875 }
2876 settle_pages(&mut self.targets, &mut live, plan.released, own);
2877 continue;
2878 }
2879
2880 // A filter round is only a filter pass: no strip instance, no
2881 // composite, no viewport uniform of its own — the pass maps NDC
2882 // against the destination extent its own instance carries, which is
2883 // why that instance is written here, where the pool's quantized
2884 // extent is finally known.
2885 if let Some(filter) = plan.filter {
2886 if let Some((_, pooled)) = own.as_ref()
2887 && let Some(pipeline) = pipelines.filter.as_ref()
2888 && let Some(filters) = self.filters.as_ref()
2889 {
2890 let (width, height) = pooled.size();
2891 filters.write_instance(
2892 queue,
2893 filter.instance,
2894 &FilterInstanceData::new(
2895 &filter.step,
2896 filter.data_offset,
2897 // Both pages hold the layer at their own origin, so
2898 // neither region is offset within its page.
2899 (0, 0),
2900 (0, 0),
2901 SizeU16::from_wh(
2902 u16::try_from(width).unwrap_or(u16::MAX),
2903 u16::try_from(height).unwrap_or(u16::MAX),
2904 ),
2905 filter.original,
2906 ),
2907 );
2908 // A group with no live page samples the transparent
2909 // placeholder, which filters nothing — the same "draw less,
2910 // never wrong" answer an unresolvable paint gets.
2911 // Unreachable: the round that wrote this pass's source is
2912 // the one before it, and nothing between the two releases
2913 // that group.
2914 let (source, key) = match live[filter.source.index()].as_ref() {
2915 Some(pooled) => (pooled.view(), pooled_source(pooled)),
2916 None => (
2917 &self.resources.placeholders.layer_input,
2918 GroupSource::Placeholder,
2919 ),
2920 };
2921 let source_group = self.resources.source_groups.get_or_build(
2922 SourceGroupKey {
2923 kind: GroupKind::FilterSource,
2924 source: key,
2925 },
2926 || filters.source_group(device, pipeline, source),
2927 );
2928 filters.record_pass_with(
2929 device,
2930 encoder,
2931 &FilterPassPlan {
2932 label: FILTER_LABEL,
2933 pipeline,
2934 dest: pooled.view(),
2935 source,
2936 instance: filter.instance,
2937 },
2938 timestamps.writes(EngineSpan::Composite),
2939 Some(&source_group),
2940 );
2941 self.frame_passes.push(FramePass::Filter);
2942 }
2943
2944 settle_pages(&mut self.targets, &mut live, plan.released, own);
2945 continue;
2946 }
2947
2948 // The off-screen texture this round draws into, when it is not the
2949 // surface: the pooled page acquired above, or the retained page the
2950 // store bound to this round's body (`crate::cache::retained`). A
2951 // retained round whose page is missing — unreachable, the store
2952 // committed one for every body a round renders — draws nothing
2953 // rather than drawing onto the surface.
2954 let retained_page = plan
2955 .retained
2956 .and_then(|retained| self.retained.page_for(retained.bracket));
2957 if plan.retained.is_some() && retained_page.is_none() {
2958 settle_pages(&mut self.targets, &mut live, plan.released, own);
2959 continue;
2960 }
2961 let offscreen = own
2962 .as_ref()
2963 .map(|(_, pooled)| (pooled.view(), pooled.size()))
2964 .or(retained_page);
2965 let continued = plan.page.is_some_and(|page| page.continued)
2966 || plan.retained.is_some_and(|retained| retained.continued);
2967
2968 // The round's viewport uniform. A page's is written here rather
2969 // than with the frame's other uploads because only the pool knows
2970 // the extent it quantized the request up to, and NDC is computed
2971 // against the attachment's real extent. Distinct buffers, so the
2972 // write ordering against the frame's own config never matters.
2973 // Staged, its copy lands in the encoder here — between passes,
2974 // immediately ahead of the one round that reads it.
2975
2976 // The page-config buffer this round binds; the surface's own
2977 // config has no slot and is named by the maximum.
2978 let config_slot = offscreen.map_or(u32::MAX, |_| {
2979 u32::try_from(page_slot).unwrap_or(u32::MAX - 1)
2980 });
2981 let config = match offscreen {
2982 None => &self.resources.config,
2983 Some((_, (width, height))) => {
2984 let Some(config) = self.resources.page_configs.get(page_slot) else {
2985 continue;
2986 };
2987 let bytes = GpuConfig::new(width, height, dim, dim);
2988 let bytes = bytemuck::bytes_of(&bytes);
2989 let staged = stage
2990 .as_deref_mut()
2991 .is_some_and(|stage| stage.copy_to_buffer(encoder, bytes, config, 0));
2992 if !staged {
2993 queue.write_buffer(config, 0, bytes);
2994 }
2995 page_slot = page_slot.saturating_add(1);
2996 config
2997 }
2998 };
2999
3000 let variant = match offscreen {
3001 None => pipelines.alpha.0,
3002 Some(_) => EnginePipeline::StripIntermediate,
3003 };
3004 let class_pipelines = match (offscreen, pipelines.page.as_ref()) {
3005 (None, _) => &pipelines.alpha.1,
3006 (Some(_), Some(page)) => page,
3007 (Some(_), None) => continue,
3008 };
3009 // The pipeline group 0 is built against: any of the pass's class
3010 // pipelines will do, since every strip pipeline shares one layout.
3011 let Some(pipeline) = class_pipelines.any() else {
3012 continue;
3013 };
3014 let Some(groups) = self.resources.bind_groups.get(&variant) else {
3015 continue;
3016 };
3017
3018 // A page round binds its own viewport uniform; the root round's is
3019 // already the one the shared set carries. Cached by the config
3020 // slot: the buffers are grown-only, so the group stays valid until
3021 // the coverage view it also names is recreated.
3022 let page_resources = offscreen.map(|_| {
3023 self.resources.source_groups.get_or_build(
3024 SourceGroupKey {
3025 kind: GroupKind::Resources { slot: config_slot },
3026 source: GroupSource::Placeholder,
3027 },
3028 || {
3029 resources_bind_group(
3030 device,
3031 pipeline,
3032 &self.resources.alphas.view,
3033 config,
3034 &self.resources.placeholders.layer_input,
3035 )
3036 },
3037 )
3038 });
3039 let resources = page_resources.as_ref().unwrap_or(&groups.resources);
3040
3041 let segments = self
3042 .scratch
3043 .segments
3044 .get(plan.segments.clone())
3045 .unwrap_or(&[]);
3046 let mut composites = std::mem::take(&mut self.composite_groups);
3047 composites.clear();
3048 for segment in segments {
3049 // A backdrop's draws sample the page holding its blurred
3050 // region through the same group-0 layer input a composite
3051 // reads, so its group is built here, in segment order, beside
3052 // the composites' own.
3053 if let Segment::Backdrop(_, _, parity) = *segment {
3054 let (view, source) = match live[parity.index()].as_ref() {
3055 Some(pooled) => (pooled.view(), pooled_source(pooled)),
3056 None => (
3057 &self.resources.placeholders.layer_input,
3058 GroupSource::Placeholder,
3059 ),
3060 };
3061 composites.push(self.resources.source_groups.get_or_build(
3062 SourceGroupKey {
3063 kind: GroupKind::Resources { slot: config_slot },
3064 source,
3065 },
3066 || {
3067 resources_bind_group(
3068 device,
3069 pipeline,
3070 &self.resources.alphas.view,
3071 config,
3072 view,
3073 )
3074 },
3075 ));
3076 }
3077 if let Segment::Composite(_, source) = *segment {
3078 // A group with no live page — or a retained bracket with no
3079 // page bound — samples the transparent placeholder, which
3080 // composites nothing: the same "draw less, never wrong"
3081 // answer an unresolvable paint gets.
3082 let placeholder = (
3083 &self.resources.placeholders.layer_input,
3084 GroupSource::Placeholder,
3085 );
3086 let (view, source) = match source {
3087 CompositeSource::Page(parity) => live[parity.index()]
3088 .as_ref()
3089 .map_or(placeholder, |pooled| (pooled.view(), pooled_source(pooled))),
3090 CompositeSource::Retained(bracket) => self
3091 .retained
3092 .page_for(bracket)
3093 .map_or(placeholder, |(view, size)| {
3094 (
3095 view,
3096 GroupSource::Retained {
3097 view: std::ptr::from_ref(view) as usize,
3098 generation: self.retained.generation(),
3099 size,
3100 },
3101 )
3102 }),
3103 };
3104 composites.push(self.resources.source_groups.get_or_build(
3105 SourceGroupKey {
3106 kind: GroupKind::Resources { slot: config_slot },
3107 source,
3108 },
3109 || {
3110 resources_bind_group(
3111 device,
3112 pipeline,
3113 &self.resources.alphas.view,
3114 config,
3115 view,
3116 )
3117 },
3118 ));
3119 }
3120 }
3121
3122 // A page round draws into an off-screen layer page, the frame's own
3123 // rounds into the surface — the split the two spans name.
3124 let span = if offscreen.is_some() {
3125 EngineSpan::Composite
3126 } else {
3127 EngineSpan::Main
3128 };
3129
3130 let (view, pass, load, depth) = match offscreen {
3131 Some((view, _)) => (
3132 view,
3133 FramePass::Page,
3134 // A pooled texture holds whatever its last holder left
3135 // there, and a retained one whatever this body's last frame
3136 // left, so a layer's first round into a page clears it —
3137 // and a round continuing that same page loads it, because
3138 // clearing again would wipe the half already drawn.
3139 if continued {
3140 wgpu::LoadOp::Load
3141 } else {
3142 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT)
3143 },
3144 None,
3145 ),
3146 None => (
3147 target.view,
3148 FramePass::Surface,
3149 wgpu::LoadOp::Load,
3150 depth_view,
3151 ),
3152 };
3153
3154 // A surface plan with nothing of its own to draw records no pass:
3155 // loading and storing the target unchanged is exactly nothing. A
3156 // punch cut falling at the very start of a round — a `ClearRect`
3157 // recorded before anything the round draws — is how one arises, and
3158 // so is a surface whose every span went to the opaque strips; the
3159 // opening stays owed past it either way. A *page* plan with no
3160 // segments still records its pass, because that is what clears the
3161 // pooled page.
3162 if !segments.is_empty() || offscreen.is_some() {
3163 let round = PassPlan {
3164 pass,
3165 view,
3166 load,
3167 depth,
3168 state: variant,
3169 pipelines: class_pipelines,
3170 groups,
3171 resources,
3172 composites: &composites,
3173 external_keys: self.resources.external_runs.keys(),
3174 instances,
3175 base,
3176 segments,
3177 composite_class: self.routing.composite_class(),
3178 };
3179 // The first surface round that draws takes the opening: one
3180 // pass clears, draws the opaque strips, then this round's own
3181 // segments in order — the pipeline switching between them
3182 // inside the pass — and ends where this round's pass would
3183 // have ended.
3184 let opening = if offscreen.is_none() {
3185 opening.take()
3186 } else {
3187 None
3188 };
3189 match opening {
3190 Some(opening) => opening.record(
3191 encoder,
3192 Some(&round),
3193 timestamps.writes(span),
3194 &mut self.frame_passes,
3195 ),
3196 None => record_pass(
3197 encoder,
3198 &round,
3199 timestamps.writes(span),
3200 &mut self.frame_passes,
3201 ),
3202 }
3203 }
3204
3205 // Handed back empty so the buffer is reused and no cached group is
3206 // held past the round.
3207 composites.clear();
3208 self.composite_groups = composites;
3209
3210 settle_pages(&mut self.targets, &mut live, plan.released, own);
3211
3212 // The cut this plan ends at, erased once the ops before it have
3213 // been recorded and before the plan after it draws a thing. Inert
3214 // on every plan the walk did not cut, which is every plan of a
3215 // frame that punches nothing. A punch issued before the opening
3216 // was taken has to erase a cleared surface, so the opening is
3217 // recorded first, as a pass of its own.
3218 if let Some(punch) = punch_pass.as_ref()
3219 && plan.punch.1 > 0
3220 {
3221 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
3222 punch.record(encoder, plan.punch, timestamps, &mut self.frame_passes);
3223 }
3224 }
3225
3226 // An opening no surface round took — a frame whose surface draws
3227 // nothing of its own past the opaque strips, or nothing at all — is
3228 // recorded here, ahead of the trailing punch it must precede.
3229 record_owed_opening(&mut opening, encoder, timestamps, &mut self.frame_passes);
3230
3231 // The punches past every op of the frame, in the position the pass held
3232 // unconditionally before painter order was restored.
3233 if let Some(punch) = punch_pass.as_ref() {
3234 punch.record(
3235 encoder,
3236 self.scratch.punch,
3237 timestamps,
3238 &mut self.frame_passes,
3239 );
3240 }
3241
3242 for page in live.into_iter().flatten() {
3243 self.targets.release(page);
3244 }
3245
3246 #[cfg(debug_assertions)]
3247 self.check_pass_list(merge, true, opaque_drawn, punch_drawn);
3248 #[cfg(not(debug_assertions))]
3249 let _ = (opaque_drawn, punch_drawn);
3250 }
3251
3252 /// Checks that the passes the walk just recorded are exactly the ones
3253 /// [`Scratch::plan_passes`] says a frame of this shape records — the
3254 /// host-testable statement of the pass list, held to the recording on
3255 /// every frame a debug build encodes.
3256 #[cfg(debug_assertions)]
3257 fn check_pass_list(&self, merge: bool, instances: bool, opaque: bool, punch: bool) {
3258 let mut planned = Vec::new();
3259 self.scratch.plan_passes(
3260 &PassShape {
3261 merge,
3262 instances,
3263 opaque,
3264 punch,
3265 },
3266 &mut planned,
3267 );
3268 debug_assert_eq!(
3269 planned, self.frame_passes,
3270 "the recorded pass list disagrees with the planned one"
3271 );
3272 }
3273}
3274
3275/// The cache key naming `pooled`'s texture as a group's layer input.
3276fn pooled_source(pooled: &PooledTexture) -> GroupSource {
3277 GroupSource::Pooled(pooled.texture().as_scene_texture())
3278}
3279
3280/// One render pass a frame recorded into the caller's encoder — what
3281/// [`EngineRenderer::last_frame_passes`] lists, in recording order.
3282///
3283/// The module doc's *The frame's passes* is the contract these name. The
3284/// glyph-atlas replay's passes are not among them: they are recorded on an
3285/// encoder of the engine's own, ahead of the frame (see [`crate::gpu::atlas`]).
3286#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
3287pub enum FramePass {
3288 /// The surface opening: clears the colour target (and the depth
3289 /// attachment, unless pre-cleared), then draws the opaque strips and the
3290 /// first surface round that draws, whichever of the two the frame has.
3291 /// The first pass over the surface whenever passes are merged.
3292 Opening,
3293 /// The clear alone, drawing nothing — the first pass of a frame recorded
3294 /// with passes unmerged.
3295 Clear,
3296 /// The opaque strips alone, depth-writing, loading what the clear left —
3297 /// recorded only with passes unmerged.
3298 Opaque,
3299 /// A surface round's blended strips, loading the surface.
3300 Surface,
3301 /// A layer page round, rendering into a pooled page.
3302 Page,
3303 /// One pass of a filter's sequence, between two pooled pages.
3304 Filter,
3305 /// A region copy: a region of the frame's own target read back into a
3306 /// pooled page by one pass that clears the page and draws the region into
3307 /// it — preceded, on a target that can only be copied, by a transfer into
3308 /// the renderer's staging texture (see [`RegionCopyMode`]) — ahead of a
3309 /// backdrop blur's filter passes.
3310 Copy,
3311 /// The destination-out hole punch, at a cut or after the last round.
3312 Punch,
3313}
3314
3315impl FramePass {
3316 /// The label the pass is recorded under, as a GPU capture shows it.
3317 #[must_use]
3318 pub const fn label(self) -> &'static str {
3319 match self {
3320 FramePass::Opening => "frust-engine surface opening",
3321 FramePass::Clear => "frust-engine clear",
3322 FramePass::Opaque => "frust-engine opaque strips",
3323 FramePass::Surface => "frust-engine alpha strips",
3324 FramePass::Page => PAGE_LABEL,
3325 FramePass::Filter => FILTER_LABEL,
3326 FramePass::Copy => "frust-engine region copy",
3327 FramePass::Punch => "frust-engine hole punch",
3328 }
3329 }
3330
3331 /// Whether the pass renders into the frame's own target rather than into
3332 /// a pooled page.
3333 #[must_use]
3334 pub const fn targets_surface(self) -> bool {
3335 !matches!(self, FramePass::Page | FramePass::Filter | FramePass::Copy)
3336 }
3337}
3338
3339/// What a frame's pass list depends on beyond its plan, as
3340/// [`Scratch::plan_passes`] reads it.
3341#[cfg(any(test, debug_assertions))]
3342#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3343struct PassShape {
3344 /// Whether the surface opening is merged (the renderer's
3345 /// [`EngineRenderer::pass_merge`]).
3346 merge: bool,
3347 /// Whether the frame has an instance buffer to draw from at all.
3348 instances: bool,
3349 /// Whether the frame draws opaque strips — depth in play, opaque spans to
3350 /// route, and their pipelines resolved.
3351 opaque: bool,
3352 /// Whether the frame records its hole punches.
3353 punch: bool,
3354}
3355
3356/// The pipelines one frame's passes are recorded with, resolved once before the
3357/// first `begin_render_pass`.
3358///
3359/// Only `alpha` is unconditional: the rest exist exactly when the frame has
3360/// work for them, so a plain frame compiles and binds nothing it will not draw.
3361#[derive(Debug)]
3362struct FramePipelines {
3363 /// The blended pass over the frame's own target, with its variant — which
3364 /// of the two it is depends on whether depth is in play.
3365 alpha: (EnginePipeline, ClassPipelines),
3366 /// The depth-writing pass, when depth is available and the frame has
3367 /// fully covered spans to route into it.
3368 opaque: Option<ClassPipelines>,
3369 /// The pass a layer page is rendered through, when the frame has one.
3370 page: Option<ClassPipelines>,
3371 /// The destination-out pass, with its variant, when the frame punches.
3372 punch: Option<(EnginePipeline, ClassPipelines)>,
3373 /// The pass one filter round runs through, when the frame filters a layer.
3374 filter: Option<wgpu::RenderPipeline>,
3375 /// The pass a fragment region copy runs through, when the frame copies a
3376 /// region of a target it can only sample ([`RegionCopyMode::Fragment`]).
3377 copy: Option<wgpu::RenderPipeline>,
3378}
3379
3380/// How a renderer cuts its passes into paint-class runs.
3381///
3382/// Settled from [`crate::config`] when the renderer is built
3383/// (`FRUST_ENGINE_UBER_PROGRAM`, `FRUST_ENGINE_RUN_SPLIT_MIN`) and overridable
3384/// per renderer, so a caller can compare the two arms in one process.
3385#[derive(Debug, Clone, Copy, PartialEq, Eq)]
3386struct ClassRouting {
3387 /// The kill switch: every run is drawn with [`PaintClass::Uber`] and no
3388 /// segment is split.
3389 uber: bool,
3390 /// The shortest run worth a pipeline switch of its own.
3391 min_run: u32,
3392}
3393
3394impl Default for ClassRouting {
3395 fn default() -> Self {
3396 Self {
3397 uber: false,
3398 min_run: crate::gpu::strips::RUN_SPLIT_MIN,
3399 }
3400 }
3401}
3402
3403impl ClassRouting {
3404 /// The routing a renderer starts with, from the process configuration.
3405 fn from_config() -> Self {
3406 Self {
3407 uber: config::uber_program_forced(),
3408 min_run: config::run_split_min(),
3409 }
3410 }
3411
3412 /// The class a composite quad is drawn with.
3413 fn composite_class(self) -> PaintClass {
3414 if self.uber {
3415 PaintClass::Uber
3416 } else {
3417 PaintClass::LayerComposite
3418 }
3419 }
3420
3421 /// The class a hole punch is drawn with.
3422 fn punch_class(self) -> PaintClass {
3423 if self.uber {
3424 PaintClass::Uber
3425 } else {
3426 PaintClass::SolidCoverage
3427 }
3428 }
3429}
3430
3431/// One strip state's pipelines, by paint class — the ones a pass resolved for
3432/// the classes its runs name.
3433#[derive(Debug, Default)]
3434struct ClassPipelines {
3435 by_class: [Option<wgpu::RenderPipeline>; PaintClass::ALL.len()],
3436}
3437
3438impl ClassPipelines {
3439 /// The pipeline drawing `class`, if the frame resolved one.
3440 fn get(&self, class: PaintClass) -> Option<&wgpu::RenderPipeline> {
3441 self.by_class.get(class.index()).and_then(Option::as_ref)
3442 }
3443
3444 fn set(&mut self, class: PaintClass, pipeline: wgpu::RenderPipeline) {
3445 if let Some(slot) = self.by_class.get_mut(class.index()) {
3446 *slot = Some(pipeline);
3447 }
3448 }
3449
3450 /// Any one of the pipelines — what a bind group is built against, every
3451 /// strip pipeline sharing one explicit layout.
3452 fn any(&self) -> Option<&wgpu::RenderPipeline> {
3453 self.by_class.iter().flatten().next()
3454 }
3455}
3456
3457/// The strip pipelines a renderer has resolved so far, by `(state, class)`,
3458/// for the target format they were built against.
3459///
3460/// A memo in front of the pipeline cache: a steady-state frame takes every
3461/// pipeline it draws with from here, by indexing a dense `state x class` table,
3462/// rather than rebuilding each description (a vertex-layout allocation) and
3463/// packing its cache key — which keeps the frame's `pipelines` phase from
3464/// growing with the number of classes it draws.
3465#[derive(Debug)]
3466struct ResolvedPipelines {
3467 /// The format every entry was built for; a change drops them all.
3468 format: Option<wgpu::TextureFormat>,
3469 /// Indexed by [`Self::slot`].
3470 entries: [Option<wgpu::RenderPipeline>; RESOLVED_SLOTS],
3471 /// The filter pass's pipeline; its description is pinned to one format, so
3472 /// it is built once per renderer rather than once per filtering frame.
3473 filter: Option<wgpu::RenderPipeline>,
3474 /// The fragment region copy's pipeline, memoised on the same terms.
3475 copy: Option<wgpu::RenderPipeline>,
3476}
3477
3478/// One [`ResolvedPipelines`] entry per render state and paint class.
3479const RESOLVED_SLOTS: usize = EnginePipeline::ALL.len() * PaintClass::ALL.len();
3480
3481impl Default for ResolvedPipelines {
3482 fn default() -> Self {
3483 Self {
3484 format: None,
3485 entries: [const { None }; RESOLVED_SLOTS],
3486 filter: None,
3487 copy: None,
3488 }
3489 }
3490}
3491
3492impl ResolvedPipelines {
3493 /// The table slot of `state` drawing `class`.
3494 const fn slot(state: EnginePipeline, class: PaintClass) -> usize {
3495 state.index() * PaintClass::ALL.len() + class.index()
3496 }
3497}
3498
3499/// One pass's full recording state, assembled before the pass is begun.
3500///
3501/// A struct rather than an argument list because the composite groups have to
3502/// be built (and so borrowed) before `begin_render_pass` takes the encoder, and
3503/// naming them together is what makes that ordering obvious at the call site.
3504///
3505/// The attachment half (`pass`, `view`, `load`, `depth`) describes the pass the
3506/// plan begins when it is recorded on its own; the rest is what it draws, which
3507/// is all the [surface opening](SurfaceOpening) reads of the plans it draws
3508/// inside its own pass.
3509struct PassPlan<'a> {
3510 /// Which of the frame's passes this is — its label, and its entry in
3511 /// [`EngineRenderer::last_frame_passes`].
3512 pass: FramePass,
3513 view: &'a wgpu::TextureView,
3514 load: wgpu::LoadOp<wgpu::Color>,
3515 depth: Option<&'a wgpu::TextureView>,
3516 /// The render state the pass draws with, which folds each run's class to
3517 /// one it serves.
3518 state: EnginePipeline,
3519 /// The state's pipeline per paint class — at least every class the pass's
3520 /// segments name, folded.
3521 pipelines: &'a ClassPipelines,
3522 /// The pipeline variant's own groups 1-3.
3523 groups: &'a StripBindGroups,
3524 /// Group 0 for the pass's ordinary strip segments.
3525 resources: &'a wgpu::BindGroup,
3526 /// Group 0 per composite segment, in the order the segments name them.
3527 composites: &'a [wgpu::BindGroup],
3528 /// The externally bound texture each [`Segment::External`] slot names,
3529 /// indexed by slot — the frame's own
3530 /// [`ExternalRuns::keys`](crate::gpu::bindings::ExternalRuns::keys).
3531 /// Empty for a pass that draws no external texture, which is every pass of
3532 /// every frame that records no `SceneTexture`.
3533 external_keys: &'a [u64],
3534 instances: &'a wgpu::Buffer,
3535 /// The instance index every segment's own index is relative to.
3536 base: u32,
3537 segments: &'a [Segment],
3538 /// The class a [`Segment::Composite`] is drawn with —
3539 /// [`PaintClass::LayerComposite`], or Uber under the kill switch.
3540 composite_class: PaintClass,
3541}
3542
3543/// Records one pass: load the colour target, then draw each segment in order.
3544///
3545/// `timestamps` is the query pair the pass's GPU time is stamped into, `None`
3546/// for an untimed pass — which is every pass of every frame encoded without a
3547/// timestamp sink (see [`crate::diag`]). The pass is logged into `log` as it
3548/// is begun.
3549fn record_pass(
3550 encoder: &mut wgpu::CommandEncoder,
3551 plan: &PassPlan<'_>,
3552 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3553 log: &mut Vec<FramePass>,
3554) {
3555 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3556 label: Some(plan.pass.label()),
3557 color_attachments: &[Some(color_attachment(plan.view, plan.load))],
3558 depth_stencil_attachment: plan
3559 .depth
3560 .map(|view| depth_attachment(view, wgpu::LoadOp::Load)),
3561 timestamp_writes: timestamps,
3562 occlusion_query_set: None,
3563 multiview_mask: None,
3564 });
3565 log.push(plan.pass);
3566 draw_segments(&mut pass, plan);
3567}
3568
3569/// Draws `plan`'s segments, in order, into an already begun `pass`.
3570///
3571/// Two groups are re-bound per segment. Group 0, because a composite reads its
3572/// page through it while an ordinary strip reads the placeholder; and group 1,
3573/// because a run of instances sampling an externally bound texture needs that
3574/// texture bound where the atlas placeholder otherwise sits. Groups 2-3 are the
3575/// variant's own and never change within a plan. A plan with one segment —
3576/// every frame that records no layer and no external texture — sets them all
3577/// exactly once.
3578///
3579/// A segment naming a slot with no bind group behind it draws nothing rather
3580/// than drawing with whatever group 1 last held: the texture was unbound
3581/// between the frame's paint resolution and its recording, and a wrongly
3582/// sampled run is worse than a missing one.
3583///
3584/// Each segment is one paint-class run, and the pipeline is set only where
3585/// the class changes from the segment before — so a plan of one class (every
3586/// plan under the Uber kill switch) sets its pipeline exactly once. A segment
3587/// whose class has no resolved pipeline draws nothing, for the same reason; it
3588/// cannot occur, since the frame resolves every class its segments name before
3589/// recording.
3590///
3591/// Everything a plan draws with is set here, its first pipeline included, so
3592/// two plans drawn back to back into one pass — the opening's opaque strips
3593/// and then its round's — each start from their own state rather than from
3594/// whatever the plan before them left bound.
3595fn draw_segments(pass: &mut wgpu::RenderPass<'_>, plan: &PassPlan<'_>) {
3596 pass.set_vertex_buffer(0, plan.instances.slice(..));
3597
3598 let mut composite = 0_usize;
3599 let mut bound: Option<PaintClass> = None;
3600 for segment in plan.segments {
3601 // Claimed before the class lookup, so a composite whose pipeline is
3602 // missing still consumes its own group and the next one pairs up.
3603 let class = plan.state.fold_class(match *segment {
3604 Segment::Strips(_, _, class) | Segment::External(_, _, _, class) => class,
3605 Segment::Composite(..) | Segment::Backdrop(..) => plan.composite_class,
3606 });
3607 let (group, images, range) = match *segment {
3608 Segment::Strips(first, count, _) => {
3609 if count == 0 {
3610 continue;
3611 }
3612 (
3613 plan.resources,
3614 None,
3615 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3616 )
3617 }
3618 Segment::External(first, count, slot, _) => {
3619 if count == 0 {
3620 continue;
3621 }
3622 let Some(images) = plan
3623 .external_keys
3624 .get(slot as usize)
3625 .and_then(|key| plan.groups.externals.get(key))
3626 else {
3627 continue;
3628 };
3629 (
3630 plan.resources,
3631 Some(images),
3632 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3633 )
3634 }
3635 Segment::Composite(first, _) => {
3636 let Some(group) = plan.composites.get(composite) else {
3637 continue;
3638 };
3639 composite = composite.saturating_add(1);
3640 (
3641 group,
3642 None,
3643 GpuStrip::instance_range(plan.base.saturating_add(first), 1),
3644 )
3645 }
3646 // Its group is claimed before its count is read, like a
3647 // composite's, so the groups after it stay paired.
3648 Segment::Backdrop(first, count, _) => {
3649 let Some(group) = plan.composites.get(composite) else {
3650 continue;
3651 };
3652 composite = composite.saturating_add(1);
3653 if count == 0 {
3654 continue;
3655 }
3656 (
3657 group,
3658 None,
3659 GpuStrip::instance_range(plan.base.saturating_add(first), count),
3660 )
3661 }
3662 };
3663 if bound != Some(class) {
3664 let Some(pipeline) = plan.pipelines.get(class) else {
3665 continue;
3666 };
3667 pass.set_pipeline(pipeline);
3668 bound = Some(class);
3669 }
3670 plan.groups.bind_with(pass, group, images);
3671 pass.draw(GpuStrip::vertex_range(), range);
3672 }
3673}
3674
3675/// The pass that opens the frame's own surface: the clear, the opaque strips,
3676/// and the first surface round that draws, in one render pass.
3677///
3678/// Merging is pixel-identical to recording the three apart, because nothing
3679/// between them differs but the pass boundary: the clear becomes the pass's
3680/// `LoadOp::Clear` on colour (and on depth, unless the caller stated its
3681/// attachment is already populated), and the two strip plans draw in the order
3682/// their passes ran, each through its own pipelines and bind groups, against
3683/// the one colour and depth attachment both already shared. What it saves is
3684/// the pass boundary itself: on a tile-based GPU every pass over the surface
3685/// is a whole-framebuffer load into tile memory and a store back out, and the
3686/// clear and the opaque pass were two of those that drew little or nothing.
3687///
3688/// Recorded exactly once per frame, by whichever comes first of the first
3689/// surface round that draws (which it then carries) and the first punch
3690/// (ahead of which it is recorded alone) — or at the end of the walk when
3691/// neither came. See [`EngineRenderer::set_pass_merge`] for the switch that
3692/// records the three apart again.
3693struct SurfaceOpening<'a> {
3694 view: &'a wgpu::TextureView,
3695 depth: Option<&'a wgpu::TextureView>,
3696 /// The frame's base colour, already in the target's alpha convention.
3697 clear: wgpu::Color,
3698 /// The depth clear, or a load when the caller's attachment is pre-cleared
3699 /// (see [`crate::gpu::depth`]).
3700 depth_load: wgpu::LoadOp<f32>,
3701 /// The opaque strips, when the frame has any to draw.
3702 opaque: Option<PassPlan<'a>>,
3703}
3704
3705impl SurfaceOpening<'_> {
3706 /// Records the opening, drawing `round`'s segments after the opaque strips
3707 /// when a surface round is folded into it.
3708 fn record(
3709 &self,
3710 encoder: &mut wgpu::CommandEncoder,
3711 round: Option<&PassPlan<'_>>,
3712 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
3713 log: &mut Vec<FramePass>,
3714 ) {
3715 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
3716 label: Some(FramePass::Opening.label()),
3717 color_attachments: &[Some(color_attachment(
3718 self.view,
3719 wgpu::LoadOp::Clear(self.clear),
3720 ))],
3721 depth_stencil_attachment: self
3722 .depth
3723 .map(|view| depth_attachment(view, self.depth_load)),
3724 timestamp_writes: timestamps,
3725 occlusion_query_set: None,
3726 multiview_mask: None,
3727 });
3728 log.push(FramePass::Opening);
3729 if let Some(opaque) = self.opaque.as_ref() {
3730 draw_segments(&mut pass, opaque);
3731 }
3732 if let Some(round) = round {
3733 draw_segments(&mut pass, round);
3734 }
3735 }
3736}
3737
3738/// Records the surface opening as a pass of its own if it is still owed, and
3739/// marks it taken.
3740fn record_owed_opening(
3741 opening: &mut Option<SurfaceOpening<'_>>,
3742 encoder: &mut wgpu::CommandEncoder,
3743 timestamps: FrameTimestamps<'_>,
3744 log: &mut Vec<FramePass>,
3745) {
3746 if let Some(opening) = opening.take() {
3747 opening.record(encoder, None, timestamps.writes(EngineSpan::Main), log);
3748 }
3749}
3750
3751/// Everything the frame's hole-punch passes are recorded with, resolved once
3752/// before the round walk begins.
3753///
3754/// A frame can now record several: the punches are issued at the painter-order
3755/// positions they were hoisted from, so a surface round carrying two of them is
3756/// cut twice and each cut erases through this same pipeline and these same
3757/// groups (see [`crate::compile::clear`]). Resolving them once is what keeps
3758/// that from becoming a per-cut lookup, and holding the borrows in one value is
3759/// what keeps them out of the page pool's way — the walk holds that mutably
3760/// throughout.
3761struct PunchPass<'a> {
3762 view: &'a wgpu::TextureView,
3763 depth: Option<&'a wgpu::TextureView>,
3764 /// Which destination-out state the punch draws with.
3765 state: EnginePipeline,
3766 /// That state's pipelines, holding at least [`Self::class`].
3767 pipelines: &'a ClassPipelines,
3768 /// The class every punch run is drawn with: the solid program, or Uber
3769 /// under the kill switch.
3770 class: PaintClass,
3771 /// The destination-out variant's own groups 1-3.
3772 groups: &'a StripBindGroups,
3773 instances: &'a wgpu::Buffer,
3774 /// The instance index the punch's own `(first, count)` is relative to — the
3775 /// alpha region's start, the same one every round's segments use.
3776 base: u32,
3777}
3778
3779impl PunchPass<'_> {
3780 /// Records `punch`'s instances as one destination-out pass, or nothing at
3781 /// all when the cut issued none — which is every cut of every frame that
3782 /// records no `ClearRect`. A pass that is not recorded takes no query pair.
3783 fn record(
3784 &self,
3785 encoder: &mut wgpu::CommandEncoder,
3786 punch: (u32, u32),
3787 timestamps: FrameTimestamps<'_>,
3788 log: &mut Vec<FramePass>,
3789 ) {
3790 let (first, count) = punch;
3791 if count == 0 {
3792 return;
3793 }
3794 record_pass(
3795 encoder,
3796 &PassPlan {
3797 pass: FramePass::Punch,
3798 view: self.view,
3799 load: wgpu::LoadOp::Load,
3800 depth: self.depth,
3801 state: self.state,
3802 pipelines: self.pipelines,
3803 groups: self.groups,
3804 resources: &self.groups.resources,
3805 composites: &[],
3806 // A punch erases with a solid source; it samples nothing.
3807 external_keys: &[],
3808 instances: self.instances,
3809 base: self.base,
3810 // Every punch instance is the same solid source, so the run
3811 // is one class whole — no split to plan.
3812 segments: &[Segment::Strips(first, count, self.class)],
3813 composite_class: self.class,
3814 },
3815 timestamps.writes(EngineSpan::Main),
3816 log,
3817 );
3818 }
3819}
3820
3821/// Hands back the page groups a finished round consumed, and parks the page it
3822/// wrote in its own group.
3823///
3824/// The same bookkeeping after every round, strip and filter alike: a page is
3825/// free the moment the pass that sampled it ends, which is what bounds a chain
3826/// of any depth — and a filter layer's own pair of pages — to the two ping-pong
3827/// groups, and every shape this scheduler serves to
3828/// [`MAX_LIVE_PAGES`](crate::schedule::MAX_LIVE_PAGES) live intermediates.
3829///
3830/// A round that is not continuing a page of its own takes a *fresh* texture out
3831/// of the pool rather than the group's current occupant, and the occupant it
3832/// displaces goes back here. That is what keeps a filter pass from ever holding
3833/// one texture as both its attachment and its source: a filter round never
3834/// continues a page — it clears — so what it writes is always a different
3835/// texture from the one the pass before it wrote and this pass reads.
3836fn settle_pages(
3837 targets: &mut IntermediateTargets,
3838 live: &mut [Option<PooledTexture>; MAX_LIVE_PAGES],
3839 released: [bool; MAX_LIVE_PAGES],
3840 own: Option<(PageParity, PooledTexture)>,
3841) {
3842 for (index, slot) in live.iter_mut().enumerate() {
3843 if released.get(index).copied().unwrap_or(false)
3844 && let Some(page) = slot.take()
3845 {
3846 targets.release(page);
3847 }
3848 }
3849 if let Some((parity, pooled)) = own
3850 && let Some(previous) = live[parity.index()].replace(pooled)
3851 {
3852 targets.release(previous);
3853 }
3854}
3855
3856/// The GPU resources a frame's [filter](crate::filters) rounds are executed
3857/// with, beyond the two pooled pages they ping-pong between.
3858///
3859/// Three of them, and each is the engine's only one of its kind: the
3860/// filter-data texture holding every filter in the frame's 48-byte parameter
3861/// block, the bilinear sampler
3862/// ([`filter_sampler`](crate::gpu::targets::filter_sampler)) the blur kernels
3863/// read their source page through, and the instance buffer one quad per pass is
3864/// drawn from.
3865///
3866/// Public because this *is* executing a filter pass — the renderer holds one
3867/// and drives it over the pages the scheduler named, and `tests/filters.rs`
3868/// drives the same type over pages of its own on real hardware. That second
3869/// caller is not a convenience: `frust_scene` carries no *layer*-filter
3870/// command, so a filter layer cannot reach [`EngineRenderer::encode`] through a
3871/// `Scene` at all, and driving this type directly is how the drop shadow's
3872/// WGSL is exercised on a device. The blur's is also reached through `encode`
3873/// itself, by a backdrop blur's pass rounds.
3874#[derive(Debug)]
3875pub struct FilterResources {
3876 /// Every filter in the frame's parameter block, back to back.
3877 data: ResourceTexture,
3878 /// Group 0, naming [`Self::data`]'s view. Rebuilt whenever that texture is,
3879 /// and valid for the renderer's whole life otherwise: a filter pipeline's
3880 /// description does not depend on the frame's target format, so there is
3881 /// only ever one layout to have derived it from.
3882 data_group: Option<wgpu::BindGroup>,
3883 sampler: wgpu::Sampler,
3884 instances: Option<wgpu::Buffer>,
3885 instance_capacity: u64,
3886 /// Reusable staging for the parameter-block upload, padded to the
3887 /// texture's own footprint.
3888 staging: Vec<u8>,
3889 /// Bind groups [`Self::prepare`] has built, over this value's life.
3890 groups_created: u64,
3891}
3892
3893impl FilterResources {
3894 /// A renderer's filter resources, with nothing uploaded yet.
3895 #[must_use]
3896 pub fn new(device: &wgpu::Device) -> Self {
3897 Self {
3898 data: ResourceTexture::new(
3899 device,
3900 &filter_data_texture_descriptor(gpu::MIN_RESOURCE_TEXTURE_HEIGHT),
3901 ),
3902 data_group: None,
3903 sampler: filter_sampler(device),
3904 instances: None,
3905 instance_capacity: 0,
3906 staging: Vec::new(),
3907 groups_created: 0,
3908 }
3909 }
3910
3911 /// Uploads this frame's parameter `blocks` and reserves room for `passes`
3912 /// pass instances, growing either resource if the frame outgrew it.
3913 ///
3914 /// `pipeline` is the one [`EnginePipeline::Filter`] describes; it is needed
3915 /// because wgpu derives a pipeline's bind-group layouts from its shader
3916 /// module, so the group naming the filter-data texture can only be built
3917 /// against the pipeline that will bind it.
3918 ///
3919 /// A block count no filter-data texture could hold (see
3920 /// [`filter_data_texture_height`]) leaves the group unbuilt, which leaves
3921 /// every filter pass of the frame issuing no draw — the page is still
3922 /// cleared, so the layer composites as transparent rather than as whatever
3923 /// its page last held. Unreachable in practice, and "draw less, never
3924 /// wrong" when it is not.
3925 pub fn prepare(
3926 &mut self,
3927 device: &wgpu::Device,
3928 queue: &wgpu::Queue,
3929 pipeline: &wgpu::RenderPipeline,
3930 blocks: &[GpuFilterData],
3931 passes: u32,
3932 ) {
3933 let Some(height) = filter_data_texture_height(blocks.len()) else {
3934 self.data_group = None;
3935 return;
3936 };
3937 if height > self.data.height {
3938 self.data = ResourceTexture::new(device, &filter_data_texture_descriptor(height));
3939 self.data_group = None;
3940 }
3941 if self.data_group.is_none() {
3942 self.groups_created = self.groups_created.saturating_add(1);
3943 self.data_group = Some(device.create_bind_group(&wgpu::BindGroupDescriptor {
3944 label: Some("frust-engine filter data"),
3945 layout: &pipeline.get_bind_group_layout(0),
3946 entries: &[wgpu::BindGroupEntry {
3947 binding: 0,
3948 resource: wgpu::BindingResource::TextureView(&self.data.view),
3949 }],
3950 }));
3951 }
3952
3953 // A queue write covers the whole texture extent, so the blocks are
3954 // padded out to its footprint; the trailing texels are never addressed,
3955 // because a pass names its own block by a texel offset the host handed
3956 // it.
3957 let footprint = gpu::resource_texture_bytes(self.data.width, self.data.height);
3958 self.staging.clear();
3959 self.staging
3960 .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
3961 let bytes: &[u8] = bytemuck::cast_slice(blocks);
3962 if let Some(head) = self.staging.get_mut(..bytes.len()) {
3963 head.copy_from_slice(bytes);
3964 }
3965 queue.write_texture(
3966 self.data.copy_target(),
3967 &self.staging,
3968 resource_layout(
3969 gpu::resource_bytes_per_row(self.data.width),
3970 self.data.height,
3971 ),
3972 self.data.extent(),
3973 );
3974
3975 self.reserve_instances(device, passes);
3976 }
3977
3978 /// Grows the instance buffer if this frame's pass count outgrew it.
3979 fn reserve_instances(&mut self, device: &wgpu::Device, passes: u32) {
3980 let stride = size_of::<FilterInstanceData>() as u64;
3981 let required = u64::from(passes).saturating_mul(stride).max(stride);
3982 if self.instances.is_some() && self.instance_capacity >= required {
3983 return;
3984 }
3985 let capacity = required
3986 .checked_next_power_of_two()
3987 .unwrap_or(required)
3988 .max(MIN_FILTER_INSTANCE_CAPACITY.saturating_mul(stride));
3989 self.instance_capacity = capacity;
3990 self.instances = Some(device.create_buffer(&wgpu::BufferDescriptor {
3991 label: Some("frust-engine filter instances"),
3992 size: capacity,
3993 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
3994 mapped_at_creation: false,
3995 }));
3996 }
3997
3998 /// Writes one pass's instance into slot `index` of the instance buffer.
3999 ///
4000 /// Separate from [`Self::prepare`] because a pass's `dest_texture_size` is
4001 /// the extent the texture pool *quantized* its destination page up to, and
4002 /// NDC is computed against the attachment's real extent — only the round
4003 /// that acquires the page knows it. A queue write issued while the frame is
4004 /// being recorded still lands ahead of the command buffers it is submitted
4005 /// with, which is the same ordering a page's viewport uniform already
4006 /// relies on.
4007 ///
4008 /// A slot past the reserved capacity is dropped rather than written, which
4009 /// leaves that pass drawing whatever the slot last held; unreachable, since
4010 /// `prepare` reserved one slot per pass of this very frame.
4011 pub fn write_instance(&self, queue: &wgpu::Queue, index: u32, instance: &FilterInstanceData) {
4012 let Some(buffer) = self.instances.as_ref() else {
4013 return;
4014 };
4015 let stride = size_of::<FilterInstanceData>() as u64;
4016 let offset = u64::from(index).saturating_mul(stride);
4017 if offset.saturating_add(stride) > self.instance_capacity {
4018 return;
4019 }
4020 queue.write_buffer(buffer, offset, bytemuck::bytes_of(instance));
4021 }
4022
4023 /// Records one filter pass into `encoder`: clear the destination page, then
4024 /// draw the one instanced quad that filters `plan`'s source into it.
4025 ///
4026 /// The clear is unconditional and the draw is not. A filter pass writes only
4027 /// the region its step names — a decimated one a quarter of the texels the
4028 /// pass before it did — and the kernels sample past that region without
4029 /// bounds checks, so whatever surrounds it has to be transparent rather than
4030 /// a previous holder's pixels. That has to hold even on the path where the
4031 /// pass itself cannot be issued, or the layer's composite would sample the
4032 /// page's previous tenant instead of nothing.
4033 ///
4034 /// The pass is opened and closed here, on the caller's own encoder: a filter
4035 /// round is not an exception to [`EngineRenderer::encode`]'s contract.
4036 pub fn record_pass(
4037 &self,
4038 device: &wgpu::Device,
4039 encoder: &mut wgpu::CommandEncoder,
4040 plan: &FilterPassPlan<'_>,
4041 ) {
4042 self.record_pass_timed(device, encoder, plan, None);
4043 }
4044
4045 /// [`Self::record_pass`], stamping the pass's GPU time into `timestamps`.
4046 ///
4047 /// A separate method rather than a field on [`FilterPassPlan`]: the plan is
4048 /// public and built by struct literal outside this crate, so a new required
4049 /// field would break every one of those call sites to serve a diagnostic
4050 /// they do not use. `None` records exactly what [`Self::record_pass`] does.
4051 pub fn record_pass_timed(
4052 &self,
4053 device: &wgpu::Device,
4054 encoder: &mut wgpu::CommandEncoder,
4055 plan: &FilterPassPlan<'_>,
4056 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
4057 ) {
4058 self.record_pass_with(device, encoder, plan, timestamps, None);
4059 }
4060
4061 /// The group 1 a pass samples `source` through: the page and this
4062 /// resource's sampler. Built against `pipeline`'s layout, which every
4063 /// filter pass shares.
4064 fn source_group(
4065 &self,
4066 device: &wgpu::Device,
4067 pipeline: &wgpu::RenderPipeline,
4068 source: &wgpu::TextureView,
4069 ) -> wgpu::BindGroup {
4070 device.create_bind_group(&wgpu::BindGroupDescriptor {
4071 label: Some("frust-engine filter source"),
4072 layout: &pipeline.get_bind_group_layout(1),
4073 entries: &[
4074 wgpu::BindGroupEntry {
4075 binding: 0,
4076 resource: wgpu::BindingResource::TextureView(source),
4077 },
4078 wgpu::BindGroupEntry {
4079 binding: 1,
4080 resource: wgpu::BindingResource::Sampler(&self.sampler),
4081 },
4082 ],
4083 })
4084 }
4085
4086 /// [`Self::record_pass_timed`] over a source group the caller already
4087 /// holds (`Some`, built by [`Self::source_group`] for `plan.source`), or one
4088 /// built here (`None`).
4089 fn record_pass_with(
4090 &self,
4091 device: &wgpu::Device,
4092 encoder: &mut wgpu::CommandEncoder,
4093 plan: &FilterPassPlan<'_>,
4094 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
4095 source: Option<&wgpu::BindGroup>,
4096 ) {
4097 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
4098 label: Some(plan.label),
4099 color_attachments: &[Some(color_attachment(
4100 plan.dest,
4101 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
4102 ))],
4103 // A pooled page carries no depth attachment, which is also why the
4104 // filter pipeline declares no depth state.
4105 depth_stencil_attachment: None,
4106 timestamp_writes: timestamps,
4107 occlusion_query_set: None,
4108 multiview_mask: None,
4109 });
4110
4111 let (Some(data_group), Some(instances)) =
4112 (self.data_group.as_ref(), self.instances.as_ref())
4113 else {
4114 return;
4115 };
4116
4117 let built;
4118 let source = match source {
4119 Some(source) => source,
4120 None => {
4121 built = self.source_group(device, plan.pipeline, plan.source);
4122 &built
4123 }
4124 };
4125
4126 pass.set_pipeline(plan.pipeline);
4127 pass.set_vertex_buffer(0, instances.slice(..));
4128 pass.set_bind_group(0, data_group, &[]);
4129 pass.set_bind_group(1, source, &[]);
4130 pass.draw(
4131 0..FILTER_QUAD_VERTICES,
4132 plan.instance..plan.instance.saturating_add(1),
4133 );
4134 }
4135}
4136
4137/// One filter pass's full recording state.
4138///
4139/// The two pages are named as views rather than as parities because
4140/// [`FilterResources`] holds no pool: which texture each group is holding is the
4141/// caller's bookkeeping, whether that caller is the frame path or a test.
4142#[derive(Debug)]
4143pub struct FilterPassPlan<'a> {
4144 /// Label for captures and validation messages.
4145 pub label: &'a str,
4146 /// The pipeline [`EnginePipeline::Filter`] describes.
4147 pub pipeline: &'a wgpu::RenderPipeline,
4148 /// The page this pass writes. Cleared to transparent before it is written.
4149 pub dest: &'a wgpu::TextureView,
4150 /// The page this pass reads — the one the pass before it wrote.
4151 pub source: &'a wgpu::TextureView,
4152 /// Which instance of the filter instance buffer this pass draws.
4153 pub instance: u32,
4154}
4155
4156/// A colour attachment over `view` with `load`, keeping what it stores.
4157fn color_attachment(
4158 view: &wgpu::TextureView,
4159 load: wgpu::LoadOp<wgpu::Color>,
4160) -> wgpu::RenderPassColorAttachment<'_> {
4161 wgpu::RenderPassColorAttachment {
4162 view,
4163 depth_slice: None,
4164 resolve_target: None,
4165 ops: wgpu::Operations {
4166 load,
4167 store: wgpu::StoreOp::Store,
4168 },
4169 }
4170}
4171
4172/// A depth attachment over `view` with `load`, keeping what it stores so the
4173/// pass after it tests against the same buffer.
4174fn depth_attachment(
4175 view: &wgpu::TextureView,
4176 load: wgpu::LoadOp<f32>,
4177) -> wgpu::RenderPassDepthStencilAttachment<'_> {
4178 wgpu::RenderPassDepthStencilAttachment {
4179 view,
4180 depth_ops: Some(wgpu::Operations {
4181 load,
4182 store: wgpu::StoreOp::Store,
4183 }),
4184 stencil_ops: None,
4185 }
4186}
4187
4188/// The frame's base colour as a clear value.
4189///
4190/// The strip pipelines blend premultiplied and the shader emits premultiplied
4191/// colour, so a [`OutputAlpha::Premultiplied`] target's clear value has to be
4192/// premultiplied too — otherwise the background sits in a different alpha
4193/// convention from everything drawn over it. `Straight` is honoured here and
4194/// only here: the pipelines themselves are fixed premultiplied, so a straight
4195/// target's *drawn* content is premultiplied regardless.
4196fn clear_color(base_color: Color, output: OutputAlpha) -> wgpu::Color {
4197 let components = match output {
4198 OutputAlpha::Premultiplied => base_color.premultiply().components,
4199 OutputAlpha::Straight => base_color.components,
4200 };
4201 wgpu::Color {
4202 r: f64::from(components[0]),
4203 g: f64::from(components[1]),
4204 b: f64::from(components[2]),
4205 a: f64::from(components[3]),
4206 }
4207}
4208
4209/// Whether a frame cleared to `base_color` presents opaquely, and so whether
4210/// its alpha channel carries anything a hole punch could reveal.
4211///
4212/// The engine's only statement about the target's own alpha handling:
4213/// [`EngineTarget`] describes how the alpha it produces is *interpreted*
4214/// (premultiplied or straight), never whether it is used at all. A base colour
4215/// at full alpha seals every pixel of the surface, which is exactly the
4216/// presentation [`crate::compile::clear`]'s contract says to skip the
4217/// destination-out pass on.
4218fn is_opaque(base_color: Color) -> bool {
4219 base_color.components[3] >= 1.0
4220}
4221
4222/// The target extent on the `u16` device grid the strip pipeline addresses.
4223fn grid_size(width: u32, height: u32) -> Result<(u16, u16), EngineError> {
4224 let width = u16::try_from(width).map_err(|_| EngineError::TargetTooLarge)?;
4225 let height = u16::try_from(height).map_err(|_| EngineError::TargetTooLarge)?;
4226 Ok((width, height))
4227}
4228
4229/// One unit of drawing inside a round's pass, in execution order.
4230///
4231/// Instance indices are relative to the alpha region of the shared instance
4232/// buffer, which is why [`PassPlan::base`] exists rather than the indices being
4233/// absolute: the opaque region is laid out first and its own segment addresses
4234/// from zero.
4235///
4236/// Every strip segment is also one paint-class run: it names the
4237/// [`PaintClass`] whose program draws it, and a pass sets that class's
4238/// pipeline where the class changes (see the module doc's *Paint-class runs*).
4239#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4240enum Segment {
4241 /// Ordinary strip instances, as `(first, count, class)`, drawn with the
4242 /// frame's own atlas binding.
4243 Strips(u32, u32, PaintClass),
4244 /// Strip instances sampling an externally bound texture, as
4245 /// `(first, count, slot, class)` — a *run*, in the sense
4246 /// [`crate::gpu::bindings::ExternalRuns`] gives the word: the maximal span
4247 /// of consecutive instances that share one texture, drawn with that
4248 /// texture's own group 1 bound.
4249 External(u32, u32, u32, PaintClass),
4250 /// One composite quad at `first`, sampling the finished page it names.
4251 Composite(u32, CompositeSource),
4252 /// A backdrop blur's own strip instances, as `(first, count, parity)`:
4253 /// its rectangle's coverage, each instance sampling the blurred page in
4254 /// `parity`'s group at its own position — drawn through the composite
4255 /// class with that page bound as group 0's layer input.
4256 Backdrop(u32, u32, PageParity),
4257}
4258
4259/// The page a [`Segment::Composite`] samples.
4260#[derive(Debug, Clone, Copy, PartialEq, Eq)]
4261enum CompositeSource {
4262 /// The live pooled page in this group.
4263 Page(PageParity),
4264 /// The retained page of this retained bracket (see
4265 /// [`crate::schedule::RetainedSource`]).
4266 Retained(u32),
4267}
4268
4269/// The retained page one round renders a body into.
4270#[derive(Debug, Clone, Copy)]
4271struct RetainedRound {
4272 /// The retained bracket the page belongs to.
4273 bracket: u32,
4274 /// Whether an earlier round of the same body already rendered into the
4275 /// page this frame, so this one loads it rather than clearing it.
4276 continued: bool,
4277}
4278
4279/// One scheduled round, resolved to the instances and target it draws with.
4280///
4281/// One *plan* rather than one scheduled round: a surface round carrying a hole
4282/// punch is cut into a plan per span between its punches, so the punch pass can
4283/// be recorded at the painter-order position it was hoisted from (see
4284/// [`Scratch::cut_for_punches`]).
4285#[derive(Debug, Clone)]
4286struct RoundPlan {
4287 /// The page this round renders into, or `None` for the frame's own target
4288 /// and for a retained page.
4289 page: Option<PagePlan>,
4290 /// The retained page this round renders a body into, if it does — an
4291 /// off-screen round like a page one, into the store's texture.
4292 retained: Option<RetainedRound>,
4293 /// This round's slice of [`Scratch::segments`], in execution order.
4294 segments: Range<usize>,
4295 /// Page groups this round consumed, indexed by
4296 /// [`PageParity::index`]; each returns to the pool once the pass ends.
4297 released: [bool; MAX_LIVE_PAGES],
4298 /// The filter pass this round runs, on a filter round — which draws no
4299 /// strip and composites nothing, so its `segments` range is empty.
4300 filter: Option<FilterPlan>,
4301 /// The region of the frame's own target this round copies into its page,
4302 /// on a region-copy round — which, like a filter round, draws no strip.
4303 copy: Option<RectU16>,
4304 /// The hole punches to erase with once this plan's own pass has been
4305 /// recorded, as `(first, count)` into the alpha region — the cut this plan
4306 /// ends at.
4307 ///
4308 /// Only a *surface* plan ever carries one: a punch recorded inside an
4309 /// isolated layer was hoisted out of it to the frame root (see
4310 /// [`crate::compile::clear`]), so a page round is never cut and a filter
4311 /// round — which draws nothing of the frame's own — never is either.
4312 punch: (u32, u32),
4313}
4314
4315/// One filter pass, resolved to the instance slot it draws and the page group
4316/// it reads.
4317#[derive(Debug, Clone, Copy)]
4318struct FilterPlan {
4319 /// Which pass of the filter's sequence this is, and at what extents.
4320 step: FilterStep,
4321 /// The group holding the page this pass reads.
4322 source: PageParity,
4323 /// The texel this filter's parameter block starts at in the filter-data
4324 /// texture — where the fragment stage reads its kernel from.
4325 data_offset: u32,
4326 /// The filter layer's own extent, before any decimation; it bounds the
4327 /// transparent border a decimated pass overdraws.
4328 original: SizeU16,
4329 /// This pass's slot in the frame's filter instance buffer.
4330 instance: u32,
4331}
4332
4333/// The pooled page one round renders into.
4334#[derive(Debug, Clone, Copy)]
4335struct PagePlan {
4336 parity: PageParity,
4337 size: PageSize,
4338 /// Whether an earlier round of the same layer already rendered into this
4339 /// page, so this round takes that texture back rather than acquiring a
4340 /// fresh one, and loads it rather than clearing it.
4341 continued: bool,
4342}
4343
4344/// The frame's instance buffers and pass plan, retained so a steady-state frame
4345/// refills them rather than reallocating them.
4346///
4347/// The segments of every round live in one flat vector rather than a vector per
4348/// round, so a frame with layers costs no allocation once the first one has
4349/// grown these buffers.
4350#[derive(Debug, Default)]
4351struct Scratch {
4352 /// Fully-covered spans of the opaque draws targeting the frame's own
4353 /// surface, drawn unblended with depth write in one pass ahead of every
4354 /// round — including the surface's own, of which a frame may have several.
4355 opaque: Vec<GpuStrip>,
4356 /// Everything else, drawn premultiplied-blended in painter order and laid
4357 /// out round by round in execution order.
4358 alpha: Vec<GpuStrip>,
4359 /// Every round's segments, back to back; [`RoundPlan::segments`] slices it.
4360 segments: Vec<Segment>,
4361 /// The frame's rounds, each layer's before the round that composites it and
4362 /// the surface's last round last.
4363 rounds: Vec<RoundPlan>,
4364 /// The instances of the punches no cut reached — those recorded past every
4365 /// op of the frame — as `(first, count)` into the alpha region.
4366 ///
4367 /// Their pass is the last thing the frame records, which is where a
4368 /// `ClearRect` recorded last belongs in painter order anyway. Punches the
4369 /// walk *did* cut at are held on their own [`RoundPlan::punch`] instead.
4370 ///
4371 /// No round's segments name a punch instance, wherever it sits in the
4372 /// buffer, so dropping the punch passes leaves the frame exactly as the
4373 /// display list would read without the clear.
4374 punch: (u32, u32),
4375 /// The deepest painter's-order index inside each recorded layer, which is
4376 /// the depth its composite carries. Filled as the rounds are walked, which
4377 /// is sound because a layer's own round always precedes the round that
4378 /// composites it.
4379 layer_depth: Vec<u32>,
4380 /// One parameter block per *filter layer* of the frame, in the order the
4381 /// rounds first named them — which is the order the texel offsets in
4382 /// [`FilterPlan::data_offset`] were taken from.
4383 filter_blocks: Vec<GpuFilterData>,
4384 /// The layers `filter_blocks` holds, parallel to it, so a filter's second
4385 /// and later passes reuse the block its first one packed rather than
4386 /// repacking one per pass.
4387 filter_layers: Vec<u32>,
4388 /// How many filter passes this frame runs, and so how many instances its
4389 /// filter instance buffer has to hold.
4390 filter_passes: u32,
4391 /// How many region-copy rounds this frame runs — one per backdrop blur
4392 /// the scheduler served.
4393 copies: u32,
4394 /// How segments are cut into paint-class runs this frame — copied in from
4395 /// the renderer ahead of [`Self::build`].
4396 routing: ClassRouting,
4397 /// The opaque region's own segments: `(0, opaque.len())` cut into class
4398 /// runs, drawn by the one depth-writing pass.
4399 opaque_segments: Vec<Segment>,
4400 /// The classes the opaque pass's runs name.
4401 opaque_classes: ClassSet,
4402 /// The classes the frame's surface rounds name — composites included.
4403 surface_classes: ClassSet,
4404 /// The classes the frame's page rounds name — composites included.
4405 page_classes: ClassSet,
4406 /// Whether the round being planned renders into a page, which decides the
4407 /// class set [`Self::push_run`] records a run's class in.
4408 planning_page: bool,
4409 /// Reusable output of [`merge_class_runs`].
4410 class_runs: Vec<ClassRun>,
4411 /// Where the paint class changes in [`Self::alpha`], as `(first instance,
4412 /// class)` in ascending order — one entry per draw whose class differs
4413 /// from the draw before it. Every instance of a draw shares its paint, so
4414 /// these are the only places a class run can start, and a segment is split
4415 /// by walking them instead of re-reading every instance.
4416 alpha_marks: Vec<(u32, PaintClass)>,
4417 /// The same marks for [`Self::opaque`].
4418 opaque_marks: Vec<(u32, PaintClass)>,
4419}
4420
4421impl Scratch {
4422 /// Turns a scheduled frame into instances and a per-round pass plan,
4423 /// routing each instance to the pass that can draw it.
4424 ///
4425 /// A draw's anti-aliased spans always land in the blended buffer: partial
4426 /// coverage is not opaque however opaque the paint is. Its fully-covered
4427 /// spans land in the opaque buffer only when the paint is opaque, depth is
4428 /// available to re-establish their ordering against the blended ones, *and*
4429 /// the draw targets the frame's own surface — a page has no depth
4430 /// attachment, so a page round is a plain painter's-algorithm walk.
4431 ///
4432 /// `backdrops` is the frame's served backdrop blurs — `frame.backdrops`,
4433 /// or none at all on a target no region can be read back from. A draw the
4434 /// scheduler placed at a backdrop's own depth is that backdrop's rectangle
4435 /// and samples its blurred page instead of its recorded paint; and no draw
4436 /// at or past the first backdrop's depth takes the depth-writing pass,
4437 /// which is recorded ahead of every surface round — so ahead of the copy
4438 /// that reads the target back, which must not see what is painted over the
4439 /// backdrop.
4440 fn build(
4441 &mut self,
4442 frame: &CompiledFrame,
4443 rounds: &[Round],
4444 depth_active: bool,
4445 punches: bool,
4446 paint_slots: &[Option<ResolvedPaint>],
4447 backdrops: &[Backdrop],
4448 ) {
4449 self.opaque.clear();
4450 self.alpha.clear();
4451 self.segments.clear();
4452 self.rounds.clear();
4453 self.punch = (0, 0);
4454 self.layer_depth.clear();
4455 self.layer_depth.resize(frame.recorder.layers.len(), 0);
4456 // A `DrawRetained`'s layer holds no draw, so its composite's depth is
4457 // the slot the command itself was recorded at rather than the deepest
4458 // index inside it.
4459 for bracket in &frame.retained {
4460 if let RetainedKind::Draw { depth, .. } = bracket.kind
4461 && let Some(slot) = self.layer_depth.get_mut(bracket.layer as usize)
4462 {
4463 *slot = depth;
4464 }
4465 }
4466 self.filter_blocks.clear();
4467 self.filter_layers.clear();
4468 self.filter_passes = 0;
4469 self.copies = 0;
4470 self.opaque_segments.clear();
4471 self.alpha_marks.clear();
4472 self.opaque_marks.clear();
4473 self.opaque_classes = ClassSet::EMPTY;
4474 self.surface_classes = ClassSet::EMPTY;
4475 self.page_classes = ClassSet::EMPTY;
4476
4477 let draws = frame.draws();
4478 let strips = frame.strip_buf();
4479 // The punches still to be issued, in the order the compiler hoisted
4480 // them — which is depth order, since each took its own index from the
4481 // frame's monotonic painter-order counter. Empty on a frame with no
4482 // clear and on a target that disregards alpha (`punches`), and then
4483 // nothing below ever cuts: a non-punching frame is planned exactly as
4484 // it always was.
4485 let mut pending: &[ClearPunch] = if punches { &frame.clears } else { &[] };
4486 // Every draw before the first backdrop may take the depth-writing pass;
4487 // none from it on. `u32::MAX` on a frame with no backdrop, which is
4488 // then routed exactly as it always was.
4489 let opaque_limit = backdrops
4490 .first()
4491 .map_or(u32::MAX, |backdrop| backdrop.depth);
4492
4493 for round in rounds {
4494 let page = round.page();
4495 let retained = round.retained();
4496 let offscreen = page.is_some() || retained.is_some();
4497
4498 // A region-copy round draws nothing: it is planned as its page and
4499 // the region it copies. Any punch recorded ahead of the backdrop
4500 // it serves is erased first, as a cut of its own, so the copy reads
4501 // the target the display list says lies beneath the backdrop.
4502 if let Some(copy) = round.region_copy() {
4503 if let Some(page) = page {
4504 if !pending.is_empty()
4505 && let Some(backdrop) = backdrops
4506 .iter()
4507 .find(|backdrop| backdrop.layer == page.layer)
4508 {
4509 let mut first_segment = self.segments.len();
4510 let mut run_start = self.alpha.len() as u32;
4511 let mut run_external = None;
4512 self.cut_for_punches(
4513 &mut pending,
4514 strips,
4515 backdrop.depth,
4516 &mut first_segment,
4517 &mut run_start,
4518 &mut run_external,
4519 );
4520 }
4521 self.copies = self.copies.saturating_add(1);
4522 self.rounds.push(RoundPlan {
4523 page: Some(PagePlan {
4524 parity: page.parity,
4525 size: page.size,
4526 continued: false,
4527 }),
4528 retained: None,
4529 segments: self.segments.len()..self.segments.len(),
4530 released: released_groups(&round.released),
4531 filter: None,
4532 copy: Some(copy.source),
4533 punch: (0, 0),
4534 });
4535 }
4536 continue;
4537 }
4538
4539 let filter = self.plan_filter(frame, round, backdrops);
4540 // A page holds its layer — or one column band of it — at the page's
4541 // own origin, so every instance of the round is shifted into that
4542 // column and clipped to it (see `PageWindow`). A retained page holds
4543 // its body the same way.
4544 let window = match (page, retained) {
4545 (Some(page), _) => PageWindow::of(page),
4546 (None, Some(retained)) => PageWindow::of_retained(retained),
4547 (None, None) => PageWindow::ROOT,
4548 };
4549 let split_opaque = !offscreen && depth_active;
4550 self.planning_page = offscreen;
4551 // Only the frame's own target is punched — a punch inside an
4552 // isolated layer was hoisted out of it — so only a surface round is
4553 // ever cut.
4554 let cuts = !offscreen;
4555
4556 let mut first_segment = self.segments.len();
4557 let mut deepest = 0_u32;
4558 let mut run_start = self.alpha.len() as u32;
4559 // The external texture the open run is drawn with, `None` while it
4560 // is drawn with the frame's own atlas binding. A draw that names a
4561 // different one closes the run: there is a single external binding
4562 // to set (see [`crate::gpu::bindings`]).
4563 let mut run_external: Option<u32> = None;
4564
4565 for op in &round.ops {
4566 match op {
4567 RoundOp::Draws(range) => {
4568 let batch = draws
4569 .get(range.start as usize..range.end as usize)
4570 .unwrap_or(&[]);
4571 for draw in batch {
4572 deepest = deepest.max(draw.depth);
4573 // Ahead of the paint lookup rather than after it, so
4574 // the cut follows the order the display list was
4575 // recorded in rather than the subset of it that
4576 // survived lowering.
4577 if cuts && !pending.is_empty() {
4578 self.cut_for_punches(
4579 &mut pending,
4580 strips,
4581 draw.depth,
4582 &mut first_segment,
4583 &mut run_start,
4584 &mut run_external,
4585 );
4586 }
4587 // A backdrop's rectangle: drawn from the blurred
4588 // page this round samples rather than from the
4589 // transparent paint it was recorded with.
4590 if let Ok(at) = backdrops
4591 .binary_search_by_key(&draw.depth, |backdrop| backdrop.depth)
4592 {
4593 if let Some(backdrop) = backdrops.get(at)
4594 && let Some(sample) = round
4595 .samples
4596 .iter()
4597 .find(|sample| sample.layer == backdrop.layer)
4598 && let Some(run) = strips.get(draw.strip_range.clone())
4599 {
4600 let end = self.alpha.len() as u32;
4601 self.push_run(run_start, end, run_external);
4602 run_external = None;
4603 self.push_backdrop(
4604 backdrop,
4605 sample.parity,
4606 draw.depth,
4607 run,
4608 window,
4609 );
4610 run_start = self.alpha.len() as u32;
4611 }
4612 continue;
4613 }
4614 let Some(paint) = pack_paint(&draw.paint, draw.depth, paint_slots)
4615 else {
4616 continue;
4617 };
4618 if paint.glyph_quad {
4619 // One rect instance, from the quad alone: the
4620 // draw's strips are its rectangle's coverage,
4621 // which the instance covers whole.
4622 let Some(quad) = glyph_quad(&frame.glyph_quads, draw.depth) else {
4623 continue;
4624 };
4625 if paint.external != run_external {
4626 let end = self.alpha.len() as u32;
4627 self.push_run(run_start, end, run_external);
4628 run_start = end;
4629 run_external = paint.external;
4630 }
4631 mark_class(
4632 &mut self.alpha_marks,
4633 self.alpha.len() as u32,
4634 paint.class,
4635 );
4636 if let Some(instance) = window.place_quad(quad, paint) {
4637 self.alpha.push(instance);
4638 }
4639 continue;
4640 }
4641 let Some(run) = strips.get(draw.strip_range.clone()) else {
4642 continue;
4643 };
4644 // Before the first of this draw's instances lands,
4645 // so the run that closes holds exactly the
4646 // instances drawn with the texture it names.
4647 if paint.external != run_external {
4648 let end = self.alpha.len() as u32;
4649 self.push_run(run_start, end, run_external);
4650 run_start = end;
4651 run_external = paint.external;
4652 }
4653 let to_opaque =
4654 paint.opaque && split_opaque && draw.depth < opaque_limit;
4655 let class = paint.class;
4656 mark_class(&mut self.alpha_marks, self.alpha.len() as u32, class);
4657 if to_opaque {
4658 mark_class(&mut self.opaque_marks, self.opaque.len() as u32, class);
4659 }
4660
4661 // A generation's last strip is its sentinel, which
4662 // is what carries the preceding strip's extent — so
4663 // every instance comes from a pair, and the
4664 // sentinel itself never becomes one.
4665 for pair in run.windows(2) {
4666 self.push_span(&pair[0], &pair[1], paint, to_opaque, window);
4667 }
4668 }
4669 }
4670 RoundOp::Composite(composite) => {
4671 let depth = self
4672 .layer_depth
4673 .get(composite.layer as usize)
4674 .copied()
4675 .unwrap_or(0);
4676 // A composite carries the deepest index inside its
4677 // layer, so a layer recorded after a punch is cut
4678 // against it exactly like a draw would be — which is
4679 // what keeps a translucent layer over the slot from
4680 // being erased by it.
4681 if cuts && !pending.is_empty() {
4682 self.cut_for_punches(
4683 &mut pending,
4684 strips,
4685 depth,
4686 &mut first_segment,
4687 &mut run_start,
4688 &mut run_external,
4689 );
4690 }
4691
4692 // Consecutive draw batches merge into one segment; a
4693 // composite is what breaks the run, because it binds a
4694 // different page as its colour source.
4695 let end = self.alpha.len() as u32;
4696 self.push_run(run_start, end, run_external);
4697
4698 deepest = deepest.max(depth);
4699 let source = match composite.retained {
4700 Some(retained) => CompositeSource::Retained(retained.bracket),
4701 None => CompositeSource::Page(composite.parity),
4702 };
4703 self.segments.push(Segment::Composite(end, source));
4704 self.note_class(self.routing.composite_class());
4705 self.alpha
4706 .push(composite_instance(composite, window.origin(), depth));
4707 run_start = self.alpha.len() as u32;
4708 // A composite draws through group 1's own atlas
4709 // binding, so the run after it starts un-bound again.
4710 run_external = None;
4711 }
4712 }
4713 }
4714
4715 let end = self.alpha.len() as u32;
4716 self.push_run(run_start, end, run_external);
4717
4718 // Accumulated rather than assigned: a layer whose round was cut
4719 // renders in several rounds, and the depth its composite carries is
4720 // the deepest index across all of them.
4721 let own_layer = page
4722 .map(|page| page.layer)
4723 .or(retained.map(|retained| retained.layer));
4724 if let Some(layer) = own_layer
4725 && let Some(slot) = self.layer_depth.get_mut(layer as usize)
4726 {
4727 *slot = (*slot).max(deepest);
4728 }
4729
4730 let released = released_groups(&round.released);
4731
4732 self.rounds.push(RoundPlan {
4733 page: page.map(|page| PagePlan {
4734 parity: page.parity,
4735 size: page.size,
4736 continued: page.continued,
4737 }),
4738 retained: retained.map(|retained| RetainedRound {
4739 bracket: retained.bracket,
4740 continued: retained.continued,
4741 }),
4742 segments: first_segment..self.segments.len(),
4743 released,
4744 filter,
4745 copy: None,
4746 // The round's last plan draws to its own end; a punch past
4747 // every op of the frame is issued after the whole walk instead.
4748 punch: (0, 0),
4749 });
4750 }
4751
4752 // Whatever no op was recorded after: a `ClearRect` recorded last, which
4753 // is the ordinary shape of a platform-view slot.
4754 if !pending.is_empty() {
4755 self.punch = self.push_punches(pending, strips);
4756 }
4757
4758 // The opaque region is one segment of the one depth-writing pass, cut
4759 // into class runs like any other.
4760 self.class_runs.clear();
4761 merge_class_runs(
4762 marked_runs(&self.opaque_marks, 0, self.opaque.len() as u32),
4763 self.routing.min_run,
4764 self.routing.uber,
4765 &mut self.class_runs,
4766 );
4767 debug_assert!(
4768 runs_draw(&self.class_runs, &self.opaque),
4769 "the opaque marks disagree with the instances they mark"
4770 );
4771 for run in &self.class_runs {
4772 self.opaque_segments
4773 .push(Segment::Strips(run.first, run.count, run.class));
4774 self.opaque_classes.insert(run.class);
4775 }
4776 }
4777
4778 /// Records that the round being planned draws a run of `class`.
4779 fn note_class(&mut self, class: PaintClass) {
4780 if self.planning_page {
4781 self.page_classes.insert(class);
4782 } else {
4783 self.surface_classes.insert(class);
4784 }
4785 }
4786
4787 /// Closes the open span of a surface round at `depth`, issuing every punch
4788 /// recorded before it as a pass of its own.
4789 ///
4790 /// This is the whole painter-order restoration: the ops recorded before the
4791 /// punch become a plan that ends here, the punch's own instances follow
4792 /// them in the buffer, and the ops recorded after it start a plan of their
4793 /// own — so the erase lands between the two rather than after both. A punch
4794 /// pass is a pass of its own because it draws through a different pipeline
4795 /// (destination-out) than the strips around it, not because of what it
4796 /// covers.
4797 ///
4798 /// Does nothing when no pending punch is shallower than `depth`, which is
4799 /// every op of every frame that records no clear.
4800 fn cut_for_punches(
4801 &mut self,
4802 pending: &mut &[ClearPunch],
4803 strips: &[Strip],
4804 depth: u32,
4805 first_segment: &mut usize,
4806 run_start: &mut u32,
4807 run_external: &mut Option<u32>,
4808 ) {
4809 let cut = pending
4810 .iter()
4811 .take_while(|punch| punch.depth < depth)
4812 .count();
4813 if cut == 0 {
4814 return;
4815 }
4816 let (issued, rest) = pending.split_at(cut);
4817 *pending = rest;
4818
4819 // Close the run this cut interrupts, so the plan's segments name only
4820 // what was recorded before the punch.
4821 let end = self.alpha.len() as u32;
4822 self.push_run(*run_start, end, *run_external);
4823 // The punch's own instances go in next and are drawn solid, so the run
4824 // resumed after the cut starts with nothing bound.
4825 *run_external = None;
4826
4827 let punch = self.push_punches(issued, strips);
4828 self.rounds.push(RoundPlan {
4829 page: None,
4830 retained: None,
4831 segments: *first_segment..self.segments.len(),
4832 // The pages a cut round consumed are handed back when its LAST plan
4833 // ends: nothing between two plans of one round acquires a page, and
4834 // a composite in an earlier plan still has to sample the page it
4835 // names.
4836 released: [false; MAX_LIVE_PAGES],
4837 filter: None,
4838 copy: None,
4839 punch,
4840 });
4841 *first_segment = self.segments.len();
4842 *run_start = self.alpha.len() as u32;
4843 }
4844
4845 /// Records the open run of instances `first..end` as segments drawn with
4846 /// the texture in `external` bound — one per paint-class run the span
4847 /// splits into — or nothing at all when the run is empty.
4848 ///
4849 /// The one place a strip segment is created, so both run-breaking rules —
4850 /// a segment holds instances sharing one group-1 binding, and one paint
4851 /// class ([`merge_class_runs`]) — are stated once rather than at each of
4852 /// the four points a run can close. The class runs tile the span in order,
4853 /// so the segments still draw `first..end` front to back exactly as one
4854 /// segment did.
4855 fn push_run(&mut self, first: u32, end: u32, external: Option<u32>) {
4856 if end.checked_sub(first).filter(|count| *count > 0).is_none() {
4857 return;
4858 }
4859 let Some(span) = self.alpha.get(first as usize..end as usize) else {
4860 return;
4861 };
4862 self.class_runs.clear();
4863 merge_class_runs(
4864 marked_runs(&self.alpha_marks, first, end),
4865 self.routing.min_run,
4866 self.routing.uber,
4867 &mut self.class_runs,
4868 );
4869 debug_assert!(
4870 runs_draw(&self.class_runs, span),
4871 "the alpha marks disagree with the instances they mark"
4872 );
4873 for index in 0..self.class_runs.len() {
4874 let run = self.class_runs[index];
4875 let at = first.saturating_add(run.first);
4876 self.segments.push(match external {
4877 Some(slot) => Segment::External(at, run.count, slot, run.class),
4878 None => Segment::Strips(at, run.count, run.class),
4879 });
4880 self.note_class(run.class);
4881 }
4882 }
4883
4884 /// Appends a backdrop blur's rectangle — the strip run `run` of its one
4885 /// draw, at painter-order `depth` — as instances sampling the blurred page
4886 /// in `parity`'s group, and the [`Segment::Backdrop`] drawing them.
4887 ///
4888 /// Each instance reads the page at its own position relative to the
4889 /// backdrop's region, which the page holds at its origin, so every pixel
4890 /// of the rectangle takes the blur of the backdrop under that very pixel;
4891 /// the run's own coverage — the rectangle's edges, and any coverage-mask
4892 /// clip it was generated under — weights it exactly as it would a fill.
4893 /// Never opaque: the blur of a backdrop reaching transparent edges is not.
4894 fn push_backdrop(
4895 &mut self,
4896 backdrop: &Backdrop,
4897 parity: PageParity,
4898 depth: u32,
4899 run: &[Strip],
4900 window: PageWindow,
4901 ) {
4902 let class = self.routing.composite_class();
4903 let paint = PackedPaint {
4904 payload: PaintPayload::Layer(backdrop.region.x0, backdrop.region.y0),
4905 paint: LAYER_PAINT_SOURCE | u32::from(u8::MAX),
4906 depth_index: depth,
4907 opaque: false,
4908 external: None,
4909 class,
4910 glyph_quad: false,
4911 };
4912 let first = self.alpha.len() as u32;
4913 for pair in run.windows(2) {
4914 self.push_span(&pair[0], &pair[1], paint, false, window);
4915 }
4916 let count = (self.alpha.len() as u32).saturating_sub(first);
4917 if count > 0 {
4918 self.segments.push(Segment::Backdrop(first, count, parity));
4919 self.note_class(class);
4920 }
4921 }
4922
4923 /// Turns `punches` into destination-out instances, appended to the alpha
4924 /// region and named by no round's segments, as `(first, count)`.
4925 ///
4926 /// Each punch is a strip run like any other, drawn with an opaque source so
4927 /// the blend state's `1 − src.a` reaches zero exactly where the coverage is
4928 /// full, and carrying the painter-order depth it was hoisted from — which
4929 /// still orders it against the frame's one depth-writing pass, whose opaque
4930 /// coverage is recorded once ahead of every round and so cannot be cut.
4931 fn push_punches(&mut self, punches: &[ClearPunch], strips: &[Strip]) -> (u32, u32) {
4932 let first = self.alpha.len() as u32;
4933
4934 for punch in punches {
4935 let paint = PackedPaint {
4936 payload: PaintPayload::Solid(PUNCH_SOURCE),
4937 paint: SOLID_PAINT,
4938 depth_index: punch.depth,
4939 // An erase never joins the depth-writing pass: it establishes
4940 // no colour for a later fragment to be rejected against.
4941 opaque: false,
4942 external: None,
4943 class: PaintClass::SolidCoverage,
4944 glyph_quad: false,
4945 };
4946 let Some(run) = strips.get(punch.strip_range.clone()) else {
4947 continue;
4948 };
4949 for pair in run.windows(2) {
4950 // Only the frame's own target is punched, so a punch's
4951 // instances are never shifted and never clipped.
4952 self.push_span(&pair[0], &pair[1], paint, false, PageWindow::ROOT);
4953 }
4954 }
4955
4956 (first, (self.alpha.len() as u32).saturating_sub(first))
4957 }
4958
4959 /// Emits the instances the `strip`/`next` pair describes: the strip's own
4960 /// alpha-sampled span, plus the solid span filling the gap to `next` when
4961 /// the winding between them says there is one.
4962 ///
4963 /// `window` shifts each instance's *geometry* into the round's target and
4964 /// clips it to the column that target holds, while the paint is still
4965 /// sampled at the scene position the strip was rasterized at — a layer's
4966 /// contents move into its page, the gradient or image painting them does
4967 /// not. An instance the window culls entirely is not emitted at all.
4968 fn push_span(
4969 &mut self,
4970 strip: &Strip,
4971 next: &Strip,
4972 paint: PackedPaint,
4973 to_opaque: bool,
4974 window: PageWindow,
4975 ) {
4976 let values = paint.values_at(strip.x, strip.y);
4977 let mut span = GpuStrip::from_strip_pair(strip, next, values);
4978 if window.place(&mut span, paint) {
4979 self.alpha.push(span);
4980 }
4981 // A gap starts where the strip ends rather than where it begins, so a
4982 // position-sampled paint is re-evaluated at the gap's own origin — the
4983 // gap is a different piece of the scene, not a continuation of the
4984 // span's sampling. `place` re-evaluates it once more if the window's
4985 // own left edge moves the instance again.
4986 if let Some(mut gap) = GpuStrip::gap_fill(strip, next, values) {
4987 gap.payload = paint.payload_at(gap.x, gap.y);
4988 if window.place(&mut gap, paint) {
4989 if to_opaque {
4990 self.opaque.push(gap);
4991 } else {
4992 self.alpha.push(gap);
4993 }
4994 }
4995 }
4996 }
4997
4998 /// Resolves `round`'s filter pass, if it has one, packing the layer's
4999 /// parameter block on first sight and claiming the pass's instance slot.
5000 ///
5001 /// `None` for an ordinary round, and also for the two shapes that cannot
5002 /// occur: a filter round with no page (the scheduler always gives one a
5003 /// page) and a recorded kind [`served_filter`] does not recognise (the
5004 /// scheduler refuses one before it plans a round). Both answer by leaving
5005 /// the round's pass unissued — its
5006 /// page is still cleared — rather than by asserting (E17).
5007 fn plan_filter(
5008 &mut self,
5009 frame: &CompiledFrame,
5010 round: &Round,
5011 backdrops: &[Backdrop],
5012 ) -> Option<FilterPlan> {
5013 let pass = round.filter_pass()?;
5014 let page = round.page()?;
5015 let recorded = frame.recorder.layers.get(pass.layer as usize)?;
5016 // A backdrop's layer is a regular one: its blur is the backdrop
5017 // record's, not the recording's.
5018 let data_offset = match backdrops
5019 .iter()
5020 .find(|backdrop| backdrop.layer == pass.layer)
5021 {
5022 Some(backdrop) => self.pack_filter_block(pass.layer, || {
5023 Some(GpuFilterData::from(GpuGaussianBlur::from(&backdrop.blur())))
5024 })?,
5025 None => self.filter_block(pass.layer, &recorded.kind)?,
5026 };
5027
5028 let instance = self.filter_passes;
5029 self.filter_passes = self.filter_passes.saturating_add(1);
5030 Some(FilterPlan {
5031 step: pass.step,
5032 source: pass.source,
5033 data_offset,
5034 original: SizeU16::from(page.bounds),
5035 instance,
5036 })
5037 }
5038
5039 /// The texel offset of `layer`'s parameter block, packing the block on
5040 /// first sight.
5041 ///
5042 /// A linear scan rather than a map: a frame's filter layers are counted in
5043 /// ones (a filter layer is served only directly under the surface), and one
5044 /// allocation-free vector beats a hash map that would have to be cleared
5045 /// every frame.
5046 fn filter_block(&mut self, layer: u32, kind: &RecordedLayerKind) -> Option<u32> {
5047 // The one dispatch every filter-recognising site in this crate shares
5048 // (`schedule::layer_role`/`filter_rounds`), so the block packed here is
5049 // for the same filter the scheduler planned the passes of.
5050 self.pack_filter_block(layer, || {
5051 Some(match served_filter(layer, kind).ok()? {
5052 ServedFilter::Blur(blur) => GpuFilterData::from(GpuGaussianBlur::from(&blur)),
5053 ServedFilter::DropShadow(shadow) => {
5054 GpuFilterData::from(GpuDropShadow::from(&shadow))
5055 }
5056 })
5057 })
5058 }
5059
5060 /// The texel offset of `layer`'s parameter block, packing the one `make`
5061 /// answers on first sight — [`Self::filter_block`]'s bookkeeping, shared
5062 /// with a backdrop blur, whose block comes from its own record.
5063 fn pack_filter_block(
5064 &mut self,
5065 layer: u32,
5066 make: impl FnOnce() -> Option<GpuFilterData>,
5067 ) -> Option<u32> {
5068 let index = match self.filter_layers.iter().position(|id| *id == layer) {
5069 Some(index) => index,
5070 None => {
5071 let block = make()?;
5072 self.filter_layers.push(layer);
5073 self.filter_blocks.push(block);
5074 self.filter_layers.len().saturating_sub(1)
5075 }
5076 };
5077
5078 u32::try_from(index)
5079 .ok()?
5080 .checked_mul(GpuFilterData::SIZE_TEXELS)
5081 }
5082
5083 /// Whether this frame records a hole-punch pass at all — at a cut inside a
5084 /// surface round, or after every round of the frame.
5085 ///
5086 /// Derived rather than counted alongside the instances: the two places a
5087 /// punch can land are the two places its instances are recorded from, and
5088 /// one answer read off both is one fewer field to keep in step.
5089 fn punches(&self) -> bool {
5090 self.punch.1 > 0 || self.rounds.iter().any(|plan| plan.punch.1 > 0)
5091 }
5092
5093 /// How many of this frame's rounds render *strips* into a pooled page, and
5094 /// so how many viewport uniforms of their own the frame needs.
5095 ///
5096 /// A filter round targets a page too and is deliberately not counted: it
5097 /// binds no viewport uniform at all, mapping NDC against the destination
5098 /// extent its own instance carries.
5099 fn page_rounds(&self) -> usize {
5100 self.rounds
5101 .iter()
5102 .filter(|plan| {
5103 (plan.page.is_some() || plan.retained.is_some())
5104 && plan.filter.is_none()
5105 && plan.copy.is_none()
5106 })
5107 .count()
5108 }
5109
5110 /// The passes a frame planned like this one records, in order, into
5111 /// `out` — the pass list of the module doc's *The frame's passes*, stated
5112 /// over the plan alone so it can be checked without a device.
5113 ///
5114 /// `record_frame` is what actually records them, and a debug build holds
5115 /// every frame it records to this list (`EngineRenderer::check_pass_list`).
5116 /// The two agree by construction on every shape the walk serves; the
5117 /// unreachable skips the walk guards against (a page the pool refuses, a
5118 /// pipeline the frame never resolved) are the only places they could part.
5119 #[cfg(any(test, debug_assertions))]
5120 fn plan_passes(&self, shape: &PassShape, out: &mut Vec<FramePass>) {
5121 out.clear();
5122 if !shape.merge {
5123 out.push(FramePass::Clear);
5124 }
5125 if !shape.instances {
5126 if shape.merge {
5127 out.push(FramePass::Opening);
5128 }
5129 return;
5130 }
5131 if !shape.merge && shape.opaque {
5132 out.push(FramePass::Opaque);
5133 }
5134
5135 let mut owed = shape.merge;
5136 for plan in &self.rounds {
5137 // A copy reads the surface back, so the opening — the clear and
5138 // the opaque strips — is recorded ahead of it.
5139 if plan.copy.is_some() {
5140 if owed {
5141 out.push(FramePass::Opening);
5142 owed = false;
5143 }
5144 out.push(FramePass::Copy);
5145 continue;
5146 }
5147 if plan.filter.is_some() {
5148 out.push(FramePass::Filter);
5149 continue;
5150 }
5151 if plan.page.is_some() || plan.retained.is_some() {
5152 out.push(FramePass::Page);
5153 } else if !plan.segments.is_empty() {
5154 out.push(if owed {
5155 FramePass::Opening
5156 } else {
5157 FramePass::Surface
5158 });
5159 owed = false;
5160 }
5161 if shape.punch && plan.punch.1 > 0 {
5162 if owed {
5163 out.push(FramePass::Opening);
5164 owed = false;
5165 }
5166 out.push(FramePass::Punch);
5167 }
5168 }
5169 if owed {
5170 out.push(FramePass::Opening);
5171 }
5172 if shape.punch && self.punch.1 > 0 {
5173 out.push(FramePass::Punch);
5174 }
5175 }
5176
5177 /// The instance bytes, opaque buffer first, so one vertex buffer serves
5178 /// every pass and each is a contiguous instance range into it.
5179 fn instance_bytes(&self) -> (&[u8], &[u8]) {
5180 (
5181 bytemuck::cast_slice(&self.opaque),
5182 bytemuck::cast_slice(&self.alpha),
5183 )
5184 }
5185}
5186
5187/// Records that the instances from `at` on carry `class`, unless the marks
5188/// already say so.
5189///
5190/// A draw that emitted nothing leaves its mark at the same index as the next
5191/// draw's, which then overwrites it — and drops it altogether when that leaves
5192/// two neighbouring marks of one class.
5193fn mark_class(marks: &mut Vec<(u32, PaintClass)>, at: u32, class: PaintClass) {
5194 match marks.last_mut() {
5195 Some(last) if last.1 == class => {}
5196 Some(last) if last.0 == at => {
5197 last.1 = class;
5198 let len = marks.len();
5199 if len >= 2 && marks[len - 2].1 == class {
5200 marks.pop();
5201 }
5202 }
5203 _ => marks.push((at, class)),
5204 }
5205}
5206
5207/// The class runs `marks` says cover instances `first..end`, relative to
5208/// `first` — the per-draw input [`merge_class_runs`] merges.
5209///
5210/// Every instance of a planned segment was emitted after its draw's mark, so
5211/// the mark at or before `first` names its class; a composite or punch
5212/// instance between two marks is never inside a segment's range.
5213fn marked_runs(
5214 marks: &[(u32, PaintClass)],
5215 first: u32,
5216 end: u32,
5217) -> impl Iterator<Item = ClassRun> + '_ {
5218 let from = marks
5219 .partition_point(|(at, _)| *at <= first)
5220 .saturating_sub(1);
5221 let tail = marks.get(from..).unwrap_or(&[]);
5222 tail.iter()
5223 .enumerate()
5224 .take_while(move |(_, (at, _))| *at < end)
5225 .filter_map(move |(index, (at, class))| {
5226 let start = (*at).max(first);
5227 let stop = tail.get(index + 1).map_or(end, |next| next.0.min(end));
5228 (stop > start).then(|| ClassRun {
5229 first: start - first,
5230 count: stop - start,
5231 class: *class,
5232 })
5233 })
5234}
5235
5236/// Whether `runs` tile `instances` front to back and each run's program draws
5237/// every instance in it, as far as the instance's own flag can tell
5238/// ([`PaintClass::admits_flag`]) — what the marks must agree with, checked in
5239/// debug builds.
5240fn runs_draw(runs: &[ClassRun], instances: &[GpuStrip]) -> bool {
5241 let mut next = 0_u32;
5242 for run in runs {
5243 if run.first != next || run.count == 0 {
5244 return false;
5245 }
5246 let Some(span) = instances.get(run.first as usize..(run.first + run.count) as usize) else {
5247 return false;
5248 };
5249 if !span
5250 .iter()
5251 .all(|instance| run.class.admits_flag(instance.paint_and_rect_flag))
5252 {
5253 return false;
5254 }
5255 next += run.count;
5256 }
5257 next as usize == instances.len()
5258}
5259
5260/// The column of device space one round's target holds: the origin its
5261/// instances are shifted to, and the edges they are clipped against.
5262///
5263/// A page holds its layer at the page's own origin, and its composite samples
5264/// exactly `(0, 0)`-to-its-own-extent back out
5265/// ([`Composite::source`](crate::schedule::Composite::source)). For a layer
5266/// [banded](crate::schedule::pages::page_bands) into column pages that makes
5267/// the band's own rectangle two things at once: the shift, and the *clip*. The
5268/// scheduler replays the layer's whole op list into every band — a band differs
5269/// only in which page it writes and where that page lands — so this is the site
5270/// that decides which part of each replayed strip belongs to the band at hand.
5271/// Emission is where the decision lives because it is the one place that knows
5272/// both the origin and the width; the alternative, clipping the ops
5273/// scheduler-side, would have to re-rasterize geometry the scheduler only holds
5274/// as draw ranges.
5275///
5276/// Two rules, and both halves of each matter:
5277///
5278/// - a span entirely outside the column contributes **no instance** — shifting
5279/// it by a saturating subtraction instead would clamp it onto the page's own
5280/// edge, stretching a span the scene drew elsewhere across content this band
5281/// really holds;
5282/// - a span straddling an edge has its geometry **and** its first alpha column
5283/// ([`GpuStrip::col_idx_or_rect_frac`]) advanced by the same amount, because
5284/// the shader reads that column unshifted and steps one column per pixel of
5285/// the instance (`shaders/strip.wgsl`'s `col_offset`): advancing the x
5286/// without the column would sample another pixel's coverage.
5287///
5288/// The frame's own surface is [`ROOT`](Self::ROOT), a window over the whole
5289/// device grid that shifts nothing and clips nothing. A layer that fits one
5290/// page is one band covering all of it, and a layer's bounds are the
5291/// tile-aligned union of its own draws' bounds, so the clip is a no-op for
5292/// every layer that is not banded.
5293#[derive(Debug, Clone, Copy, PartialEq, Eq)]
5294struct PageWindow {
5295 /// Left edge of the column in device space — what every instance's `x` is
5296 /// shifted by, and what one left of it is culled against.
5297 x0: u16,
5298 /// Right edge of the column in device space, exclusive.
5299 x1: u16,
5300 /// Top edge of the column in device space — what every instance's `y` is
5301 /// shifted by. A band spans its layer's whole height, so the vertical axis
5302 /// is shifted and never clipped.
5303 y0: u16,
5304}
5305
5306impl PageWindow {
5307 /// The whole device grid: the window a surface round carries.
5308 const ROOT: Self = Self {
5309 x0: 0,
5310 x1: u16::MAX,
5311 y0: 0,
5312 };
5313
5314 /// The window `page` renders through — its own tile-aligned bounds, which
5315 /// are one column band's for a banded layer and the whole layer's for
5316 /// every other.
5317 fn of(page: &PageTarget) -> Self {
5318 Self {
5319 x0: page.bounds.x0,
5320 x1: page.bounds.x1,
5321 y0: page.bounds.y0,
5322 }
5323 }
5324
5325 /// The window a retained page's round renders through: the body's own
5326 /// tile-aligned bounds, never banded.
5327 fn of_retained(target: &RetainedTarget) -> Self {
5328 Self {
5329 x0: target.bounds.x0,
5330 x1: target.bounds.x1,
5331 y0: target.bounds.y0,
5332 }
5333 }
5334
5335 /// The origin instances are shifted by.
5336 fn origin(self) -> (u16, u16) {
5337 (self.x0, self.y0)
5338 }
5339
5340 /// Clips `span` — a strip instance still carrying its scene coordinates —
5341 /// to this window and shifts it into the target, answering whether any of
5342 /// it survived.
5343 ///
5344 /// `paint` is the draw's own paint, needed because a left-clipped instance
5345 /// starts at a different scene position than the one it was rasterized at:
5346 /// a position-sampled paint (a gradient, an image) is re-evaluated there,
5347 /// exactly as a gap fill is at its own origin, so the paint keeps landing
5348 /// where the scene put it rather than being squeezed into the clipped span.
5349 fn place(self, span: &mut GpuStrip, paint: PackedPaint) -> bool {
5350 let end = span.x.saturating_add(span.width);
5351 if span.width == 0 || end <= self.x0 || span.x >= self.x1 {
5352 return false;
5353 }
5354
5355 let cut = self.x0.saturating_sub(span.x);
5356 if cut > 0 {
5357 span.x = self.x0;
5358 span.width = span.width.saturating_sub(cut);
5359 // The dense part of an instance starts at its left edge, so the
5360 // columns cut off the geometry are cut off the coverage too.
5361 let dense = cut.min(span.dense_width_or_rect_height);
5362 span.dense_width_or_rect_height = span.dense_width_or_rect_height.saturating_sub(dense);
5363 // A sparse instance names no column at all: the fragment stage
5364 // reads `col + dense_width` as "does this instance sample
5365 // coverage", so leaving a non-zero column on one whose dense part
5366 // is gone would send it to the alpha texture for coverage it never
5367 // wrote.
5368 span.col_idx_or_rect_frac = if span.dense_width_or_rect_height == 0 {
5369 0
5370 } else {
5371 span.col_idx_or_rect_frac.saturating_add(u32::from(dense))
5372 };
5373 span.payload = paint.payload_at(span.x, span.y);
5374 }
5375
5376 // Past the right edge is outside the region the composite samples, so
5377 // it reaches no pixel of the parent either way; clipping it keeps a
5378 // band's instances inside the band's own rectangle rather than relying
5379 // on the page's quantized extent to swallow the overhang.
5380 let over = end.saturating_sub(self.x1);
5381 if over > 0 {
5382 span.width = span.width.saturating_sub(over);
5383 span.dense_width_or_rect_height = span.dense_width_or_rect_height.min(span.width);
5384 }
5385
5386 span.x = span.x.saturating_sub(self.x0);
5387 span.y = span.y.saturating_sub(self.y0);
5388 span.width > 0
5389 }
5390
5391 /// The rect instance drawing `quad` with `paint`, clipped to this window
5392 /// and shifted into its target, or `None` when none of it survives.
5393 ///
5394 /// The same two rules [`Self::place`] keeps, for a whole rectangle: a quad
5395 /// outside the column contributes nothing, and one straddling an edge is
5396 /// trimmed — its atlas texel advanced by exactly the columns cut off its
5397 /// left edge, so every surviving pixel still samples the texel it did. A
5398 /// quad's coverage is its rectangle whole (it lands on whole pixels), so
5399 /// there is no coverage column to advance and the instance carries no edge
5400 /// fractions at all.
5401 fn place_quad(self, quad: &GlyphQuad, paint: PackedPaint) -> Option<GpuStrip> {
5402 let end = quad.x.saturating_add(quad.width);
5403 if quad.width == 0 || quad.height == 0 || end <= self.x0 || quad.x >= self.x1 {
5404 return None;
5405 }
5406 let cut = self.x0.saturating_sub(quad.x);
5407 let x = quad.x.max(self.x0);
5408 let width = end.min(self.x1).saturating_sub(x);
5409 let instance = GpuStrip::from_rect(
5410 x.saturating_sub(self.x0),
5411 quad.y.saturating_sub(self.y0),
5412 width,
5413 quad.height,
5414 0,
5415 StripDraw {
5416 payload: pack_u16_pair(quad.atlas_x.saturating_add(cut), quad.atlas_y),
5417 paint: paint.paint,
5418 depth_index: paint.depth_index,
5419 },
5420 );
5421 (width > 0).then_some(instance)
5422 }
5423}
5424
5425/// The glyph quad the draw at `depth` is, out of a frame's quads in depth
5426/// order — `None` only for a frame whose quads disagree with its draws, which
5427/// then draws nothing for that glyph rather than something wrong.
5428fn glyph_quad(quads: &[GlyphQuad], depth: u32) -> Option<&GlyphQuad> {
5429 let found = quads
5430 .binary_search_by_key(&depth, |quad| quad.depth)
5431 .ok()
5432 .and_then(|index| quads.get(index));
5433 debug_assert!(
5434 found.is_some(),
5435 "a glyph-quad draw with no quad at depth {depth}"
5436 );
5437 found
5438}
5439
5440/// The scheduler roles `plan` gives `frame`'s retained layers, written into
5441/// `roles` (cleared first). A body the store refused gets none and is
5442/// scheduled as the ordinary layer it lowered to.
5443fn retained_roles(frame: &CompiledFrame, plan: &RetainedPlan, roles: &mut RetainedLayers) {
5444 roles.clear();
5445 for (index, (bracket, decision)) in frame.retained.iter().zip(&plan.decisions).enumerate() {
5446 let id = u32::try_from(index).unwrap_or(u32::MAX);
5447 let role = match *decision {
5448 RetainedDecision::Record { .. } => RetainedRole::Record { bracket: id },
5449 RetainedDecision::Draw { dest, origin } => RetainedRole::Draw {
5450 bracket: id,
5451 dest,
5452 origin,
5453 },
5454 RetainedDecision::Hidden => RetainedRole::Hidden,
5455 RetainedDecision::Inline(_) => continue,
5456 };
5457 roles.set(bracket.layer, role);
5458 }
5459}
5460
5461/// Which of `frame`'s retained brackets the schedule renders into a retained
5462/// page — the bodies whose pages the store may commit. A body the store
5463/// admitted can still render nothing, nested inside a layer that composites
5464/// nothing.
5465fn retained_rendered(frame: &CompiledFrame, rounds: &[Round]) -> Vec<bool> {
5466 let mut rendered = vec![false; frame.retained.len()];
5467 for round in rounds {
5468 if let Some(target) = round.retained()
5469 && let Some(slot) = rendered.get_mut(target.bracket as usize)
5470 {
5471 *slot = true;
5472 }
5473 }
5474 rendered
5475}
5476
5477/// The single quad that composites a finished page onto `origin`-shifted
5478/// target.
5479///
5480/// A whole-rectangle instance with no fractional edges: a layer's bounds are
5481/// tile-aligned, so the quad covers whole pixels and the fragment stage leaves
5482/// its coverage at one. The payload is the page texel the quad's top-left
5483/// corner samples — the page holds the layer at its own origin, so that is
5484/// `Composite::source`'s origin and not the layer's device position.
5485fn composite_instance(composite: &Composite, origin: (u16, u16), depth: u32) -> GpuStrip {
5486 let bounds = composite.bounds;
5487 let source = composite.source();
5488
5489 GpuStrip::from_rect(
5490 bounds.x0.saturating_sub(origin.0),
5491 bounds.y0.saturating_sub(origin.1),
5492 bounds.width(),
5493 bounds.height(),
5494 0,
5495 StripDraw {
5496 payload: pack_u16_pair(source.x0, source.y0),
5497 paint: LAYER_PAINT_SOURCE | u32::from(pack_opacity(composite.opacity)),
5498 depth_index: depth,
5499 },
5500 )
5501}
5502
5503/// A layer's constant opacity as the eight bits a composite instance carries.
5504///
5505/// Clamped rather than refused: the scheduler already admits only opacities
5506/// strictly between zero and one, and rounding is what keeps a 0.5 layer at
5507/// exactly the 128 the reference renderer's own packing produces.
5508fn pack_opacity(opacity: f32) -> u8 {
5509 (opacity.clamp(0.0, 1.0) * 255.0).round() as u8
5510}
5511
5512/// Where a strip instance's payload comes from.
5513///
5514/// A solid paint carries its colour there; every other paint reads its colour
5515/// from a record instead, and spends the payload on the scene position the
5516/// paint is sampled at — which is why the payload is per instance rather than
5517/// per draw.
5518#[derive(Debug, Clone, Copy)]
5519enum PaintPayload {
5520 /// A premultiplied RGBA8 colour, the same for every instance of the draw.
5521 Solid(u32),
5522 /// The instance's own scene-space origin, packed as a `u16` pair.
5523 Position,
5524 /// The page texel under the instance's own origin, for a page holding the
5525 /// device region starting at `(x, y)` at its own origin: a backdrop blur's
5526 /// rectangle sampling its blurred page.
5527 Layer(u16, u16),
5528}
5529
5530/// One draw's paint, resolved to the values its instances repeat.
5531#[derive(Debug, Clone, Copy)]
5532struct PackedPaint {
5533 payload: PaintPayload,
5534 /// The packed paint descriptor, without [`RECT_STRIP_FLAG`](crate::gpu::strips::RECT_STRIP_FLAG).
5535 paint: u32,
5536 depth_index: u32,
5537 /// Whether every pixel this paint produces is opaque, and so whether its
5538 /// fully-covered spans may take the depth-writing pass.
5539 opaque: bool,
5540 /// The external-texture slot this paint's instances have to be drawn with
5541 /// bound, or `None` for every paint the frame's atlas binding serves.
5542 ///
5543 /// What breaks a pass's instances into runs: the strip pipelines have one
5544 /// external binding, so consecutive instances may share a segment only
5545 /// while this stays equal (see [`crate::gpu::bindings::ExternalRuns`]).
5546 external: Option<u32>,
5547 /// The strip program class this paint's instances are drawn with — from
5548 /// the resolved record, so an image is told apart by the path it samples
5549 /// through (see [`PaintClass::of_image`]).
5550 class: PaintClass,
5551 /// Whether this is a glyph run's shared record, whose draws are each one
5552 /// glyph quad ([`crate::compile::GlyphQuad`]) rather than a strip run.
5553 glyph_quad: bool,
5554}
5555
5556impl PackedPaint {
5557 /// The payload an instance whose scene origin is `(x, y)` carries.
5558 fn payload_at(self, x: u16, y: u16) -> u32 {
5559 match self.payload {
5560 PaintPayload::Solid(rgba) => rgba,
5561 PaintPayload::Position => pack_u16_pair(x, y),
5562 PaintPayload::Layer(x0, y0) => {
5563 pack_u16_pair(x.saturating_sub(x0), y.saturating_sub(y0))
5564 }
5565 }
5566 }
5567
5568 /// The per-instance values for an instance whose scene origin is `(x, y)`.
5569 fn values_at(self, x: u16, y: u16) -> StripDraw {
5570 StripDraw {
5571 payload: self.payload_at(x, y),
5572 paint: self.paint,
5573 depth_index: self.depth_index,
5574 }
5575 }
5576}
5577
5578/// Two `u16`s in one word, low half first — the packing the shader's
5579/// `unpack_u16_pair` reverses.
5580fn pack_u16_pair(x: u16, y: u16) -> u32 {
5581 u32::from(x) | (u32::from(y) << 16)
5582}
5583
5584/// The shader values for `paint`, resolved against the frame's paint slots.
5585///
5586/// Answers `None` for an indexed paint the frame could not resolve — one whose
5587/// entry is missing from `paint_slots` entirely, or one the lowering refused
5588/// because the engine has nowhere to hold its pixels yet (see the module
5589/// header). The draw is then skipped rather than stamped in a wrong colour.
5590fn pack_paint(
5591 paint: &Paint,
5592 depth: u32,
5593 paint_slots: &[Option<ResolvedPaint>],
5594) -> Option<PackedPaint> {
5595 match paint {
5596 Paint::Solid(color) => Some(PackedPaint {
5597 payload: PaintPayload::Solid(color.as_premul_rgba8().to_u32()),
5598 paint: SOLID_PAINT,
5599 depth_index: depth,
5600 opaque: color.is_opaque(),
5601 external: None,
5602 class: PaintClass::SolidCoverage,
5603 glyph_quad: false,
5604 }),
5605 Paint::Indexed(indexed) => {
5606 let Some(Some(resolved)) = paint_slots.get(indexed.index()) else {
5607 INDEXED_PAINT_WARNING.call_once(|| {
5608 log::warn!(
5609 "an indexed paint could not be lowered to a GPU record; those draws are \
5610 skipped (logged once)"
5611 );
5612 });
5613 return None;
5614 };
5615 Some(PackedPaint {
5616 payload: PaintPayload::Position,
5617 paint: pack_paint_descriptor(resolved.paint_type, resolved.texel_offset),
5618 depth_index: depth,
5619 opaque: resolved.opaque,
5620 external: resolved.external,
5621 class: resolved.class,
5622 glyph_quad: resolved.paint_type == PaintType::Glyph,
5623 })
5624 }
5625 }
5626}
5627
5628/// One encoded paint after lowering: where its record landed and how a strip
5629/// instance names it.
5630#[derive(Debug, Clone, Copy)]
5631struct ResolvedPaint {
5632 /// How the fragment shader reads the record.
5633 paint_type: PaintType,
5634 /// The texel the record starts at in the encoded-paint texture.
5635 texel_offset: u32,
5636 /// Whether every pixel the paint produces is opaque.
5637 opaque: bool,
5638 /// The external-texture slot this paint samples, for a paint that reads a
5639 /// caller-owned texture rather than the atlas (see
5640 /// [`crate::gpu::bindings`]). `None` — the ordinary case — means the
5641 /// frame's atlas binding serves it.
5642 external: Option<u32>,
5643 /// The strip program class the paint is drawn with ([`class_of_record`]).
5644 class: PaintClass,
5645}
5646
5647/// The strip program class a lowered record is drawn with.
5648///
5649/// The paint type decides it for every record but an image, whose record also
5650/// names the path it samples through: nearest or bilinear from the atlas is
5651/// [`PaintClass::Image`], bicubic or the external texture
5652/// [`PaintClass::ImageFiltered`]. The bits are the ones the shader reads —
5653/// quality in bits 0-1 of `image_params`, the source kind in bit 14.
5654fn class_of_record(record: &GpuEncodedPaint) -> PaintClass {
5655 match record {
5656 GpuEncodedPaint::Image(image) => PaintClass::of_image(
5657 image.image_params & IMAGE_QUALITY_BITS,
5658 (image.image_params >> IMAGE_SOURCE_KIND_SHIFT) & 1 == 1,
5659 ),
5660 _ => PaintClass::of_flag(pack_paint_descriptor(record.paint_type(), 0)),
5661 }
5662}
5663
5664/// Bits 0-1 of an image record's `image_params`: the sampling quality.
5665const IMAGE_QUALITY_BITS: u32 = 0b11;
5666
5667/// The bit of an image record's `image_params` marking an external texture.
5668const IMAGE_SOURCE_KIND_SHIFT: u32 = 14;
5669
5670/// One resource texture and the extent it currently holds.
5671#[derive(Debug)]
5672struct ResourceTexture {
5673 texture: wgpu::Texture,
5674 view: wgpu::TextureView,
5675 width: u32,
5676 height: u32,
5677 /// The rows, counted from the top, that may differ from what a
5678 /// full-footprint upload of the next frame would leave there.
5679 ///
5680 /// A queue upload writes the whole texture every frame: the frame's bytes,
5681 /// then the upload buffer's tail — zeroes for coverage, the reused staging
5682 /// vector's earlier records for encoded paints. A staged upload writes only
5683 /// the rows the frame fills plus every row the last frame left live, and
5684 /// so leaves the texture byte-for-byte as the full upload would: the rows
5685 /// below this mark already hold exactly that tail. Rows at or beyond the
5686 /// current frame's filled rows are never read, so a stale row there (a
5687 /// dropped encoder leaves its rows past `filled` as they were) cannot reach
5688 /// a pixel. A new texture is live throughout, so its first upload is a full
5689 /// one.
5690 live_rows: u32,
5691}
5692
5693impl ResourceTexture {
5694 fn new(device: &wgpu::Device, descriptor: &wgpu::TextureDescriptor<'_>) -> Self {
5695 let texture = device.create_texture(descriptor);
5696 let view = texture.create_view(&wgpu::TextureViewDescriptor::default());
5697 Self {
5698 texture,
5699 view,
5700 width: descriptor.size.width,
5701 height: descriptor.size.height,
5702 live_rows: descriptor.size.height,
5703 }
5704 }
5705
5706 /// The rows a frame whose bytes fill `filled_rows` rows uploads through a
5707 /// stage: those, and every row the last upload left live (see
5708 /// [`Self::live_rows`]), never past the texture's height.
5709 fn staged_rows(&self, filled_rows: u32) -> u32 {
5710 staged_rows(filled_rows, self.live_rows, self.height)
5711 }
5712
5713 /// The extent of an upload of the texture's top `rows` rows.
5714 fn rows_extent(&self, rows: u32) -> wgpu::Extent3d {
5715 wgpu::Extent3d {
5716 width: self.width,
5717 height: rows,
5718 depth_or_array_layers: 1,
5719 }
5720 }
5721
5722 /// The whole texture as a copy destination.
5723 fn copy_target(&self) -> wgpu::TexelCopyTextureInfo<'_> {
5724 wgpu::TexelCopyTextureInfo {
5725 texture: &self.texture,
5726 mip_level: 0,
5727 origin: wgpu::Origin3d::ZERO,
5728 aspect: wgpu::TextureAspect::All,
5729 }
5730 }
5731
5732 /// The extent one full-texture upload covers.
5733 fn extent(&self) -> wgpu::Extent3d {
5734 wgpu::Extent3d {
5735 width: self.width,
5736 height: self.height,
5737 depth_or_array_layers: 1,
5738 }
5739 }
5740}
5741
5742/// The GPU resources one renderer holds across frames.
5743#[derive(Debug)]
5744struct FrameResources {
5745 alphas: ResourceTexture,
5746 paints: ResourceTexture,
5747 gradients: ResourceTexture,
5748 /// Stand-ins for the strip shader's bindings a given pass has nothing real
5749 /// for: the layer input (a real page when a composite is being drawn, this
5750 /// otherwise), the glyph/image atlas array until the first image is
5751 /// resident, and an externally bound texture, which nothing writes yet.
5752 /// Every declared binding has to be bound for a pass to validate, whether
5753 /// or not an instance samples it.
5754 placeholders: Placeholders,
5755 /// The linear, clamp-to-edge sampler the image-sampler program family
5756 /// filters bilinear atlas images through, bound beside the atlas array in
5757 /// every strip group 1; `None` for the manual family, whose layout
5758 /// declares no sampler (see [`EngineRenderer::with_image_sampler`]).
5759 atlas_sampler: Option<wgpu::Sampler>,
5760 config: wgpu::Buffer,
5761 /// One viewport uniform per page round of the busiest frame so far.
5762 ///
5763 /// A round's NDC mapping is against the extent of the attachment it writes,
5764 /// and a page's extent is neither the frame's nor the same from one round
5765 /// to the next, so each needs a buffer of its own — a bind group holds the
5766 /// whole buffer, not an offset into one. Grown only, like every other
5767 /// retained resource here.
5768 page_configs: Vec<wgpu::Buffer>,
5769 instances: Option<wgpu::Buffer>,
5770 instance_capacity: u64,
5771 /// The GPU records this frame's indexed paints resolve against, in
5772 /// serialization order — which is the order the texel offsets in
5773 /// `paint_slots` were taken from.
5774 paints_data: Vec<GpuEncodedPaint>,
5775 /// One slot per encoded paint of the frame, indexed by
5776 /// [`Paint::Indexed`](vello_common::paint::Paint::Indexed): `None` for one
5777 /// the engine could not lower. Kept parallel to the *encoded* paints
5778 /// rather than compacted to the lowered ones, because a draw names its
5779 /// paint by the compiler's index.
5780 paint_slots: Vec<Option<ResolvedPaint>>,
5781 /// Reusable per-paint ramp residency, filled from a frame's LUT requests
5782 /// before its paints are lowered.
5783 paint_ramps: Vec<Option<CachedRamp>>,
5784 /// Reusable staging for the encoded-paint upload, padded to the texture's
5785 /// footprint.
5786 paint_staging: Vec<u8>,
5787 /// The distinct external textures this frame's paints sample, in the order
5788 /// they were first named — the slot numbering the pass segments and the
5789 /// group-1 bind groups both address by.
5790 external_runs: ExternalRuns,
5791 /// One set of bind groups per strip render state, built against any one
5792 /// of that state's class pipelines. Every strip pipeline — each state,
5793 /// each paint class — is built against one explicit layout
5794 /// ([`crate::gpu::bindings::strip_pipeline_layout`]), so a state's set binds
5795 /// under all of its classes; keeping one set per state is what the map
5796 /// already did when each pipeline derived a layout of its own.
5797 bind_groups: HashMap<EnginePipeline, StripBindGroups>,
5798 /// The target format the live bind groups were built against; a change
5799 /// means different pipelines, so the whole map is dropped rather than
5800 /// accumulating a set per format ever rendered to.
5801 bind_group_format: Option<wgpu::TextureFormat>,
5802 /// The real image atlas array, created lazily by the first frame that
5803 /// makes an image resident and grown from then on — `None` is exactly
5804 /// [`Placeholders::atlas_array`]'s domain, a renderer that has never
5805 /// drawn an image.
5806 atlas: Option<AtlasArray>,
5807 /// The atlas rectangle every image the renderer has ever drawn currently
5808 /// holds, keyed by the stable [`ImageId`] the compiler's own residency
5809 /// minted it. See [`FrameResources::resolve_paints`] for how this is
5810 /// kept in step with residency across frames without reaching into the
5811 /// compiler's own cache.
5812 image_registry: HashMap<ImageId, ResidentImage>,
5813 /// Reused zero fill for the atlas eviction clears.
5814 clear_zeros: Vec<u8>,
5815 /// Group-0 and filter-source bind groups naming a page, kept across frames
5816 /// and swept at every frame's end (see [`SourceGroupCache`]). Cleared with
5817 /// the coverage texture they all name.
5818 source_groups: SourceGroupCache,
5819 /// Strip, external and image bind groups built by the group-set paths.
5820 groups_created: u64,
5821 /// How many atlas regions have been declined — a write or clear the array
5822 /// refused, or one the budget says the array could not hold. Counted rather
5823 /// than dropped silently, because each one is a region the frame believed
5824 /// it had filled.
5825 refused_regions: u64,
5826}
5827
5828impl FrameResources {
5829 fn new(device: &wgpu::Device, dim: u32, image_sampler: bool) -> Self {
5830 let min = gpu::MIN_RESOURCE_TEXTURE_HEIGHT;
5831 Self {
5832 alphas: ResourceTexture::new(device, &gpu::alpha_texture_descriptor(dim, min)),
5833 paints: ResourceTexture::new(
5834 device,
5835 &gpu::paint_texture::encoded_paints_texture_descriptor(dim, min),
5836 ),
5837 gradients: ResourceTexture::new(device, &gradient_texture_descriptor(dim, min)),
5838 placeholders: Placeholders::new(device),
5839 atlas_sampler: image_sampler.then(|| atlas_sampler(device)),
5840 config: device.create_buffer(&config_descriptor("frust-engine config uniform")),
5841 page_configs: Vec::new(),
5842 instances: None,
5843 instance_capacity: 0,
5844 paints_data: Vec::new(),
5845 paint_slots: Vec::new(),
5846 paint_ramps: Vec::new(),
5847 paint_staging: Vec::new(),
5848 external_runs: ExternalRuns::new(),
5849 bind_groups: HashMap::new(),
5850 bind_group_format: None,
5851 atlas: None,
5852 image_registry: HashMap::new(),
5853 clear_zeros: Vec::new(),
5854 source_groups: SourceGroupCache::default(),
5855 groups_created: 0,
5856 refused_regions: 0,
5857 }
5858 }
5859
5860 /// Drops the atlas array, everything recorded about what lives in it, and
5861 /// the bind groups naming it.
5862 ///
5863 /// What a re-budget needs: the rectangles the registry holds were allocated
5864 /// in a geometry that no longer exists, so keeping any of them would point
5865 /// a paint at a rectangle of a texture that is gone.
5866 fn reset_atlas(&mut self) {
5867 self.atlas = None;
5868 self.image_registry.clear();
5869 self.bind_groups.clear();
5870 }
5871
5872 /// Count `regions` atlas regions as declined, saying so once.
5873 ///
5874 /// Once, not per region: a budget and an array that disagree disagree about
5875 /// every region, and a per-frame line would bury the fact under itself. The
5876 /// count on [`EngineRenderer::refused_atlas_regions`] is the measure.
5877 fn note_refused_regions(&mut self, regions: u64) {
5878 self.refused_regions = self.refused_regions.saturating_add(regions);
5879 ATLAS_REFUSAL_WARNING.call_once(|| {
5880 log::warn!(
5881 "an atlas region was refused by the image atlas array; those images are skipped \
5882 and their uploads re-offered on a later frame (logged once — see \
5883 EngineRenderer::refused_atlas_regions for the count)"
5884 );
5885 });
5886 }
5887
5888 /// Services `frame`'s LUT and image residency, then lowers its encoded
5889 /// paints into the records the shader samples.
5890 ///
5891 /// Leaves `paints_data` holding the lowered records in serialization order
5892 /// and `paint_slots` naming, per *encoded* paint, the texel its record
5893 /// starts at — or `None` where the paint could not be lowered.
5894 ///
5895 /// A solid-only frame leaves both empty and touches neither the cache nor
5896 /// the paint texture, so it costs exactly what it did before paints were
5897 /// wired up. Image residency is still serviced even then, since an image
5898 /// can be evicted on a frame that draws nothing at all (see
5899 /// [`Self::update_image_registry`]).
5900 fn resolve_paints(
5901 &mut self,
5902 frame: &CompiledFrame,
5903 cache: &mut GradientCache,
5904 budget: AtlasBudget,
5905 externals: &ExternalTextures,
5906 ) {
5907 self.paints_data.clear();
5908 self.paint_slots.clear();
5909 self.external_runs.clear();
5910
5911 // Kept in step every frame, not only when this frame's own paints
5912 // need it: an image reaped by the compiler's age-based eviction while
5913 // nothing draws it must still be forgotten here, or a later draw that
5914 // reuses its freed rectangle's `ImageId` would read the stale entry.
5915 self.update_image_registry(frame, budget);
5916
5917 if frame.encoded_paints.is_empty() {
5918 return;
5919 }
5920
5921 // Ramp residency next, for the whole frame: a ramp's offset is only
5922 // meaningful once the cache has finished baking this frame's misses,
5923 // and a record built before that would name a ramp that had not been
5924 // packed yet.
5925 self.paint_ramps.clear();
5926 self.paint_ramps.resize(frame.encoded_paints.len(), None);
5927 for request in &frame.lut_requests {
5928 let ramp = resolve_lut_request(*request, &frame.encoded_paints, cache);
5929 if let Some(slot) = self.paint_ramps.get_mut(request.paint_index) {
5930 *slot = ramp;
5931 }
5932 }
5933
5934 let mut texel_offset = 0;
5935 // Ascending, like the walk below, so one cursor marks them.
5936 let mut glyph_records = frame.glyph_run_paints.iter().copied().peekable();
5937 for (index, paint) in frame.encoded_paints.iter().enumerate() {
5938 // A glyph run's shared record: lowered like any atlas image, but
5939 // drawn by glyph quads (see `crate::compile::GlyphQuad`).
5940 let glyph_record = glyph_records.next_if_eq(&index).is_some();
5941 // An external texture's slot travels with its record: the record
5942 // itself only says "sample the external binding", and which
5943 // texture that binding holds is settled per run when the pass is
5944 // recorded rather than per paint.
5945 let mut external = None;
5946 let lowered = match paint {
5947 EncodedPaint::Image(image) => image_id(image)
5948 .and_then(|id| self.image_registry.get(&id))
5949 .and_then(|resident| {
5950 lower_encoded_image(image, resident)
5951 .map(|record| fit_minified(record, resident))
5952 }),
5953 // A paint naming a texture nothing is bound under lowers to
5954 // nothing, so its draws are skipped rather than sampling
5955 // whichever texture the binding happens to hold.
5956 EncodedPaint::ExternalTexture(entry) => externals
5957 .view(entry.texture_id.0)
5958 .and_then(|_| self.external_runs.slot_of(entry.texture_id.0))
5959 .map(|slot| {
5960 external = Some(slot);
5961 lower_encoded_external(entry)
5962 }),
5963 _ => {
5964 let ramp = self.paint_ramps.get(index).copied().flatten();
5965 lower_encoded_paint(paint, ramp)
5966 }
5967 };
5968 let slot = lowered.map(|record| {
5969 let resolved = ResolvedPaint {
5970 paint_type: if glyph_record {
5971 PaintType::Glyph
5972 } else {
5973 record.paint_type()
5974 },
5975 texel_offset,
5976 opaque: !paint.may_have_transparency(),
5977 external,
5978 class: if glyph_record {
5979 PaintClass::Glyph
5980 } else {
5981 class_of_record(&record)
5982 },
5983 };
5984 texel_offset += record.texel_len();
5985 self.paints_data.push(record);
5986 resolved
5987 });
5988 self.paint_slots.push(slot);
5989 }
5990 }
5991
5992 /// Keeps [`Self::image_registry`] in step with the compiler's own image
5993 /// residency, without reaching into it: the residency's rectangles are
5994 /// not reachable from here (see [`crate::gpu::paint_texture::lower_encoded_paint`]'s
5995 /// doc for why), so this reconstructs the same information from what a
5996 /// compiled frame already reports.
5997 ///
5998 /// Two passes, and each reads the frame's plan directly rather than
5999 /// inferring anything from draw order. First, every region this frame's
6000 /// residency reaped is forgotten — `frame.image_forgets` names it by
6001 /// rectangle, and a rectangle uniquely identifies the one image that held it
6002 /// (padding is always zero in this engine, so the reported and the stored
6003 /// rectangle are the same value; see [`crate::cache::images`]'s module doc).
6004 /// Second, every entry of `frame.image_uploads` is registered under the
6005 /// [`ImageId`] it carries.
6006 ///
6007 /// The order matters and the id does. Evictions run first so a same-frame
6008 /// evict-then-reallocate that reuses a rectangle registers the new tenant
6009 /// rather than having it removed again. And the upload naming its own id is
6010 /// what makes this sound under a *re-offered* plan: an upload the previous
6011 /// frame did not service is reported again alongside no new draw of its own,
6012 /// so a walk pairing uploads positionally against this frame's encoded
6013 /// paints would hand a fresh image the stale upload's rectangle.
6014 ///
6015 /// A region the atlas array could not hold at `budget`'s geometry and this
6016 /// frame's depth is counted and left unregistered instead. Its draws are
6017 /// then skipped, which is the whole point: a registered rectangle nothing
6018 /// wrote would be sampled as whatever the texture happened to contain.
6019 fn update_image_registry(&mut self, frame: &CompiledFrame, budget: AtlasBudget) {
6020 if !frame.image_forgets.is_empty() {
6021 // A set rather than `Vec::contains`: both sides of this scan scale
6022 // with content — the registry with how many images and glyph slots
6023 // are resident, the plan with how hard the atlas is churning — and
6024 // their product is the frame path's, not a report's.
6025 let cleared: HashSet<AtlasRegion> = frame.image_forgets.iter().copied().collect();
6026 self.image_registry
6027 .retain(|_, resident| !cleared.contains(&resident.region));
6028 }
6029
6030 let mut refused = 0_u64;
6031 for upload in &frame.image_uploads {
6032 if !budget.contains(upload.region, frame.atlas_layers) {
6033 refused = refused.saturating_add(1);
6034 continue;
6035 }
6036 self.image_registry.insert(
6037 upload.id,
6038 ResidentImage {
6039 id: upload.id,
6040 region: upload.region,
6041 natural: upload.natural,
6042 padding: u32::from(ATLAS_PADDING),
6043 may_have_transparency: upload.may_have_transparency,
6044 },
6045 );
6046 }
6047 // The glyph half of the same registry, and the reason it is a second
6048 // loop rather than a branch inside the first: a glyph slot carries no
6049 // pixels and is *not* re-offered across frames, because the pixels are
6050 // produced by the replay pass rather than uploaded from here. Its
6051 // rectangle is reported by every draw that names it (see
6052 // [`crate::compile::GlyphSlot`]), so registering it here is what makes
6053 // a handle `glifo` recycled resolve against its current occupant.
6054 for slot in &frame.glyph_slots {
6055 if !budget.contains(slot.region, frame.atlas_layers) {
6056 refused = refused.saturating_add(1);
6057 continue;
6058 }
6059 self.image_registry.insert(
6060 slot.id,
6061 ResidentImage {
6062 id: slot.id,
6063 region: slot.region,
6064 // Never minified: a glyph is rasterized straight into the
6065 // rectangle it was allocated, so the natural extent and the
6066 // resident one are the same value by construction.
6067 natural: slot.region.size,
6068 padding: slot.padding,
6069 may_have_transparency: true,
6070 },
6071 );
6072 }
6073
6074 if refused > 0 {
6075 self.note_refused_regions(refused);
6076 }
6077 }
6078
6079 /// Grows or creates the image atlas array to hold `layers` layers at
6080 /// `budget`'s per-layer extent, clearing the live bind groups when it
6081 /// does — a bind group built against the old (or absent) atlas view would
6082 /// otherwise sample nothing, or a freed texture.
6083 ///
6084 /// A frame that has never made an image resident (`layers == 0`) leaves
6085 /// the atlas unset, so the strip shader's binding stays on
6086 /// [`Placeholders::atlas_array`] until the first one is.
6087 ///
6088 /// Growth submits a maintenance command buffer of its own rather than
6089 /// recording into the frame's encoder — see [`crate::gpu::atlas`]. So this
6090 /// must be called after the frame's last fallible step (a refused frame
6091 /// must submit nothing) and before its atlas writes are issued (the copy has
6092 /// to precede them, and a submit flushes whatever is already queued).
6093 fn ensure_atlas(
6094 &mut self,
6095 device: &wgpu::Device,
6096 queue: &wgpu::Queue,
6097 budget: AtlasBudget,
6098 layers: u32,
6099 ) -> bool {
6100 let grew = match &mut self.atlas {
6101 None if layers == 0 => false,
6102 None => {
6103 self.atlas = Some(AtlasArray::with_layers(
6104 device,
6105 budget.atlas_size.0,
6106 budget.atlas_size.1,
6107 layers,
6108 ));
6109 true
6110 }
6111 Some(atlas) => atlas.ensure_layers(device, queue, layers),
6112 };
6113 if grew {
6114 self.bind_groups.clear();
6115 }
6116 grew
6117 }
6118
6119 /// Flushes `frame`'s atlas evictions and uploads against the live atlas
6120 /// array, evictions first — a rectangle this frame's residency freed may
6121 /// already hold a fresh upload by the time this runs (see
6122 /// [`crate::cache::images`]'s module doc), so clearing after writing
6123 /// would erase the image that just moved in.
6124 ///
6125 /// Answers whether the whole plan reached the array — which is what tells
6126 /// the compiler it may stop re-offering it. A region the array declined is
6127 /// counted (see [`Self::note_refused_regions`]) and the plan stays pending,
6128 /// so a later frame that has grown the array writes it rather than the
6129 /// image being lost.
6130 ///
6131 /// An absent array with a plan to service is that same disagreement rather
6132 /// than a quiet no-op: both are driven by the same `frame.atlas_layers`, so
6133 /// an empty plan and an absent atlas normally agree.
6134 fn upload_atlas(&mut self, queue: &wgpu::Queue, frame: &CompiledFrame) -> bool {
6135 let refused = match self.atlas.as_ref() {
6136 None => (frame.image_evictions.len() + frame.image_uploads.len()) as u64,
6137 Some(atlas) => {
6138 let mut refused = 0_u64;
6139 for region in &frame.image_evictions {
6140 if !atlas.clear_region_with(queue, *region, &mut self.clear_zeros) {
6141 refused = refused.saturating_add(1);
6142 }
6143 }
6144 for upload in &frame.image_uploads {
6145 if !atlas.write_region(queue, upload.region, upload.pixels.data_as_u8_slice()) {
6146 refused = refused.saturating_add(1);
6147 }
6148 }
6149 refused
6150 }
6151 };
6152
6153 if refused == 0 {
6154 return true;
6155 }
6156 self.note_refused_regions(refused);
6157 false
6158 }
6159
6160 /// Makes sure there is one viewport uniform per page round of this frame.
6161 ///
6162 /// Grown only: the buffers are 32 bytes each and a frame that once needed
6163 /// four keeps them rather than reallocating on the next frame that does.
6164 fn ensure_page_configs(&mut self, device: &wgpu::Device, rounds: usize) {
6165 while self.page_configs.len() < rounds {
6166 self.page_configs
6167 .push(device.create_buffer(&config_descriptor("frust-engine page config uniform")));
6168 }
6169 }
6170
6171 /// The texels this frame's encoded paints occupy.
6172 fn paint_texels(&self) -> u32 {
6173 self.paints_data
6174 .iter()
6175 .map(GpuEncodedPaint::texel_len)
6176 .sum()
6177 }
6178
6179 /// The height the gradient LUT texture must grow to for every ramp the
6180 /// cache has packed, or `None` when it already fits.
6181 ///
6182 /// # Errors
6183 ///
6184 /// [`EngineError::PaintCapacity`]: a LUT set past the resource dimension
6185 /// squared has nowhere to live, and the frame path reports it rather than
6186 /// asserting.
6187 fn grown_gradient_height(&self, cache: &GradientCache) -> Result<Option<u32>, EngineError> {
6188 let width = self.gradients.width;
6189 let texels = u32::try_from(cache.luts_size() / BYTES_PER_TEXEL as usize)
6190 .map_err(|_| EngineError::PaintCapacity)?;
6191 let required = texels.div_ceil(width).max(gpu::MIN_RESOURCE_TEXTURE_HEIGHT);
6192 if required > width {
6193 return Err(EngineError::PaintCapacity);
6194 }
6195 Ok((required > self.gradients.height).then_some(required))
6196 }
6197
6198 fn resize_alphas(&mut self, device: &wgpu::Device, height: Option<u32>) {
6199 if let Some(height) = height {
6200 self.alphas = ResourceTexture::new(
6201 device,
6202 &gpu::alpha_texture_descriptor(self.alphas.width, height),
6203 );
6204 self.bind_groups.clear();
6205 // Every page group names the old coverage view.
6206 self.source_groups.clear();
6207 }
6208 }
6209
6210 fn resize_paints(&mut self, device: &wgpu::Device, height: Option<u32>) {
6211 if let Some(height) = height {
6212 self.paints = ResourceTexture::new(
6213 device,
6214 &gpu::paint_texture::encoded_paints_texture_descriptor(self.paints.width, height),
6215 );
6216 self.bind_groups.clear();
6217 }
6218 }
6219
6220 fn resize_gradients(&mut self, device: &wgpu::Device, height: Option<u32>) {
6221 if let Some(height) = height {
6222 self.gradients = ResourceTexture::new(
6223 device,
6224 &gradient_texture_descriptor(self.gradients.width, height),
6225 );
6226 self.bind_groups.clear();
6227 }
6228 }
6229
6230 /// Uploads the frame's coverage, paint records, colour ramps, atlas
6231 /// evictions/uploads and config, answering whether the atlas plan was
6232 /// serviced in full (see [`Self::upload_atlas`]).
6233 ///
6234 /// The per-frame uploads — coverage, paint records, config — go through
6235 /// `stage` when the frame has one, as copies recorded into `encoder` ahead
6236 /// of every pass; the on-change ones (colour ramps, atlas regions) stay
6237 /// queue writes, which is also what every one of them is without a stage.
6238 #[expect(
6239 clippy::too_many_arguments,
6240 reason = "the upload's two routes (queue, encoder + stage) beside the \
6241 frame, the ramp cache and the extents it uploads against"
6242 )]
6243 fn upload(
6244 &mut self,
6245 queue: &wgpu::Queue,
6246 encoder: &mut wgpu::CommandEncoder,
6247 mut stage: Option<&mut StagingFrame>,
6248 frame: &mut CompiledFrame,
6249 cache: &mut GradientCache,
6250 size: (u16, u16),
6251 dim: u32,
6252 ) -> bool {
6253 let atlas_serviced = self.upload_atlas(queue, frame);
6254
6255 let alphas = &self.alphas;
6256 let bytes_per_row = gpu::resource_bytes_per_row(alphas.width);
6257 let coverage = &frame.strips.alphas;
6258 // Staged, only the rows this frame's coverage fills and the rows the
6259 // last frame left live are written — the rest of the texture already
6260 // holds the zeroes a full upload would put there.
6261 let filled = rows_filled(coverage.len(), bytes_per_row).min(alphas.height);
6262 let rows = alphas.staged_rows(filled);
6263 let staged = stage.as_deref_mut().is_some_and(|stage| {
6264 rows == 0
6265 || stage.copy_to_texture(
6266 encoder,
6267 alphas.copy_target(),
6268 bytes_per_row,
6269 rows,
6270 alphas.rows_extent(rows),
6271 |dst| gpu::write_zero_padded(dst, coverage),
6272 )
6273 });
6274 if !staged {
6275 gpu::with_padded_alphas(
6276 &mut frame.strips.alphas,
6277 alphas.width,
6278 alphas.height,
6279 |bytes| {
6280 queue.write_texture(
6281 alphas.copy_target(),
6282 bytes,
6283 resource_layout(bytes_per_row, alphas.height),
6284 alphas.extent(),
6285 );
6286 },
6287 );
6288 }
6289 self.alphas.live_rows = filled;
6290
6291 if !self.paints_data.is_empty() {
6292 let footprint = gpu::resource_texture_bytes(self.paints.width, self.paints.height);
6293 self.paint_staging
6294 .resize(usize::try_from(footprint).unwrap_or(usize::MAX), 0);
6295 // The height was checked to fit before any pass was recorded, so a
6296 // buffer too short for the records is unreachable; skipping the
6297 // upload rather than unwrapping keeps the frame path total anyway.
6298 if let Ok(len) =
6299 GpuEncodedPaint::serialize_to_buffer(&self.paints_data, &mut self.paint_staging)
6300 {
6301 let paints = &self.paints;
6302 let staging = &self.paint_staging;
6303 let bytes_per_row = gpu::resource_bytes_per_row(paints.width);
6304 // The rows below the live mark already hold the staging
6305 // vector's own tail, which is what a full upload rewrites.
6306 let filled = rows_filled(len, bytes_per_row).min(paints.height);
6307 let rows = paints.staged_rows(filled);
6308 let staged = stage.as_deref_mut().is_some_and(|stage| {
6309 rows == 0
6310 || stage.copy_to_texture(
6311 encoder,
6312 paints.copy_target(),
6313 bytes_per_row,
6314 rows,
6315 paints.rows_extent(rows),
6316 |dst| gpu::write_zero_padded(dst, staging),
6317 )
6318 });
6319 if !staged {
6320 queue.write_texture(
6321 paints.copy_target(),
6322 staging,
6323 resource_layout(bytes_per_row, paints.height),
6324 paints.extent(),
6325 );
6326 }
6327 self.paints.live_rows = filled;
6328 }
6329 }
6330
6331 if cache.has_changed() {
6332 let layout = GradientTextureLayout {
6333 width: self.gradients.width,
6334 height: self.gradients.height,
6335 };
6336 if let Some(upload) = cache.begin_upload(layout) {
6337 queue.write_texture(
6338 self.gradients.copy_target(),
6339 &upload,
6340 resource_layout(upload.bytes_per_row(), self.gradients.height),
6341 self.gradients.extent(),
6342 );
6343 }
6344 cache.mark_synced();
6345 }
6346
6347 // The strip program addresses all three resource textures by the
6348 // config's width shifts alone — the LUT by the paint texture's — so
6349 // the config built from `dim` is only sound while each of them really
6350 // is `dim` wide (see `gpu::config`'s module doc). Every resize above
6351 // grows height only, so this holds by construction; the assertion is
6352 // what notices a change that breaks it.
6353 debug_assert!(
6354 resource_widths_agree(
6355 dim,
6356 self.alphas.width,
6357 self.paints.width,
6358 self.gradients.width
6359 ),
6360 "strip resource textures must all be resource_texture_dim ({dim}) wide: \
6361 alphas {}, paints {}, gradients {}",
6362 self.alphas.width,
6363 self.paints.width,
6364 self.gradients.width,
6365 );
6366 let config = GpuConfig::new(u32::from(size.0), u32::from(size.1), dim, dim);
6367 let config = bytemuck::bytes_of(&config);
6368 let staged =
6369 stage.is_some_and(|stage| stage.copy_to_buffer(encoder, config, &self.config, 0));
6370 if !staged {
6371 queue.write_buffer(&self.config, 0, config);
6372 }
6373
6374 atlas_serviced
6375 }
6376
6377 /// Grows the instance buffer if this frame outgrew it, then uploads the
6378 /// opaque and alpha instances back to back — as one staged copy when the
6379 /// frame has a stage, two queue writes otherwise.
6380 fn upload_instances(
6381 &mut self,
6382 device: &wgpu::Device,
6383 queue: &wgpu::Queue,
6384 encoder: &mut wgpu::CommandEncoder,
6385 stage: Option<&mut StagingFrame>,
6386 scratch: &Scratch,
6387 ) {
6388 let (opaque, alpha) = scratch.instance_bytes();
6389 let required = (opaque.len() + alpha.len()) as u64;
6390 if required == 0 {
6391 return;
6392 }
6393
6394 let buffer = match &mut self.instances {
6395 Some(buffer) if self.instance_capacity >= required => buffer,
6396 slot => {
6397 let floor = MIN_INSTANCE_CAPACITY * size_of::<GpuStrip>() as u64;
6398 let capacity = required
6399 .checked_next_power_of_two()
6400 .unwrap_or(required)
6401 .max(floor);
6402 self.instance_capacity = capacity;
6403 slot.insert(device.create_buffer(&wgpu::BufferDescriptor {
6404 label: Some("frust-engine strip instances"),
6405 size: capacity,
6406 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
6407 mapped_at_creation: false,
6408 }))
6409 }
6410 };
6411 if let Some(stage) = stage
6412 && stage.copy_parts_to_buffer(encoder, &[opaque, alpha], buffer, 0)
6413 {
6414 return;
6415 }
6416 if !opaque.is_empty() {
6417 queue.write_buffer(buffer, 0, opaque);
6418 }
6419 if !alpha.is_empty() {
6420 queue.write_buffer(buffer, opaque.len() as u64, alpha);
6421 }
6422 }
6423
6424 /// Builds `variant`'s bind groups if it has none, first dropping every
6425 /// set when the target format changed under them.
6426 fn ensure_bind_groups(
6427 &mut self,
6428 device: &wgpu::Device,
6429 variant: EnginePipeline,
6430 pipeline: &wgpu::RenderPipeline,
6431 format: wgpu::TextureFormat,
6432 ) {
6433 if self.bind_group_format != Some(format) {
6434 self.bind_groups.clear();
6435 self.bind_group_format = Some(format);
6436 }
6437 if self.bind_groups.contains_key(&variant) {
6438 return;
6439 }
6440 let atlas_view = self
6441 .atlas
6442 .as_ref()
6443 .map(AtlasArray::view)
6444 .unwrap_or(&self.placeholders.atlas_array);
6445 let groups = StripBindGroups::new(
6446 device,
6447 pipeline,
6448 &self.alphas.view,
6449 &self.config,
6450 &self.placeholders,
6451 atlas_view,
6452 self.atlas_sampler.as_ref(),
6453 &self.paints.view,
6454 &self.gradients.view,
6455 );
6456 self.bind_groups.insert(variant, groups);
6457 self.groups_created = self.groups_created.saturating_add(4);
6458 }
6459
6460 /// Builds `variant`'s group 1 for every external texture this frame draws
6461 /// that it has none for yet.
6462 ///
6463 /// Called after [`Self::ensure_bind_groups`] has built the variant's own
6464 /// set, and only for the variants the frame records with, so a frame that
6465 /// draws no external texture builds nothing. What is built is retained
6466 /// alongside the rest of the variant's groups and dropped with them — on a
6467 /// format change, an atlas growth or a re-budget, all of which change the
6468 /// atlas view this group also holds.
6469 ///
6470 /// A texture with no registered view is skipped rather than substituted:
6471 /// its paint did not resolve either, so nothing in the frame names its
6472 /// slot.
6473 fn ensure_external_groups(
6474 &mut self,
6475 device: &wgpu::Device,
6476 variant: EnginePipeline,
6477 pipeline: &wgpu::RenderPipeline,
6478 externals: &ExternalTextures,
6479 ) {
6480 if self.external_runs.is_empty() {
6481 return;
6482 }
6483 // Destructured rather than reached through `self`: the group map is
6484 // borrowed mutably while the atlas view and the placeholders are read.
6485 let Self {
6486 bind_groups,
6487 external_runs,
6488 atlas,
6489 placeholders,
6490 atlas_sampler,
6491 groups_created,
6492 ..
6493 } = self;
6494 let atlas_view = atlas
6495 .as_ref()
6496 .map(AtlasArray::view)
6497 .unwrap_or(&placeholders.atlas_array);
6498 let Some(groups) = bind_groups.get_mut(&variant) else {
6499 return;
6500 };
6501 for key in external_runs.keys() {
6502 if groups.externals.contains_key(key) {
6503 continue;
6504 }
6505 let Some(view) = externals.view(*key) else {
6506 continue;
6507 };
6508 groups.externals.insert(
6509 *key,
6510 images_bind_group(device, pipeline, atlas_view, view, atlas_sampler.as_ref()),
6511 );
6512 *groups_created = groups_created.saturating_add(1);
6513 }
6514 }
6515
6516 /// Drops every bind group naming the texture registered under `key`.
6517 ///
6518 /// Called when that registration changes, because a group holds its view
6519 /// by value: keeping one past a re-bind would go on sampling the texture
6520 /// the caller replaced, and keeping one past an unbind would hold the
6521 /// caller's texture alive for as long as this renderer lives.
6522 fn forget_external(&mut self, key: u64) {
6523 for groups in self.bind_groups.values_mut() {
6524 groups.externals.remove(&key);
6525 }
6526 }
6527}
6528
6529/// Lower one atlas page's recorded commands into the strips that draw it,
6530/// answering whether the whole page could be expressed.
6531///
6532/// The caller-supplied half of the render-to-atlas seam: `gpu::atlas` owns the
6533/// pass, the orderings and the submit, while turning a command stream into
6534/// strips is compiler work and stays on this side of the edge — `compile`
6535/// already depends on `gpu::atlas`, so taking the reverse dependency would make
6536/// the two mutually recursive.
6537///
6538/// The stream is replayed as a scene in *page* space and compiled by
6539/// `lowering`, which is why that compiler is sized to the page rather than to
6540/// the surface. Only the four commands an outline glyph produces are lowered;
6541/// anything else — a clip path, a blend layer, a gradient paint, which is to
6542/// say every COLR shape — refuses the page whole rather than drawing part of
6543/// it. An indexed paint coming back out of the compile means the same thing:
6544/// the atlas pass binds no paint texture, so a record it would have to sample
6545/// cannot be drawn.
6546///
6547/// Refusing a page is a *last* line rather than the design, because refusal
6548/// cannot be made harmless here: `glifo` clears a recorder's commands whether
6549/// or not this answered `true`, and it offers no way to withdraw the entries
6550/// whose pixels those commands were going to be. Nothing that would reach this
6551/// refusal is therefore admitted to the atlas in the first place — a colour
6552/// face never takes the atlas route at all (`crate::text::atlas_policy`), so
6553/// what arrives here is the solid outline stream this lowers.
6554fn lower_atlas_page(
6555 recorder: &AtlasCommandRecorder,
6556 buffers: &mut AtlasPageBuffers,
6557 lowering: &mut SceneCompiler,
6558 page: (u16, u16),
6559) -> bool {
6560 let mut scene = Scene::new();
6561 {
6562 let mut builder = SceneBuilder::new(&mut scene);
6563 let mut transform = Affine::IDENTITY;
6564 let mut brush = Brush::Solid(Color::BLACK);
6565
6566 for command in &recorder.commands {
6567 match command {
6568 AtlasCommand::SetTransform(next) => transform = *next,
6569 AtlasCommand::SetPaint(AtlasPaint::Solid(color)) => brush = Brush::Solid(*color),
6570 AtlasCommand::FillPath(path) => {
6571 builder.push_transform(transform);
6572 builder.fill_path((**path).clone(), brush.clone());
6573 builder.pop_transform();
6574 }
6575 AtlasCommand::FillRect(rect) => {
6576 builder.push_transform(transform);
6577 builder.fill_rect(*rect, brush.clone());
6578 builder.pop_transform();
6579 }
6580 _ => return false,
6581 }
6582 }
6583 }
6584
6585 let Ok(frame) = lowering.compile(&scene, Affine::IDENTITY, page) else {
6586 return false;
6587 };
6588
6589 let strips = frame.strip_buf();
6590 for draw in frame.draws() {
6591 let Paint::Solid(color) = &draw.paint else {
6592 return false;
6593 };
6594 let Some(run) = strips.get(draw.strip_range.clone()) else {
6595 continue;
6596 };
6597 push_solid_strips(
6598 run,
6599 color.as_premul_rgba8().to_u32(),
6600 draw.depth,
6601 &mut buffers.instances,
6602 );
6603 }
6604 buffers.alphas.extend_from_slice(frame.alphas());
6605 true
6606}
6607
6608/// The [`ImageId`] an encoded image paint names, or `None` for the one
6609/// [`ImageSource`] variant no residency ever mints — the paint carrying its
6610/// pixels inline as a [`vello_common::pixmap::Pixmap`] rather than through a
6611/// handle. The compiler's own image encoding always produces the handle form
6612/// (see [`crate::compile::paint::encode_image`]), so a compiled frame never
6613/// exercises the `None` arm; it exists because the type itself admits both.
6614fn image_id(image: &EncodedImage) -> Option<ImageId> {
6615 match image.source {
6616 ImageSource::OpaqueId { id, .. } => Some(id),
6617 ImageSource::Pixmap(_) => None,
6618 }
6619}
6620
6621/// A lowered image record corrected for an atlas rectangle that holds a
6622/// *minified* copy of the source.
6623///
6624/// The record's transform maps a device position back onto the image's own
6625/// texels, and the compiler composed it against the source's declared extent —
6626/// it had to, since that is all it knows before residency is consulted. Scaling
6627/// its output by [`ResidentImage::minify_scale`] retargets it at the smaller
6628/// rectangle actually uploaded, which is the whole correction a downsampled
6629/// image needs: the record's `image_size` and `image_offset` already describe
6630/// the resident rectangle.
6631///
6632/// A record stored at full size is returned untouched, which is every image but
6633/// one larger than an atlas layer.
6634fn fit_minified(record: GpuEncodedPaint, resident: &ResidentImage) -> GpuEncodedPaint {
6635 let Some((x, y)) = resident.minify_scale() else {
6636 return record;
6637 };
6638 let GpuEncodedPaint::Image(mut image) = record else {
6639 return record;
6640 };
6641
6642 // `[a, b, c, d, tx, ty]`, mapping `(u, v)` to `(a·u + c·v + tx, b·u + d·v +
6643 // ty)`: the x row is scaled by one factor and the y row by the other.
6644 image.transform[0] *= x;
6645 image.transform[2] *= x;
6646 image.transform[4] *= x;
6647 image.transform[1] *= y;
6648 image.transform[3] *= y;
6649 image.transform[5] *= y;
6650
6651 GpuEncodedPaint::Image(image)
6652}
6653
6654/// The descriptor every `Config` uniform buffer is created with.
6655fn config_descriptor(label: &str) -> wgpu::BufferDescriptor<'_> {
6656 wgpu::BufferDescriptor {
6657 label: Some(label),
6658 size: GpuConfig::SIZE,
6659 usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,
6660 mapped_at_creation: false,
6661 }
6662}
6663
6664/// The rows a staged upload writes: the `filled` rows the frame's bytes take
6665/// and every row the last upload left `live`, never past `height` — see
6666/// [`ResourceTexture::live_rows`].
6667fn staged_rows(filled: u32, live: u32, height: u32) -> u32 {
6668 filled.max(live).min(height)
6669}
6670
6671/// The rows of a resource texture `len` bytes fill, the last one partly.
6672fn rows_filled(len: usize, bytes_per_row: u32) -> u32 {
6673 let rows = (len as u64).div_ceil(u64::from(bytes_per_row.max(1)));
6674 u32::try_from(rows).unwrap_or(u32::MAX)
6675}
6676
6677/// The texel copy layout of a full resource-texture upload.
6678fn resource_layout(bytes_per_row: u32, rows: u32) -> wgpu::TexelCopyBufferLayout {
6679 wgpu::TexelCopyBufferLayout {
6680 offset: 0,
6681 bytes_per_row: Some(bytes_per_row),
6682 rows_per_image: Some(rows),
6683 }
6684}
6685
6686/// The descriptor for a gradient LUT texture of `width` x `height` texels.
6687fn gradient_texture_descriptor(width: u32, height: u32) -> wgpu::TextureDescriptor<'static> {
6688 wgpu::TextureDescriptor {
6689 label: Some("frust-engine gradient texture"),
6690 size: wgpu::Extent3d {
6691 width,
6692 height,
6693 depth_or_array_layers: 1,
6694 },
6695 mip_level_count: 1,
6696 sample_count: 1,
6697 dimension: wgpu::TextureDimension::D2,
6698 format: GradientTextureLayout::FORMAT,
6699 usage: gpu::RESOURCE_TEXTURE_USAGES,
6700 view_formats: &[],
6701 }
6702}
6703
6704/// The 1x1 stand-ins for the strip shader's not-yet-written bindings.
6705#[derive(Debug)]
6706struct Placeholders {
6707 layer_input: wgpu::TextureView,
6708 atlas_array: wgpu::TextureView,
6709 external: wgpu::TextureView,
6710}
6711
6712impl Placeholders {
6713 fn new(device: &wgpu::Device) -> Self {
6714 Self {
6715 layer_input: placeholder_view(device, "frust-engine layer input placeholder", false),
6716 atlas_array: placeholder_view(device, "frust-engine atlas placeholder", true),
6717 external: placeholder_view(device, "frust-engine external placeholder", false),
6718 }
6719 }
6720}
6721
6722/// A 1x1 transparent `Rgba8Unorm` texture's view, as a plain 2D texture or as
6723/// a 2D array.
6724///
6725/// The array variant allocates **two** layers, not one, matching
6726/// [`gpu::atlas::atlas_texture_descriptor`]'s own floor: wgpu-hal 30.0.1's
6727/// GLES backend derives the GL target from the descriptor's layer count
6728/// alone, and a one-layer array descriptor binds as `GL_TEXTURE_2D` rather
6729/// than `GL_TEXTURE_2D_ARRAY` (see that function's doc comment for the
6730/// file:line and upstream issue refs). Only layer zero of the two is ever
6731/// sampled here.
6732fn placeholder_view(device: &wgpu::Device, label: &str, array: bool) -> wgpu::TextureView {
6733 let texture = device.create_texture(&wgpu::TextureDescriptor {
6734 label: Some(label),
6735 size: wgpu::Extent3d {
6736 width: 1,
6737 height: 1,
6738 depth_or_array_layers: if array { 2 } else { 1 },
6739 },
6740 mip_level_count: 1,
6741 sample_count: 1,
6742 dimension: wgpu::TextureDimension::D2,
6743 format: wgpu::TextureFormat::Rgba8Unorm,
6744 usage: wgpu::TextureUsages::TEXTURE_BINDING,
6745 view_formats: &[],
6746 });
6747 texture.create_view(&wgpu::TextureViewDescriptor {
6748 label: Some(label),
6749 dimension: Some(if array {
6750 wgpu::TextureViewDimension::D2Array
6751 } else {
6752 wgpu::TextureViewDimension::D2
6753 }),
6754 ..Default::default()
6755 })
6756}
6757
6758/// The four bind groups every strip pass sets.
6759///
6760/// The grouping is the shader's, not this module's: coverage plus config plus
6761/// layer input, atlas array plus external texture, encoded paints, gradient
6762/// ramps — four groups exactly, which is the downlevel ceiling with no
6763/// headroom left.
6764#[derive(Debug)]
6765struct StripBindGroups {
6766 resources: wgpu::BindGroup,
6767 images: wgpu::BindGroup,
6768 paints: wgpu::BindGroup,
6769 gradients: wgpu::BindGroup,
6770 /// Group 1 again, once per externally bound texture this variant has
6771 /// drawn: the same atlas array beside that texture's view instead of the
6772 /// placeholder. Keyed by the id a display list names the texture by.
6773 ///
6774 /// A second group rather than a fifth: the four-group ceiling has no
6775 /// headroom (see [`crate::gpu::pipelines`]), so an external texture is
6776 /// bound by re-setting the group the atlas already occupies — which is why
6777 /// a pass's instances are split into runs at all.
6778 externals: HashMap<u64, wgpu::BindGroup>,
6779}
6780
6781impl StripBindGroups {
6782 /// Takes the eight resources one by one rather than a `&FrameResources`
6783 /// so the caller can build a set while holding the map it lands in
6784 /// mutably.
6785 ///
6786 /// `atlas` is the live [`AtlasArray`] view once the renderer has one, or
6787 /// [`Placeholders::atlas_array`] until then — the caller picks, since
6788 /// only it knows which the frame's own `atlas_layers` calls for. `sampler`
6789 /// is [`FrameResources::atlas_sampler`].
6790 #[expect(
6791 clippy::too_many_arguments,
6792 reason = "one bind-group build's full resource list; a struct would \
6793 only rename the same borrows the caller already holds \
6794 mutably in `FrameResources`"
6795 )]
6796 fn new(
6797 device: &wgpu::Device,
6798 pipeline: &wgpu::RenderPipeline,
6799 alphas: &wgpu::TextureView,
6800 config: &wgpu::Buffer,
6801 placeholders: &Placeholders,
6802 atlas: &wgpu::TextureView,
6803 sampler: Option<&wgpu::Sampler>,
6804 paints: &wgpu::TextureView,
6805 gradients: &wgpu::TextureView,
6806 ) -> Self {
6807 let resources_group =
6808 resources_bind_group(device, pipeline, alphas, config, &placeholders.layer_input);
6809 let images = images_bind_group(device, pipeline, atlas, &placeholders.external, sampler);
6810 let paints = device.create_bind_group(&wgpu::BindGroupDescriptor {
6811 label: Some("frust-engine strip paints"),
6812 layout: &pipeline.get_bind_group_layout(2),
6813 entries: &[wgpu::BindGroupEntry {
6814 binding: 0,
6815 resource: wgpu::BindingResource::TextureView(paints),
6816 }],
6817 });
6818 let gradients = device.create_bind_group(&wgpu::BindGroupDescriptor {
6819 label: Some("frust-engine strip gradients"),
6820 layout: &pipeline.get_bind_group_layout(3),
6821 entries: &[wgpu::BindGroupEntry {
6822 binding: 0,
6823 resource: wgpu::BindingResource::TextureView(gradients),
6824 }],
6825 });
6826
6827 Self {
6828 resources: resources_group,
6829 images,
6830 paints,
6831 gradients,
6832 externals: HashMap::new(),
6833 }
6834 }
6835
6836 /// Sets all four groups on `pass`, taking group 0 from `resources` and
6837 /// group 1 from `images` when a segment names one, rather than from this
6838 /// set.
6839 ///
6840 /// Two of the four vary within a pass. Group 0 carries both the pass's
6841 /// viewport uniform and the layer input a composite samples, so a page
6842 /// round and every composite in it substitute their own. Group 1 carries
6843 /// the external texture a run is drawn with, so a segment that samples one
6844 /// substitutes the group holding it; every other segment takes this set's
6845 /// own, which pairs the atlas array with a placeholder nothing reads.
6846 /// Groups 2-3 are frame-wide and belong to the pipeline variant.
6847 fn bind_with(
6848 &self,
6849 pass: &mut wgpu::RenderPass<'_>,
6850 resources: &wgpu::BindGroup,
6851 images: Option<&wgpu::BindGroup>,
6852 ) {
6853 pass.set_bind_group(0, resources, &[]);
6854 pass.set_bind_group(1, images.unwrap_or(&self.images), &[]);
6855 pass.set_bind_group(2, &self.paints, &[]);
6856 pass.set_bind_group(3, &self.gradients, &[]);
6857 }
6858}
6859
6860/// Group 1 of a strip pass: the image atlas array, the caller-owned texture
6861/// an external image paint samples, and — in the image-sampler program family
6862/// only — the sampler the atlas array is filtered through.
6863///
6864/// One function for both shapes the group takes — `external` is
6865/// [`Placeholders::external`] for the frame-wide group, or a registered view
6866/// for the group a run of external instances is drawn with (see
6867/// [`crate::gpu::bindings`]). Built against one pipeline's group layout, as
6868/// [`resources_bind_group`] is; `sampler` is present exactly when that layout
6869/// declares one, which [`FrameResources::atlas_sampler`] already decides.
6870fn images_bind_group(
6871 device: &wgpu::Device,
6872 pipeline: &wgpu::RenderPipeline,
6873 atlas: &wgpu::TextureView,
6874 external: &wgpu::TextureView,
6875 sampler: Option<&wgpu::Sampler>,
6876) -> wgpu::BindGroup {
6877 let textures = [
6878 wgpu::BindGroupEntry {
6879 binding: 0,
6880 resource: wgpu::BindingResource::TextureView(atlas),
6881 },
6882 wgpu::BindGroupEntry {
6883 binding: 1,
6884 resource: wgpu::BindingResource::TextureView(external),
6885 },
6886 ];
6887 let with_sampler;
6888 let entries: &[wgpu::BindGroupEntry<'_>] = match sampler {
6889 Some(sampler) => {
6890 let [atlas, external] = textures;
6891 with_sampler = [
6892 atlas,
6893 external,
6894 wgpu::BindGroupEntry {
6895 binding: ATLAS_SAMPLER_BINDING,
6896 resource: wgpu::BindingResource::Sampler(sampler),
6897 },
6898 ];
6899 &with_sampler
6900 }
6901 None => &textures,
6902 };
6903 device.create_bind_group(&wgpu::BindGroupDescriptor {
6904 label: Some("frust-engine strip images"),
6905 layout: &pipeline.get_bind_group_layout(1),
6906 entries,
6907 })
6908}
6909
6910/// The sampler the image-sampler program family filters bilinear atlas
6911/// images through: linear in both directions, clamped to the edge, one mip.
6912///
6913/// Clamp-to-edge is a backstop rather than the mechanism: the shader already
6914/// clamps every sample position to the centres of the image's own edge texels
6915/// (`shaders/helpers.wgsl`'s `sampled_bilinear`), so the filter never reaches
6916/// past the image's rectangle in the first place.
6917fn atlas_sampler(device: &wgpu::Device) -> wgpu::Sampler {
6918 device.create_sampler(&wgpu::SamplerDescriptor {
6919 label: Some("frust-engine atlas sampler"),
6920 address_mode_u: wgpu::AddressMode::ClampToEdge,
6921 address_mode_v: wgpu::AddressMode::ClampToEdge,
6922 address_mode_w: wgpu::AddressMode::ClampToEdge,
6923 mag_filter: wgpu::FilterMode::Linear,
6924 min_filter: wgpu::FilterMode::Linear,
6925 mipmap_filter: wgpu::MipmapFilterMode::Nearest,
6926 ..Default::default()
6927 })
6928}
6929
6930/// The groups a round's `released` list names, as the per-group flags a
6931/// [`RoundPlan`] carries.
6932fn released_groups(released: &[PageParity]) -> [bool; MAX_LIVE_PAGES] {
6933 let mut groups = [false; MAX_LIVE_PAGES];
6934 for parity in released {
6935 if let Some(slot) = groups.get_mut(parity.index()) {
6936 *slot = true;
6937 }
6938 }
6939 groups
6940}
6941
6942/// How a region of a frame's own target is read back into a pooled page —
6943/// the mechanism every effect sampling what the frame has already painted
6944/// shares (a backdrop blur today; see [`crate::filters`]' *The backdrop
6945/// blur*).
6946///
6947/// Either way the region reaches the page through one pass of
6948/// [`EnginePipeline::Copy`] that clears the page and draws the region at its
6949/// origin: a pooled page is a render target and a sampled texture, never a
6950/// copy destination, so a texture copy cannot land in one directly. The modes
6951/// differ only in what that pass reads.
6952///
6953/// Decided per frame from the target texture itself, because the caller
6954/// decides what it renders into: a swapchain texture is `RENDER_ATTACHMENT`
6955/// only on this tier and is never widened, so `frust-render` renders a frame
6956/// carrying a backdrop into an intermediate of its own (in the swapchain's own
6957/// format, sampled and copyable) and presents it with one copy pass.
6958#[derive(Debug, Clone, Copy, PartialEq, Eq)]
6959pub enum RegionCopyMode {
6960 /// The target is `COPY_SRC` in the pages' own [`INTERMEDIATE_FORMAT`]: a
6961 /// `copy_texture_to_texture` lands the region in the renderer's own
6962 /// staging texture, unconverted, and the copy pass reads it from there —
6963 /// the route a target that cannot be sampled (a headless readback target)
6964 /// still has.
6965 Texture,
6966 /// The target is `TEXTURE_BINDING`: the copy pass reads the region
6967 /// straight out of it — any format the target is rendered in, converted on
6968 /// the way into the page's own.
6969 Fragment,
6970 /// The target can be neither copied nor sampled: no region is read back,
6971 /// and a backdrop blur draws nothing.
6972 Unavailable,
6973}
6974
6975impl RegionCopyMode {
6976 /// The mode a frame rendering into `target` reads regions back with.
6977 #[must_use]
6978 pub fn of(target: &wgpu::Texture) -> Self {
6979 Self::for_usage(target.usage(), target.format(), target.sample_count())
6980 }
6981
6982 /// [`Self::of`] over the three facts it reads, so the decision is
6983 /// host-testable without a device.
6984 ///
6985 /// Sampling is preferred whenever the target allows it — it costs the one
6986 /// pass alone, where a texture copy costs a transfer as well.
6987 #[must_use]
6988 pub fn for_usage(
6989 usage: wgpu::TextureUsages,
6990 format: wgpu::TextureFormat,
6991 sample_count: u32,
6992 ) -> Self {
6993 if sample_count != 1 {
6994 Self::Unavailable
6995 } else if usage.contains(wgpu::TextureUsages::TEXTURE_BINDING) {
6996 Self::Fragment
6997 } else if format == INTERMEDIATE_FORMAT && usage.contains(wgpu::TextureUsages::COPY_SRC) {
6998 Self::Texture
6999 } else {
7000 Self::Unavailable
7001 }
7002 }
7003}
7004
7005/// Reports a frame whose backdrop blurs are drawn as nothing because `target`
7006/// can be neither copied nor sampled, once per process.
7007fn note_backdrop_unreadable(target: &wgpu::Texture) {
7008 BACKDROP_UNREADABLE_WARNING.call_once(|| {
7009 log::warn!(
7010 "backdrop blur drawn as nothing: the frame's target ({:?}, {:?}) is neither \
7011 TEXTURE_BINDING nor COPY_SRC in {INTERMEDIATE_FORMAT:?}, so its backdrop cannot be \
7012 read back (logged once)",
7013 target.format(),
7014 target.usage()
7015 );
7016 });
7017}
7018
7019/// What a frame's region copies are recorded with beyond the copy pipeline:
7020/// the instance buffer — one 16-byte slot per copy a frame can run
7021/// ([`crate::filters::MAX_BACKDROPS_PER_FRAME`]), written as each copy is
7022/// recorded — and, for a [`RegionCopyMode::Texture`] frame, the staging
7023/// texture a region is transferred into before the copy pass reads it.
7024#[derive(Debug)]
7025struct RegionCopyResources {
7026 instances: wgpu::Buffer,
7027 /// `COPY_DST | TEXTURE_BINDING` in [`INTERMEDIATE_FORMAT`], grown to the
7028 /// largest region a frame has copied and kept, like the page pool's
7029 /// textures, for the renderer's life. Created by the first texture-mode
7030 /// frame.
7031 staging: Option<ResourceTexture>,
7032 /// Group 0 of the copy pass naming [`Self::staging`]'s view, built on the
7033 /// first texture-mode copy after the staging texture was (re)created.
7034 staging_group: Option<wgpu::BindGroup>,
7035}
7036
7037impl RegionCopyResources {
7038 fn new(device: &wgpu::Device) -> Self {
7039 let slots = crate::filters::MAX_BACKDROPS_PER_FRAME as u64;
7040 Self {
7041 instances: device.create_buffer(&wgpu::BufferDescriptor {
7042 label: Some("frust-engine region copy instances"),
7043 size: slots.saturating_mul(COPY_INSTANCE_BYTES),
7044 usage: wgpu::BufferUsages::VERTEX | wgpu::BufferUsages::COPY_DST,
7045 mapped_at_creation: false,
7046 }),
7047 staging: None,
7048 staging_group: None,
7049 }
7050 }
7051
7052 /// Grows the staging texture to hold a `width` x `height` region, if it
7053 /// does not already.
7054 fn ensure_staging(&mut self, device: &wgpu::Device, width: u32, height: u32) {
7055 if width == 0 || height == 0 {
7056 return;
7057 }
7058 let (held_width, held_height) = self
7059 .staging
7060 .as_ref()
7061 .map_or((0, 0), |staging| (staging.width, staging.height));
7062 if held_width >= width && held_height >= height {
7063 return;
7064 }
7065 self.staging_group = None;
7066 self.staging = Some(ResourceTexture::new(
7067 device,
7068 &wgpu::TextureDescriptor {
7069 label: Some("frust-engine region copy staging"),
7070 size: wgpu::Extent3d {
7071 width: width.max(held_width),
7072 height: height.max(held_height),
7073 depth_or_array_layers: 1,
7074 },
7075 mip_level_count: 1,
7076 sample_count: 1,
7077 dimension: wgpu::TextureDimension::D2,
7078 format: INTERMEDIATE_FORMAT,
7079 usage: wgpu::TextureUsages::COPY_DST | wgpu::TextureUsages::TEXTURE_BINDING,
7080 view_formats: &[],
7081 },
7082 ));
7083 }
7084}
7085
7086/// One region copy's full recording state: a region of the frame's own
7087/// target, read back into a pooled page at the page's origin.
7088struct RegionCopyPass<'a> {
7089 mode: RegionCopyMode,
7090 /// The frame's own target.
7091 target: &'a wgpu::TextureView,
7092 /// The page the region lands in.
7093 page: &'a PooledTexture,
7094 /// The device region copied — inside the target by construction
7095 /// ([`crate::filters::backdrop_region`] clamps it).
7096 source: RectU16,
7097 /// [`EnginePipeline::Copy`], on every frame that copies.
7098 pipeline: Option<&'a wgpu::RenderPipeline>,
7099 /// The copies' instance buffer and staging texture, on the same frames.
7100 resources: Option<&'a RegionCopyResources>,
7101 /// This copy's instance slot.
7102 slot: u32,
7103}
7104
7105impl RegionCopyPass<'_> {
7106 /// Records the copy into `encoder`: on a texture-mode frame the region is
7107 /// transferred into the staging texture first, then one pass clears the
7108 /// page and draws the region into it at the page's origin.
7109 ///
7110 /// The page holds the region and transparency around it. A copy whose
7111 /// pipeline, instance buffer or staging texture is missing — which cannot
7112 /// happen, `encode` resolves all three for every frame that copies — still
7113 /// clears the page, and the backdrop blurs nothing into nothing: "draw
7114 /// less, never wrong".
7115 fn record(
7116 &self,
7117 device: &wgpu::Device,
7118 queue: &wgpu::Queue,
7119 encoder: &mut wgpu::CommandEncoder,
7120 timestamps: Option<wgpu::RenderPassTimestampWrites<'_>>,
7121 ) -> u64 {
7122 let width = self.source.width();
7123 let height = self.source.height();
7124 // The texture the pass reads, and where in it the region starts.
7125 let source = match (self.mode, self.resources) {
7126 (RegionCopyMode::Fragment, Some(_)) => {
7127 Some((self.target, (self.source.x0, self.source.y0)))
7128 }
7129 (RegionCopyMode::Texture, Some(resources)) => match resources.staging.as_ref() {
7130 Some(staging)
7131 if staging.width >= u32::from(width) && staging.height >= u32::from(height) =>
7132 {
7133 encoder.copy_texture_to_texture(
7134 wgpu::TexelCopyTextureInfo {
7135 texture: self.target.texture(),
7136 mip_level: 0,
7137 origin: wgpu::Origin3d {
7138 x: u32::from(self.source.x0),
7139 y: u32::from(self.source.y0),
7140 z: 0,
7141 },
7142 aspect: wgpu::TextureAspect::All,
7143 },
7144 staging.copy_target(),
7145 wgpu::Extent3d {
7146 width: u32::from(width),
7147 height: u32::from(height),
7148 depth_or_array_layers: 1,
7149 },
7150 );
7151 Some((&staging.view, (0, 0)))
7152 }
7153 _ => None,
7154 },
7155 _ => None,
7156 };
7157
7158 let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
7159 label: Some(FramePass::Copy.label()),
7160 color_attachments: &[Some(color_attachment(
7161 self.page.view(),
7162 wgpu::LoadOp::Clear(wgpu::Color::TRANSPARENT),
7163 ))],
7164 depth_stencil_attachment: None,
7165 timestamp_writes: timestamps,
7166 occlusion_query_set: None,
7167 multiview_mask: None,
7168 });
7169 let (Some((view, origin)), Some(pipeline), Some(resources)) =
7170 (source, self.pipeline, self.resources)
7171 else {
7172 return 0;
7173 };
7174 let offset = u64::from(self.slot).saturating_mul(COPY_INSTANCE_BYTES);
7175 if offset.saturating_add(COPY_INSTANCE_BYTES) > resources.instances.size() {
7176 return 0;
7177 }
7178 let (page_width, page_height) = self.page.size();
7179 let instance = [
7180 pack_u16_pair(0, 0),
7181 pack_u16_pair(origin.0, origin.1),
7182 pack_u16_pair(width, height),
7183 pack_u16_pair(
7184 u16::try_from(page_width).unwrap_or(u16::MAX),
7185 u16::try_from(page_height).unwrap_or(u16::MAX),
7186 ),
7187 ];
7188 queue.write_buffer(
7189 &resources.instances,
7190 offset,
7191 bytemuck::cast_slice(&instance),
7192 );
7193 // The staging texture's group is cached; the frame's own target is
7194 // the caller's and changes with the swapchain image, so a fragment-mode
7195 // copy builds its group per copy.
7196 let built;
7197 let (bind_group, created) = match (self.mode, resources.staging_group.as_ref()) {
7198 (RegionCopyMode::Texture, Some(group)) => (group, 0),
7199 _ => {
7200 built = copy_source_group(device, pipeline, view);
7201 (&built, 1)
7202 }
7203 };
7204 pass.set_pipeline(pipeline);
7205 pass.set_vertex_buffer(0, resources.instances.slice(..));
7206 pass.set_bind_group(0, bind_group, &[]);
7207 pass.draw(
7208 0..COPY_QUAD_VERTICES,
7209 self.slot..self.slot.saturating_add(1),
7210 );
7211 created
7212 }
7213}
7214
7215/// Group 0 of the region copy pass: the texture it reads the region from.
7216fn copy_source_group(
7217 device: &wgpu::Device,
7218 pipeline: &wgpu::RenderPipeline,
7219 view: &wgpu::TextureView,
7220) -> wgpu::BindGroup {
7221 device.create_bind_group(&wgpu::BindGroupDescriptor {
7222 label: Some("frust-engine region copy source"),
7223 layout: &pipeline.get_bind_group_layout(0),
7224 entries: &[wgpu::BindGroupEntry {
7225 binding: 0,
7226 resource: wgpu::BindingResource::TextureView(view),
7227 }],
7228 })
7229}
7230
7231/// Group 0 of a strip pass: the frame's coverage, the pass's own viewport
7232/// uniform, and the texture a composite reads its finished page from.
7233///
7234/// Built against one pipeline's group layout. Every strip pipeline names the
7235/// one explicit strip layout ([`crate::gpu::bindings::strip_pipeline_layout`]),
7236/// so the group binds under every strip state and paint class alike — which is
7237/// what lets one pass switch class pipelines between runs without rebinding.
7238fn resources_bind_group(
7239 device: &wgpu::Device,
7240 pipeline: &wgpu::RenderPipeline,
7241 alphas: &wgpu::TextureView,
7242 config: &wgpu::Buffer,
7243 layer_input: &wgpu::TextureView,
7244) -> wgpu::BindGroup {
7245 device.create_bind_group(&wgpu::BindGroupDescriptor {
7246 label: Some("frust-engine strip resources"),
7247 layout: &pipeline.get_bind_group_layout(0),
7248 entries: &[
7249 wgpu::BindGroupEntry {
7250 binding: 0,
7251 resource: wgpu::BindingResource::TextureView(alphas),
7252 },
7253 wgpu::BindGroupEntry {
7254 binding: 1,
7255 resource: config.as_entire_binding(),
7256 },
7257 wgpu::BindGroupEntry {
7258 binding: 2,
7259 resource: wgpu::BindingResource::TextureView(layer_input),
7260 },
7261 ],
7262 })
7263}
7264
7265// ---------------------------------------------------------------------------
7266// Command ranges (render-model Phase 4 overlay spike)
7267// ---------------------------------------------------------------------------
7268
7269/// How [`project_command_range`] treats the commands it is asked to carry
7270/// into a range — the one thing that differs between the renderer a range is
7271/// projected for.
7272#[derive(Debug, Clone, Copy, PartialEq, Eq)]
7273pub struct RangeOptions {
7274 /// Keep a `DrawRetained` inside the range. A renderer with a retained
7275 /// store behind it keeps them; one without (an overlay renderer built with
7276 /// retention off, whose compiler refuses a frame carrying one) has them
7277 /// dropped and counted in [`RangeCut::dropped_draw_retained`] instead.
7278 pub keep_draw_retained: bool,
7279}
7280
7281/// What cutting a command range out of a scene did — the bracket rule of the
7282/// overlay spike, reported so a caller (and a test) can see what a cut cost.
7283///
7284/// See [`project_command_range`] for the rule itself.
7285#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
7286pub struct RangeCut {
7287 /// The first command index the projection carried, after clamping.
7288 pub start: usize,
7289 /// One past the last command index the projection carried, after
7290 /// clamping.
7291 pub end: usize,
7292 /// Brackets open at `start` that were reopened ahead of the range.
7293 pub reopened: usize,
7294 /// Brackets still open at `end` that were closed after the range.
7295 pub closed: usize,
7296 /// `PushRetained` brackets spanning a cut, painted inline (their
7297 /// presentation scale folded into the body's transforms and their alpha
7298 /// as a plain layer) rather than recorded as a page.
7299 pub demoted_retained: usize,
7300 /// `DrawRetained`s dropped under `keep_draw_retained: false`.
7301 pub dropped_draw_retained: usize,
7302 /// Pops inside the range with no open bracket of their own kind — already
7303 /// ignored by every renderer, so dropped rather than carried.
7304 pub dropped_pops: usize,
7305}
7306
7307/// The bracket families the display list closes with a pop of their own.
7308#[derive(Debug, Clone, Copy, PartialEq, Eq)]
7309enum BracketKind {
7310 Clip,
7311 Layer,
7312 Retained,
7313}
7314
7315/// The bracket kind `command` opens, if it opens one.
7316fn opens(command: &frust_scene::Command) -> Option<BracketKind> {
7317 use frust_scene::Command;
7318 match command {
7319 Command::PushClip { .. }
7320 | Command::PushClipRounded { .. }
7321 | Command::PushClipPath { .. } => Some(BracketKind::Clip),
7322 Command::PushLayer { .. } => Some(BracketKind::Layer),
7323 Command::PushRetained { .. } => Some(BracketKind::Retained),
7324 _ => None,
7325 }
7326}
7327
7328/// The bracket kind `command` closes, if it is a pop.
7329fn closes(command: &frust_scene::Command) -> Option<BracketKind> {
7330 use frust_scene::Command;
7331 match command {
7332 Command::PopClip => Some(BracketKind::Clip),
7333 Command::PopLayer => Some(BracketKind::Layer),
7334 Command::PopRetained => Some(BracketKind::Retained),
7335 _ => None,
7336 }
7337}
7338
7339/// One bracket the projection holds open, and how it will be closed.
7340#[derive(Debug, Clone, Copy)]
7341enum OpenBracket {
7342 /// A clip or layer carried as recorded; closed with its own pop.
7343 Plain(BracketKind),
7344 /// A `PushRetained` carried as recorded; closed with `PopRetained`.
7345 Retained,
7346 /// A `PushRetained` spanning a cut, painted inline. `layer` says whether
7347 /// its alpha opened a layer to close; `prefix` is the correction in force
7348 /// before it, restored when it closes.
7349 Demoted { layer: bool, prefix: Affine },
7350 /// A `PushRetained` nested inside a demoted bracket: its presentation is
7351 /// ignored (only the outermost bracket's is honoured), so it opened
7352 /// nothing and closes nothing.
7353 Absorbed,
7354}
7355
7356impl OpenBracket {
7357 fn kind(self) -> BracketKind {
7358 match self {
7359 OpenBracket::Plain(kind) => kind,
7360 OpenBracket::Retained | OpenBracket::Demoted { .. } | OpenBracket::Absorbed => {
7361 BracketKind::Retained
7362 }
7363 }
7364 }
7365}
7366
7367/// Records `command` into `builder` with `prefix` composed ahead of its own
7368/// transform — the identity outside a demoted retained bracket, its
7369/// presentation correction inside one.
7370///
7371/// The builder's own transform stack is left as found: every arm pushes the
7372/// command's transform, records, and pops, so the recorded transform is
7373/// exactly `prefix * transform`.
7374fn replay(builder: &mut SceneBuilder<'_>, prefix: Affine, command: &frust_scene::Command) {
7375 use frust_scene::{Command, PathStyle};
7376 fn under(
7377 builder: &mut SceneBuilder<'_>,
7378 transform: Affine,
7379 record: impl FnOnce(&mut SceneBuilder<'_>),
7380 ) {
7381 builder.push_transform(transform);
7382 record(builder);
7383 builder.pop_transform();
7384 }
7385 match command {
7386 Command::FillRect {
7387 rect,
7388 brush,
7389 transform,
7390 } => under(builder, prefix * *transform, |b| {
7391 b.fill_rect(*rect, brush.clone())
7392 }),
7393 Command::RoundedRect {
7394 rect,
7395 radii,
7396 brush,
7397 transform,
7398 } => under(builder, prefix * *transform, |b| {
7399 b.fill_rounded_rect_radii(*rect, *radii, brush.clone());
7400 }),
7401 Command::Line {
7402 p0,
7403 p1,
7404 width,
7405 brush,
7406 transform,
7407 } => under(builder, prefix * *transform, |b| {
7408 b.stroke_line(*p0, *p1, *width, brush.clone());
7409 }),
7410 // A glyph run composes the builder's transform ahead of its own.
7411 Command::GlyphRun(run) => under(builder, prefix, |b| b.draw_glyph_run(run.clone())),
7412 Command::PushClip { rect, transform } => {
7413 under(builder, prefix * *transform, |b| b.push_clip(*rect));
7414 }
7415 Command::PushClipRounded {
7416 rect,
7417 radii,
7418 transform,
7419 } => under(builder, prefix * *transform, |b| {
7420 b.push_clip_rounded_radii(*rect, *radii);
7421 }),
7422 Command::PushClipPath { path, transform } => {
7423 under(builder, prefix * *transform, |b| {
7424 b.push_clip_path(path.clone())
7425 });
7426 }
7427 Command::PopClip => builder.pop_clip(),
7428 Command::Image {
7429 data,
7430 dest,
7431 transform,
7432 } => under(builder, prefix * *transform, |b| b.draw_image(data, *dest)),
7433 Command::BlurredRoundedRect {
7434 rect,
7435 radii,
7436 std_dev,
7437 color,
7438 transform,
7439 } => under(builder, prefix * *transform, |b| {
7440 b.draw_blurred_rounded_rect_radii(*rect, *radii, *std_dev, *color);
7441 }),
7442 Command::PushLayer {
7443 rect,
7444 alpha,
7445 transform,
7446 } => under(builder, prefix * *transform, |b| {
7447 b.push_layer(*rect, *alpha)
7448 }),
7449 Command::PopLayer => builder.pop_layer(),
7450 Command::ClearRect { rect, transform } => {
7451 under(builder, prefix * *transform, |b| b.clear_rect(*rect));
7452 }
7453 Command::Path {
7454 path,
7455 style,
7456 brush,
7457 transform,
7458 } => under(builder, prefix * *transform, |b| match style {
7459 PathStyle::Fill => b.fill_path(path.clone(), brush.clone()),
7460 PathStyle::Stroke { width, dash: None } => {
7461 b.stroke_path(path.clone(), *width, brush.clone());
7462 }
7463 PathStyle::Stroke {
7464 width,
7465 dash: Some(dash),
7466 } => b.stroke_path_dashed(path.clone(), *width, *dash, brush.clone()),
7467 }),
7468 Command::ShaderQuad {
7469 program,
7470 dest,
7471 transform,
7472 time,
7473 } => under(builder, prefix * *transform, |b| {
7474 b.draw_shader(program, *dest, *time);
7475 }),
7476 Command::PushRetained {
7477 key,
7478 rect,
7479 alpha,
7480 scale,
7481 transform,
7482 } => under(builder, prefix * *transform, |b| {
7483 b.push_retained(*key, *rect, *alpha, *scale);
7484 }),
7485 Command::PopRetained => builder.pop_retained(),
7486 Command::DrawRetained {
7487 key,
7488 rect,
7489 transform,
7490 alpha,
7491 scale,
7492 } => under(builder, prefix * *transform, |b| {
7493 b.draw_retained(*key, *rect, *alpha, *scale);
7494 }),
7495 Command::SceneTexture {
7496 id,
7497 dest,
7498 transform,
7499 } => {
7500 under(builder, prefix * *transform, |b| {
7501 b.scene_texture(*id, *dest)
7502 });
7503 }
7504 Command::BackdropBlur {
7505 rect,
7506 radius,
7507 transform,
7508 } => under(builder, prefix * *transform, |b| {
7509 b.backdrop_blur(*rect, *radius)
7510 }),
7511 }
7512}
7513
7514/// Writes the commands of `scene` in `range` into `out` (reset first) as a
7515/// self-contained display list, and reports what the cut did — the overlay
7516/// spike's range projection (render-model Phase 4, decision D10).
7517///
7518/// `range` is clamped to the scene: an `end` past the last command stops at
7519/// it, and a `start` past `end` projects nothing. The projection is a scene,
7520/// not a compiled-frame filter: the compiled frame's draws are ordered by
7521/// depth and grouped into rounds, while a command index is the one paint-order
7522/// position every producer of a cut (a platform view's place in the display
7523/// list) can name.
7524///
7525/// # The bracket rule
7526///
7527/// A cut can fall inside a bracket. The rule this spike chose:
7528///
7529/// - **Clips and layers spanning a cut are closed at it and reopened across
7530/// it.** Every clip or layer open at `start` is re-recorded, in its original
7531/// push order and with its original geometry, ahead of the range; every one
7532/// still open at `end` is popped after it, innermost first. Both halves of a
7533/// split clip therefore clip to the same shape, and both halves of a split
7534/// layer composite at the same alpha — which is exact for a clip, and for a
7535/// layer differs from the uncut frame only where content on both sides of
7536/// the cut overlaps inside it (each half is composited on its own).
7537/// - **A retained bracket spanning a cut is demoted to inline paint on both
7538/// sides.** A page recorded from half a body would be a stale page the
7539/// ledger then advertises, so neither side records one: the bracket's
7540/// presentation scale is folded into every body command's transform (the
7541/// same conjugation `crate::compile::layers::retained_correction` composes
7542/// ahead of the frame root) and its alpha below one becomes a plain layer —
7543/// exactly the lowering `frust_scene` prescribes for a renderer that does
7544/// not retain. A retained bracket nested inside a demoted one contributes
7545/// nothing, as a nested bracket's presentation never does. A retained
7546/// bracket wholly inside the range is carried as recorded.
7547/// - **A pop with no open bracket of its kind is dropped** (every renderer
7548/// ignores one anyway), and a `DrawRetained` is dropped when
7549/// `options.keep_draw_retained` is off.
7550///
7551/// Pops match the most recently opened bracket of their own kind, which is
7552/// the stack discipline of every well-nested display list.
7553///
7554/// Every carried command is re-recorded through a [`SceneBuilder`] with its
7555/// transform composed under the identity (or a demoted bracket's correction),
7556/// so geometry and paints are carried verbatim; the cost is one clone of each
7557/// carried command's payload, paid only by a caller that projects.
7558pub fn project_command_range(
7559 scene: &Scene,
7560 range: Range<usize>,
7561 options: RangeOptions,
7562 out: &mut Scene,
7563) -> RangeCut {
7564 project_command_ranges(scene, std::slice::from_ref(&range), options, out)
7565}
7566
7567/// [`project_command_range`] over several ranges at once: writes the
7568/// projection of each range of `ranges`, in the order given, one after the
7569/// other into `out` (reset first), and reports the cuts summed — the segmented
7570/// frame's way of drawing the commands *between* overlay cuts on the main
7571/// surface (render-model Phase 4, decision D10).
7572///
7573/// Each range is projected exactly as [`project_command_range`] projects it
7574/// alone — clamped to the scene, its open clips and layers reopened ahead of
7575/// it and closed after it, a retained bracket spanning either of its ends
7576/// demoted to inline paint — so every range is self-contained and the bracket
7577/// rule holds per range. A clip open across the gap between two ranges is
7578/// therefore closed at the end of the first and reopened at the start of the
7579/// second, never left open over the commands the gap removed.
7580///
7581/// The caller passes the ranges ascending and disjoint (the complement of a
7582/// frame's overlay cuts is); overlapping ranges are projected as given, each
7583/// command carried once per range it falls in. An empty range still reopens
7584/// and closes the brackets open at its start (drawing nothing), so a caller
7585/// that wants no trace of it leaves it out.
7586///
7587/// The returned [`RangeCut`]'s `start` is the first range's clamped start and
7588/// its `end` the last range's clamped end (both `0` for no range at all); its
7589/// counters are the sums over every range.
7590pub fn project_command_ranges(
7591 scene: &Scene,
7592 ranges: &[Range<usize>],
7593 options: RangeOptions,
7594 out: &mut Scene,
7595) -> RangeCut {
7596 let commands = scene.commands();
7597 let len = commands.len();
7598 let clamped: Vec<(usize, usize)> = ranges
7599 .iter()
7600 .map(|range| {
7601 let end = range.end.min(len);
7602 (range.start.min(end), end)
7603 })
7604 .collect();
7605 let mut cut = RangeCut {
7606 start: clamped.first().map_or(0, |&(start, _)| start),
7607 end: clamped.last().map_or(0, |&(_, end)| end),
7608 ..RangeCut::default()
7609 };
7610
7611 // Pass one, over the whole scene: where each bracket closes, and which
7612 // brackets are open at each range's start (in push order).
7613 let mut closed_at: Vec<Option<usize>> = vec![None; len];
7614 let mut stack: Vec<(BracketKind, usize)> = Vec::new();
7615 let mut open_at_start: Vec<Vec<usize>> = vec![Vec::new(); clamped.len()];
7616 let snapshot = |index: usize, stack: &[(BracketKind, usize)], open: &mut [Vec<usize>]| {
7617 for (slot, &(start, _)) in open.iter_mut().zip(&clamped) {
7618 if start == index {
7619 slot.extend(stack.iter().map(|&(_, at)| at));
7620 }
7621 }
7622 };
7623 for (index, command) in commands.iter().enumerate() {
7624 snapshot(index, &stack, &mut open_at_start);
7625 if let Some(kind) = opens(command) {
7626 stack.push((kind, index));
7627 } else if let Some(kind) = closes(command)
7628 && let Some(position) = stack.iter().rposition(|&(open, _)| open == kind)
7629 {
7630 let (_, at) = stack.remove(position);
7631 closed_at[at] = Some(index);
7632 }
7633 }
7634 snapshot(len, &stack, &mut open_at_start);
7635 // A retained bracket spans a cut of the range `start..end` when it opened
7636 // before the range, or is still open where the range ends short of the
7637 // scene's own end.
7638 let spans = |at: usize, start: usize, end: usize| {
7639 at < start || (end < len && closed_at[at].is_none_or(|c| c >= end))
7640 };
7641
7642 out.reset();
7643 let mut builder = SceneBuilder::new(out);
7644
7645 let push = |at: usize,
7646 (start, end): (usize, usize),
7647 builder: &mut SceneBuilder<'_>,
7648 open: &mut Vec<OpenBracket>,
7649 prefix: &mut Affine,
7650 cut: &mut RangeCut| {
7651 let command = &commands[at];
7652 match command {
7653 frust_scene::Command::PushRetained {
7654 rect,
7655 alpha,
7656 scale,
7657 transform,
7658 ..
7659 } => {
7660 if open
7661 .iter()
7662 .any(|bracket| matches!(bracket, OpenBracket::Demoted { .. }))
7663 {
7664 open.push(OpenBracket::Absorbed);
7665 } else if spans(at, start, end) {
7666 cut.demoted_retained += 1;
7667 let correction =
7668 crate::compile::layers::retained_correction(*rect, *scale, *transform);
7669 let layer = *alpha < 1.0;
7670 if layer {
7671 let corrected = *prefix * correction * *transform;
7672 builder.push_transform(corrected);
7673 builder.push_layer(*rect, *alpha);
7674 builder.pop_transform();
7675 }
7676 open.push(OpenBracket::Demoted {
7677 layer,
7678 prefix: *prefix,
7679 });
7680 *prefix *= correction;
7681 } else {
7682 replay(builder, *prefix, command);
7683 open.push(OpenBracket::Retained);
7684 }
7685 }
7686 _ => {
7687 if let Some(kind) = opens(command) {
7688 replay(builder, *prefix, command);
7689 open.push(OpenBracket::Plain(kind));
7690 }
7691 }
7692 }
7693 };
7694 let close =
7695 |bracket: OpenBracket, builder: &mut SceneBuilder<'_>, prefix: &mut Affine| match bracket {
7696 OpenBracket::Plain(BracketKind::Clip) => builder.pop_clip(),
7697 OpenBracket::Plain(BracketKind::Layer) => builder.pop_layer(),
7698 OpenBracket::Plain(BracketKind::Retained) | OpenBracket::Retained => {
7699 builder.pop_retained();
7700 }
7701 OpenBracket::Demoted {
7702 layer,
7703 prefix: before,
7704 } => {
7705 if layer {
7706 builder.pop_layer();
7707 }
7708 *prefix = before;
7709 }
7710 OpenBracket::Absorbed => {}
7711 };
7712
7713 let mut open: Vec<OpenBracket> = Vec::new();
7714 for (&bounds, reopen) in clamped.iter().zip(&open_at_start) {
7715 let (start, end) = bounds;
7716 // Every range starts from the identity with nothing open: the one
7717 // before it closed everything it opened.
7718 let mut prefix = Affine::IDENTITY;
7719 for &at in reopen {
7720 cut.reopened += 1;
7721 push(at, bounds, &mut builder, &mut open, &mut prefix, &mut cut);
7722 }
7723 for (index, command) in commands.iter().enumerate().take(end).skip(start) {
7724 if opens(command).is_some() {
7725 push(
7726 index,
7727 bounds,
7728 &mut builder,
7729 &mut open,
7730 &mut prefix,
7731 &mut cut,
7732 );
7733 } else if let Some(kind) = closes(command) {
7734 match open.iter().rposition(|bracket| bracket.kind() == kind) {
7735 Some(position) => {
7736 let bracket = open.remove(position);
7737 close(bracket, &mut builder, &mut prefix);
7738 }
7739 None => cut.dropped_pops += 1,
7740 }
7741 } else if matches!(command, frust_scene::Command::DrawRetained { .. })
7742 && !options.keep_draw_retained
7743 {
7744 cut.dropped_draw_retained += 1;
7745 } else {
7746 replay(&mut builder, prefix, command);
7747 }
7748 }
7749 while let Some(bracket) = open.pop() {
7750 cut.closed += 1;
7751 close(bracket, &mut builder, &mut prefix);
7752 }
7753 }
7754 cut
7755}
7756
7757/// The outermost `PushRetained` bracket still open just before command
7758/// `index` of `commands` — its command index — or `None` when no retained
7759/// bracket is open there.
7760///
7761/// Pops are matched the way [`project_command_range`] matches them: a
7762/// `PopRetained` closes the most recently opened retained bracket, whatever
7763/// clips or layers were opened in between, and one with no retained bracket
7764/// open is ignored. An `index` past the end asks about the end of the list,
7765/// so a bracket never closed is open there. A range starting at `index`
7766/// splits the bracket answered here: [`project_command_range`] demotes it to
7767/// inline paint on both sides of the cut. Read-only and one pass over
7768/// `commands[..index]`.
7769pub fn open_retained_bracket(commands: &[frust_scene::Command], index: usize) -> Option<usize> {
7770 let mut open: Vec<usize> = Vec::new();
7771 for (at, command) in commands.iter().enumerate().take(index) {
7772 match command {
7773 frust_scene::Command::PushRetained { .. } => open.push(at),
7774 frust_scene::Command::PopRetained => {
7775 open.pop();
7776 }
7777 _ => {}
7778 }
7779 }
7780 open.first().copied()
7781}
7782
7783#[cfg(test)]
7784mod tests {
7785 use super::*;
7786 use frust_gpu::DownlevelProfile;
7787 use kurbo::Rect;
7788 use peniko::color::palette::css::{BLUE, RED};
7789 use std::collections::BTreeMap;
7790 use vello_common::geometry::RectU16;
7791 use vello_common::paint::IndexedPaint;
7792
7793 /// Polls `future` to completion on the calling thread — device creation
7794 /// is the only future this module's GPU tests await.
7795 fn block_on<F: std::future::Future>(future: F) -> F::Output {
7796 use std::task::{Context, Poll, Waker};
7797
7798 let mut cx = Context::from_waker(Waker::noop());
7799 let mut future = std::pin::pin!(future);
7800 loop {
7801 match future.as_mut().poll(&mut cx) {
7802 Poll::Ready(value) => return value,
7803 Poll::Pending => std::thread::yield_now(),
7804 }
7805 }
7806 }
7807
7808 /// Finishing the warm-up builds the whole (de-duplicated) list exactly
7809 /// once however often it is called, waiting on a finished warm-up returns
7810 /// at once, and neither encodes a frame: the staging ring hands out no
7811 /// slot and creates no buffer.
7812 #[test]
7813 #[ignore = "needs a real GPU adapter; run with `cargo test -p frust-engine --lib -- --ignored \
7814 --test-threads=1` (pin the adapter with WGPU_BACKEND / WGPU_ADAPTER_NAME)"]
7815 fn finishing_the_warm_up_is_idempotent_and_encodes_nothing() {
7816 let mut context = frust_gpu::RenderContext::new();
7817 let handle = block_on(context.device()).expect("device creation");
7818 let caps = handle.caps.clone();
7819 let format = wgpu::TextureFormat::Rgba8Unorm;
7820 let mut renderer = EngineRenderer::new(&handle.device, &caps, format, None)
7821 .expect("the engine builds on this device");
7822 let mut listed = Vec::new();
7823 for desc in warm_up_list(&renderer.shaders, format, renderer.routing.uber) {
7824 if !listed.contains(&desc) {
7825 listed.push(desc);
7826 }
7827 }
7828
7829 renderer.finish_warm_up(&handle.device);
7830 assert!(renderer.warm_up_done());
7831 assert_eq!(
7832 renderer.compiled_pipelines(),
7833 listed.len() as u64,
7834 "the first call builds every listed variant once"
7835 );
7836 for _ in 0..3 {
7837 renderer.finish_warm_up(&handle.device);
7838 assert!(renderer.wait_warm_up(Duration::ZERO));
7839 }
7840 assert_eq!(
7841 renderer.compiled_pipelines(),
7842 listed.len() as u64,
7843 "a finished warm-up compiles nothing more"
7844 );
7845 assert_eq!(
7846 renderer.staging_stats().unwrap_or_default(),
7847 frust_gpu::staging::StagingStats::default(),
7848 "the warm-up records no frame: the staging ring was never used"
7849 );
7850 }
7851
7852 /// The warm-up list is [`warm_up_descs_for`](crate::gpu::pipelines::warm_up_descs_for)'s list plus the first-frame
7853 /// openers and the layer-composite tail: a superset of it, opening with
7854 /// the atlas pass and ending with the depth-alpha layer composite, whose
7855 /// distinct entries are exactly the base list and that tail — and under
7856 /// the Uber kill switch it is the base list unchanged. Checked over
7857 /// variant keys, which is all a device-free test can build.
7858 #[test]
7859 fn the_warm_up_list_extends_the_base_list_in_order() {
7860 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
7861 enum Key {
7862 Atlas,
7863 State(EnginePipeline, PaintClass),
7864 }
7865 let describe = Key::State;
7866 let tail = Key::State(EnginePipeline::StripDepthAlpha, PaintClass::LayerComposite);
7867
7868 for image_sampler in [false, true] {
7869 // The manual family's atlas pass IS the intermediate state's Uber
7870 // variant; the sampler family's is a variant of its own.
7871 let atlas = move || {
7872 if image_sampler {
7873 Key::Atlas
7874 } else {
7875 Key::State(
7876 crate::gpu::pipelines::ATLAS_STRIP_PIPELINE,
7877 PaintClass::Uber,
7878 )
7879 }
7880 };
7881 let base = warm_up_plan(image_sampler, false, describe, atlas);
7882 let list = ordered_warm_up(image_sampler, false, describe, atlas);
7883
7884 let listed: std::collections::HashSet<_> = list.iter().copied().collect();
7885 for key in &base {
7886 assert!(listed.contains(key), "{key:?} is missing from the list");
7887 }
7888
7889 // The first-frame openers, in the order an ordinary first frame
7890 // draws them. Pinned exactly: a reorder changes which pipeline
7891 // the frame waits behind.
7892 let openers = [
7893 atlas(),
7894 describe(EnginePipeline::StripDepthAlpha, PaintClass::Glyph),
7895 describe(EnginePipeline::StripDepthAlpha, PaintClass::SolidCoverage),
7896 describe(EnginePipeline::StripOpaque, PaintClass::SolidCoverage),
7897 describe(EnginePipeline::StripDepthAlpha, PaintClass::Uber),
7898 ];
7899 assert_eq!(
7900 &list[..5],
7901 &openers[..],
7902 "opens with the first frame's pipelines"
7903 );
7904 // The base list follows the openers in its own order, unchanged.
7905 assert_eq!(
7906 &list[5..list.len() - 1],
7907 &base[..],
7908 "the base list follows the openers in warm_up_descs_for's order"
7909 );
7910 assert_eq!(list.last(), Some(&tail), "ends with the layer composite");
7911
7912 // De-duplicated by first occurrence, as the queue does: the
7913 // distinct entries are the base list and the tail, nothing else.
7914 let mut seen = std::collections::HashSet::new();
7915 let distinct: Vec<_> = list.iter().filter(|key| seen.insert(**key)).collect();
7916 let mut expected: std::collections::HashSet<_> = base.iter().copied().collect();
7917 expected.insert(tail);
7918 assert_eq!(distinct.len(), expected.len());
7919 assert!(distinct.iter().all(|key| expected.contains(*key)));
7920
7921 let forced = ordered_warm_up(image_sampler, true, describe, atlas);
7922 assert_eq!(forced, warm_up_plan(image_sampler, true, describe, atlas));
7923 }
7924 }
7925
7926 /// The first-frame shader set is exactly the distinct programs of the
7927 /// warm-up list's opening, in the order the opening first names them —
7928 /// four modules in the sampler family (the atlas pass's manual Uber
7929 /// program is one of its own), three in the manual family (where it is the
7930 /// family's Uber program) — and under the Uber kill switch the Uber
7931 /// module(s) the first frame's atlas and surface passes draw with.
7932 #[test]
7933 fn the_first_frame_shaders_are_the_programs_of_the_list_opening() {
7934 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
7935 enum Key {
7936 Atlas,
7937 State(EnginePipeline, PaintClass),
7938 }
7939 /// A program: its family (sampler or manual) and its class.
7940 type Program = (bool, PaintClass);
7941
7942 for image_sampler in [false, true] {
7943 let atlas = move || {
7944 if image_sampler {
7945 Key::Atlas
7946 } else {
7947 Key::State(
7948 crate::gpu::pipelines::ATLAS_STRIP_PIPELINE,
7949 PaintClass::Uber,
7950 )
7951 }
7952 };
7953 // The atlas pass always draws the manual family's Uber program.
7954 let program_of = |key: Key| -> Program {
7955 match key {
7956 Key::Atlas => (false, PaintClass::Uber),
7957 Key::State(_, class) => (image_sampler, class),
7958 }
7959 };
7960 let list = ordered_warm_up(image_sampler, false, Key::State, atlas);
7961 let mut opening: Vec<Program> = Vec::new();
7962 for key in &list[..5] {
7963 let program = program_of(*key);
7964 if !opening.contains(&program) {
7965 opening.push(program);
7966 }
7967 }
7968 let set = first_frame_programs(
7969 false,
7970 |class| (image_sampler, class),
7971 || (false, PaintClass::Uber),
7972 );
7973 assert_eq!(set, opening, "the opening's programs, in its order");
7974 assert_eq!(set.len(), if image_sampler { 4 } else { 3 });
7975
7976 let forced = first_frame_programs(
7977 true,
7978 |class| (image_sampler, class),
7979 || (false, PaintClass::Uber),
7980 );
7981 let mut uber: Vec<Program> = vec![(false, PaintClass::Uber)];
7982 if image_sampler {
7983 uber.push((true, PaintClass::Uber));
7984 }
7985 assert_eq!(forced, uber, "the Uber module(s) alone");
7986 }
7987 }
7988
7989 /// The `frust-perf enc` line is a contract: a capture is graded by
7990 /// grepping its fields, so the field order and the names are pinned here
7991 /// rather than only by the formatter. `perf-trace`-only, with the line —
7992 /// `cargo test -p frust-engine --features perf-trace` is where it runs.
7993 #[cfg(feature = "perf-trace")]
7994 #[test]
7995 fn an_encode_trace_line_reports_every_column_in_order() {
7996 let mut trace = EncodeTrace {
7997 frames: 7,
7998 ..EncodeTrace::default()
7999 };
8000 // One row per column, each carrying a distinct value in its own
8001 // column, so a transposed or dropped field shows up as a wrong number
8002 // rather than only as a wrong name.
8003 let mut row = [0_u32; ENCODE_TRACE_ROW];
8004 for (column, slot) in row.iter_mut().enumerate() {
8005 *slot = (column as u32 + 1) * 1_000;
8006 }
8007 trace.window.push(row);
8008
8009 let line = trace.line();
8010 let phases: Vec<String> = ENCODE_TRACE_COLUMNS
8011 .iter()
8012 .enumerate()
8013 .map(|(column, name)| format!("{name}_us={}.0", column + 1))
8014 .collect();
8015 // The counts carry no unit suffix and are reported as the raw values
8016 // the row holds, not scaled to microseconds like the phases above.
8017 let counts: Vec<String> = ENCODE_TRACE_COUNTS
8018 .iter()
8019 .enumerate()
8020 .map(|(offset, name)| {
8021 format!(
8022 "{name}={}",
8023 (ENCODE_TRACE_COLUMNS.len() + offset + 1) * 1_000
8024 )
8025 })
8026 .collect();
8027 assert_eq!(
8028 line,
8029 format!(
8030 "frust-perf enc n=7 w=1 {} total_p95_us={}.0 {}",
8031 phases.join(" "),
8032 ENCODE_TRACE_COLUMNS.len(),
8033 counts.join(" "),
8034 ),
8035 );
8036 }
8037
8038 /// The event counts are sums over the window, the level counts medians,
8039 /// and the pinned order puts them where the contract says.
8040 #[cfg(feature = "perf-trace")]
8041 #[test]
8042 fn an_encode_trace_line_sums_events_and_takes_the_median_of_levels() {
8043 let mut trace = EncodeTrace::default();
8044 let base = ENCODE_TRACE_COLUMNS.len();
8045 for frame in 0..3_u32 {
8046 let mut row = [0_u32; ENCODE_TRACE_ROW];
8047 row[base] = 10 + frame; // draws: median 11
8048 row[base + 5] = 1; // replays: sum 3
8049 row[base + 6] = 2; // dirty_pages: sum 6
8050 row[base + 8] = frame; // glyph_misses: sum 3
8051 row[base + 10] = 4 + frame; // run_keys: median 5
8052 row[base + 11] = 100 + frame; // staged_bytes: median 101
8053 row[base + 12] = 1; // staging_busy: sum 3
8054 row[base + 15] = 2; // bind_groups: sum 6
8055 trace.window.push(row);
8056 }
8057 let line = trace.line();
8058 let tail = line.split(" total_p95_us=").nth(1).unwrap();
8059 let counts = tail.split_once(' ').unwrap().1;
8060 assert_eq!(
8061 counts,
8062 "draws=11 strips=0 alphas=0 glyph_draws=0 atlas_glyphs=0 replays=3 \
8063 dirty_pages=6 atlas_refused=0 glyph_misses=3 glyph_evicted=0 run_keys=5 \
8064 staged_bytes=101 staging_busy=3 staging_too_large=0 staging_buffers=0 \
8065 bind_groups=6"
8066 );
8067 }
8068
8069 /// The staging and bind-group counts are per-frame deltas: each row carries
8070 /// only the change since the previous frame's reading, so a window's sum is
8071 /// the change across the window.
8072 #[cfg(feature = "perf-trace")]
8073 #[test]
8074 fn a_frame_row_carries_only_the_staging_and_bind_group_change_since_the_last() {
8075 let mut trace = EncodeTrace::default();
8076 let base = ENCODE_TRACE_COLUMNS.len();
8077 let readings = [
8078 TraceCounters {
8079 busy: 2,
8080 too_large: 0,
8081 buffers_created: 1,
8082 bind_groups: 4,
8083 },
8084 TraceCounters {
8085 busy: 5,
8086 too_large: 1,
8087 buffers_created: 1,
8088 bind_groups: 4,
8089 },
8090 ];
8091 for now in readings {
8092 trace.record(&EncodeSpans::default(), now);
8093 }
8094 assert_eq!(&trace.window[0][base + 12..], &[2, 0, 1, 4]);
8095 assert_eq!(&trace.window[1][base + 12..], &[3, 1, 0, 0]);
8096 }
8097
8098 /// A window's percentile is nearest-rank over the column it names, so the
8099 /// value reported is one the window really contains.
8100 #[cfg(feature = "perf-trace")]
8101 #[test]
8102 fn an_encode_trace_percentile_is_nearest_rank_per_column() {
8103 let mut trace = EncodeTrace::default();
8104 for value in [4_000_u32, 1_000, 3_000, 2_000] {
8105 let mut row = [0_u32; ENCODE_TRACE_ROW];
8106 row[0] = value;
8107 trace.window.push(row);
8108 }
8109 assert_eq!(trace.percentile(0, 50), 2_000);
8110 assert_eq!(trace.percentile(0, 95), 4_000);
8111 // An empty window reports zero rather than reaching past its end.
8112 trace.window.clear();
8113 assert_eq!(trace.percentile(0, 50), 0);
8114 }
8115
8116 /// Nanoseconds round up to a tenth of a microsecond, so a phase that cost
8117 /// anything at all never reports as free.
8118 #[cfg(feature = "perf-trace")]
8119 #[test]
8120 fn a_span_that_cost_anything_never_formats_as_zero() {
8121 assert_eq!(format_us(0), "0.0");
8122 assert_eq!(format_us(1), "0.1");
8123 assert_eq!(format_us(4_170_000), "4170.0");
8124 }
8125
8126 /// A staged upload covers the rows the frame fills and every row the last
8127 /// upload left live — so a frame whose coverage shrank re-zeroes the rows
8128 /// it gave up exactly once — and nothing past the texture.
8129 #[test]
8130 fn a_staged_upload_covers_filled_and_previously_live_rows() {
8131 assert_eq!(rows_filled(0, 32_768), 0);
8132 assert_eq!(rows_filled(1, 32_768), 1);
8133 assert_eq!(rows_filled(32_768, 32_768), 1);
8134 assert_eq!(rows_filled(32_769, 32_768), 2);
8135 // A new texture is live throughout: its first upload is a full one.
8136 assert_eq!(staged_rows(1, 8, 8), 8);
8137 // Shrinking: the rows the last frame filled are rewritten once...
8138 assert_eq!(staged_rows(2, 5, 8), 5);
8139 // ...and after that only the frame's own.
8140 assert_eq!(staged_rows(2, 2, 8), 2);
8141 // Growing past the live mark, never past the texture.
8142 assert_eq!(staged_rows(6, 2, 8), 6);
8143 assert_eq!(staged_rows(9, 2, 8), 8);
8144 // Nothing filled and nothing live: nothing to write.
8145 assert_eq!(staged_rows(0, 0, 8), 0);
8146 }
8147
8148 /// The compile phases partition the compile they sit inside, so their sum
8149 /// is the whole call minus the overhead the renderer measures around it.
8150 #[test]
8151 fn compile_phases_sum_without_overflowing() {
8152 let spans = CompileSpans {
8153 validate: Duration::from_micros(10),
8154 prepare: Duration::from_micros(20),
8155 classify: Duration::from_micros(30),
8156 admit: Duration::from_micros(40),
8157 walk: Duration::from_micros(50),
8158 // A subset of `walk`, so the sum must not grow by it.
8159 glyphs: Duration::from_micros(45),
8160 finish: Duration::from_micros(60),
8161 };
8162 assert_eq!(spans.total(), Duration::from_micros(210));
8163 assert_eq!(
8164 CompileSpans {
8165 validate: Duration::MAX,
8166 walk: Duration::MAX,
8167 ..CompileSpans::default()
8168 }
8169 .total(),
8170 Duration::MAX,
8171 );
8172 }
8173
8174 #[test]
8175 fn a_target_past_the_u16_grid_is_refused() {
8176 assert_eq!(
8177 grid_size(1920, 1080).expect("a normal surface"),
8178 (1920, 1080)
8179 );
8180 assert_eq!(
8181 grid_size(u32::from(u16::MAX), 16).expect("the grid ceiling itself"),
8182 (u16::MAX, 16)
8183 );
8184 assert!(matches!(
8185 grid_size(u32::from(u16::MAX) + 1, 16),
8186 Err(EngineError::TargetTooLarge)
8187 ));
8188 assert!(matches!(
8189 grid_size(16, u32::from(u16::MAX) + 1),
8190 Err(EngineError::TargetTooLarge)
8191 ));
8192 }
8193
8194 #[test]
8195 fn a_premultiplied_clear_folds_alpha_into_the_colour() {
8196 let half = RED.with_alpha(0.5);
8197 let premultiplied = clear_color(half, OutputAlpha::Premultiplied);
8198 let straight = clear_color(half, OutputAlpha::Straight);
8199
8200 assert!((premultiplied.r - 0.5).abs() < 1e-6);
8201 assert!((premultiplied.a - 0.5).abs() < 1e-6);
8202 assert!((straight.r - 1.0).abs() < 1e-6);
8203 assert!((straight.a - 0.5).abs() < 1e-6);
8204
8205 // An opaque colour is identical under both conventions.
8206 assert_eq!(
8207 clear_color(BLUE, OutputAlpha::Premultiplied),
8208 clear_color(BLUE, OutputAlpha::Straight)
8209 );
8210 }
8211
8212 #[test]
8213 fn a_solid_paint_travels_premultiplied_in_the_instance_payload() {
8214 let paint = pack_paint(&Paint::from(RED), 3, &[]).expect("a solid paint always resolves");
8215 let values = paint.values_at(40, 12);
8216 assert_eq!(values.paint, SOLID_PAINT);
8217 assert_eq!(values.depth_index, 3);
8218 assert_eq!(values.payload, RED.premultiply().to_rgba8().to_u32());
8219 assert!(paint.opaque);
8220 assert_eq!(
8221 paint.values_at(0, 0).payload,
8222 values.payload,
8223 "a solid colour is the same wherever it is stamped"
8224 );
8225
8226 let paint = pack_paint(&Paint::from(RED.with_alpha(0.5)), 0, &[])
8227 .expect("a translucent solid paint still resolves");
8228 assert!(
8229 !paint.opaque,
8230 "a translucent paint never reaches the opaque pass"
8231 );
8232 }
8233
8234 #[test]
8235 fn an_unresolvable_indexed_paint_skips_its_draw() {
8236 let indexed = Paint::Indexed(IndexedPaint::new(0));
8237 assert!(
8238 pack_paint(&indexed, 0, &[]).is_none(),
8239 "an index past the frame's slots resolves to nothing"
8240 );
8241 assert!(
8242 pack_paint(&indexed, 0, &[None]).is_none(),
8243 "so does a slot the lowering refused"
8244 );
8245 }
8246
8247 /// An image record is classed by the path it samples through — the one
8248 /// split an instance's flag cannot make — and every other record by its
8249 /// paint type.
8250 #[test]
8251 fn a_lowered_record_names_its_strip_program_class() {
8252 use crate::gpu::atlas::pack_image_params;
8253 use crate::gpu::paint_texture::GpuEncodedImage;
8254
8255 let image = |quality: u32, external: bool| {
8256 GpuEncodedPaint::Image(GpuEncodedImage {
8257 image_params: pack_image_params(quality, 0, 0, 3, external),
8258 image_size: 0,
8259 image_offset: 0,
8260 transform: [1.0, 0.0, 0.0, 1.0, 0.0, 0.0],
8261 tint: 0,
8262 tint_mode: 0,
8263 image_padding: 0,
8264 })
8265 };
8266 assert_eq!(class_of_record(&image(0, false)), PaintClass::Image);
8267 assert_eq!(class_of_record(&image(1, false)), PaintClass::Image);
8268 assert_eq!(class_of_record(&image(2, false)), PaintClass::ImageFiltered);
8269 assert_eq!(class_of_record(&image(1, true)), PaintClass::ImageFiltered);
8270 }
8271
8272 #[test]
8273 fn a_resolved_indexed_paint_samples_at_each_instances_own_position() {
8274 let slots = [Some(ResolvedPaint {
8275 paint_type: PaintType::LinearGradient,
8276 texel_offset: 6,
8277 opaque: true,
8278 external: None,
8279 class: PaintClass::Gradient,
8280 })];
8281 let paint = pack_paint(&Paint::Indexed(IndexedPaint::new(0)), 2, &slots)
8282 .expect("a lowered paint resolves");
8283
8284 assert_eq!(
8285 paint.paint,
8286 pack_paint_descriptor(PaintType::LinearGradient, 6)
8287 );
8288 assert_eq!(paint.depth_index, 2);
8289 // The payload is the instance's scene origin, not a colour, so two
8290 // instances of the same draw carry different payloads.
8291 assert_eq!(paint.payload_at(8, 4), 8 | (4 << 16));
8292 assert_eq!(paint.payload_at(40, 4), 40 | (4 << 16));
8293 }
8294
8295 #[test]
8296 fn a_u16_pair_packs_low_half_first() {
8297 assert_eq!(pack_u16_pair(0, 0), 0);
8298 assert_eq!(pack_u16_pair(1, 0), 1);
8299 assert_eq!(pack_u16_pair(0, 1), 1 << 16);
8300 assert_eq!(pack_u16_pair(u16::MAX, u16::MAX), u32::MAX);
8301 }
8302
8303 #[test]
8304 fn the_minimum_resource_dimension_keeps_every_upload_row_legal() {
8305 // The gradient LUT is the narrowest resource at four bytes per texel,
8306 // so it is the one that fixes the floor.
8307 assert_eq!(
8308 MIN_RESOURCE_TEXTURE_DIM * BYTES_PER_TEXEL,
8309 wgpu::COPY_BYTES_PER_ROW_ALIGNMENT
8310 );
8311 assert!(MIN_RESOURCE_TEXTURE_DIM.is_power_of_two());
8312 }
8313
8314 // -----------------------------------------------------------------
8315 // Where the hole punch lands in the pass plan
8316 //
8317 // What a frame's pass plan COSTS, and where the erase sits inside it,
8318 // is a decision over the recording's shape — no device, no pixels.
8319 // The pixel half of the same contract is
8320 // `frust-testing`'s `tests/aa_over_punch.rs`, against `vello_cpu`.
8321 // -----------------------------------------------------------------
8322
8323 /// Viewport every plan below is built against.
8324 const PLAN_VIEWPORT: (u16, u16) = (64, 48);
8325
8326 /// The slot, and a chip straddling its right edge — so a chip drawn over
8327 /// the punch is half inside it and half over the backdrop.
8328 const PLAN_SLOT: Rect = Rect::new(4.0, 4.0, 32.0, 44.0);
8329 const PLAN_CHIP: Rect = Rect::new(16.0, 12.0, 56.0, 32.0);
8330
8331 /// The plan `Scratch::build` produces for `scene`, alongside the rounds it
8332 /// was scheduled from, under a target that carries alpha.
8333 fn plan_of(scene: &Scene) -> (Scratch, Vec<Round>) {
8334 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8335 .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
8336 .expect("an in-range scene compiles");
8337 let rounds = Schedule::build(
8338 &frame.recorder,
8339 &TierCaps::fake(DownlevelProfile::Full),
8340 &PageConfig::default(),
8341 )
8342 .expect("a scene of solid fills schedules");
8343
8344 let mut scratch = Scratch::default();
8345 // Depth on and punching on: the shape a translucent presentation
8346 // carrying a platform-view slot is planned under.
8347 scratch.build(&frame, &rounds, true, !frame.clears.is_empty(), &[], &[]);
8348 (scratch, rounds)
8349 }
8350
8351 /// A scene recorded through the public builder.
8352 fn plan_scene(record: impl FnOnce(&mut SceneBuilder<'_>)) -> Scene {
8353 let mut scene = Scene::new();
8354 let mut builder = SceneBuilder::new(&mut scene);
8355 record(&mut builder);
8356 scene
8357 }
8358
8359 fn plan_backdrop(builder: &mut SceneBuilder<'_>) {
8360 builder.fill_rect(
8361 Rect::new(
8362 0.0,
8363 0.0,
8364 f64::from(PLAN_VIEWPORT.0),
8365 f64::from(PLAN_VIEWPORT.1),
8366 ),
8367 Brush::Solid(RED),
8368 );
8369 }
8370
8371 #[test]
8372 fn a_frame_that_punches_nothing_is_planned_exactly_as_its_rounds() {
8373 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8374 plan_backdrop(builder);
8375 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8376 }));
8377
8378 assert_eq!(
8379 scratch.rounds.len(),
8380 rounds.len(),
8381 "a frame with no clear is cut nowhere, so it costs exactly its \
8382 scheduled rounds' passes"
8383 );
8384 assert!(!scratch.punches(), "and records no punch pass at all");
8385 }
8386
8387 #[test]
8388 fn a_clear_recorded_last_keeps_its_pass_after_every_round() {
8389 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8390 plan_backdrop(builder);
8391 builder.clear_rect(PLAN_SLOT);
8392 }));
8393
8394 assert_eq!(
8395 scratch.rounds.len(),
8396 rounds.len(),
8397 "nothing is recorded after the clear, so nothing is cut"
8398 );
8399 assert!(
8400 scratch.rounds.iter().all(|plan| plan.punch.1 == 0),
8401 "no round carries the punch"
8402 );
8403 assert!(
8404 scratch.punch.1 > 0,
8405 "it is issued after the last round instead — the position an \
8406 unconditionally-trailing pass would have put it in, which is why \
8407 a frame shaped like this renders byte-identically"
8408 );
8409 }
8410
8411 #[test]
8412 fn a_draw_over_a_clear_cuts_the_surface_round_and_the_punch_goes_in_the_cut() {
8413 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
8414 plan_backdrop(builder);
8415 builder.clear_rect(PLAN_SLOT);
8416 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8417 }));
8418
8419 assert_eq!(
8420 scratch.rounds.len(),
8421 rounds.len() + 1,
8422 "the surface round is cut in two — one extra pass, and one only"
8423 );
8424 assert_eq!(
8425 scratch.punch,
8426 (0, 0),
8427 "with nothing left over for a trailing pass"
8428 );
8429
8430 let cut = scratch
8431 .rounds
8432 .iter()
8433 .position(|plan| plan.punch.1 > 0)
8434 .expect("the cut plan carries the punch");
8435 assert_eq!(cut, scratch.rounds.len() - 2, "and it is the cut plan");
8436
8437 // The buffer says the same thing the plan does: the backdrop's
8438 // instances precede the punch's, and the chip's follow them.
8439 let before = scratch.rounds[cut].segments.clone();
8440 let after = scratch.rounds[cut + 1].segments.clone();
8441 let end_of = |range: Range<usize>| {
8442 scratch.segments[range]
8443 .iter()
8444 .map(|segment| match *segment {
8445 Segment::Strips(first, count, _)
8446 | Segment::External(first, count, _, _)
8447 | Segment::Backdrop(first, count, _) => first + count,
8448 Segment::Composite(first, _) => first + 1,
8449 })
8450 .max()
8451 .expect("a plan of this frame draws something")
8452 };
8453 let (punch_first, punch_count) = scratch.rounds[cut].punch;
8454 assert!(
8455 end_of(before) <= punch_first,
8456 "everything recorded before the clear is erased by the punch"
8457 );
8458 assert!(
8459 scratch.segments[after]
8460 .iter()
8461 .all(|segment| match *segment {
8462 Segment::Strips(first, _, _)
8463 | Segment::External(first, _, _, _)
8464 | Segment::Backdrop(first, _, _)
8465 | Segment::Composite(first, _) => first >= punch_first + punch_count,
8466 }),
8467 "and everything recorded after it lands on top of the erase"
8468 );
8469 }
8470
8471 // -----------------------------------------------------------------
8472 // The pass list: what a frame of each shape records, merged and apart
8473 //
8474 // Host-only: `Scratch::plan_passes` states the list over the plan, and
8475 // a debug build holds every frame `record_frame` really records to it,
8476 // so the device suites (`tests/encode_contract.rs` among them) check the
8477 // same list against a real encoder. The filter shape is pinned only here:
8478 // no `Scene` command records a filter layer, so no `encode` call reaches
8479 // a filter round.
8480 // -----------------------------------------------------------------
8481
8482 /// A rectangle off the pixel grid, so its edges are anti-aliased and the
8483 /// draw always reaches a surface round's blended strips — never only the
8484 /// opaque ones.
8485 const PLAN_SOFT: Rect = Rect::new(8.5, 8.5, 24.5, 40.5);
8486
8487 /// The passes `scratch` records with the opening merged, and apart.
8488 fn pass_lists(scratch: &Scratch) -> (Vec<FramePass>, Vec<FramePass>) {
8489 let shape = |merge| PassShape {
8490 merge,
8491 instances: true,
8492 opaque: !scratch.opaque.is_empty(),
8493 punch: scratch.punches(),
8494 };
8495 let mut merged = Vec::new();
8496 scratch.plan_passes(&shape(true), &mut merged);
8497 let mut apart = Vec::new();
8498 scratch.plan_passes(&shape(false), &mut apart);
8499 assert_opened_once(&merged);
8500 assert!(
8501 !apart.contains(&FramePass::Opening),
8502 "an unmerged frame records no opening: {apart:?}"
8503 );
8504 (merged, apart)
8505 }
8506
8507 /// A merged list opens its surface exactly once, before every other pass
8508 /// over it, and never records the clear or the opaque strips apart.
8509 fn assert_opened_once(passes: &[FramePass]) {
8510 let openings = passes
8511 .iter()
8512 .filter(|pass| **pass == FramePass::Opening)
8513 .count();
8514 assert_eq!(openings, 1, "one opening per frame: {passes:?}");
8515 let first_surface = passes.iter().position(|pass| pass.targets_surface());
8516 assert_eq!(
8517 first_surface,
8518 passes.iter().position(|pass| *pass == FramePass::Opening),
8519 "the opening is the first pass over the surface: {passes:?}"
8520 );
8521 assert!(
8522 !passes.contains(&FramePass::Clear) && !passes.contains(&FramePass::Opaque),
8523 "and the clear and opaque strips live inside it: {passes:?}"
8524 );
8525 }
8526
8527 #[test]
8528 fn an_empty_frame_is_the_opening_alone() {
8529 let (scratch, _) = plan_of(&plan_scene(|_| {}));
8530 let (merged, apart) = pass_lists(&scratch);
8531 assert_eq!(merged, [FramePass::Opening]);
8532 assert_eq!(apart, [FramePass::Clear]);
8533
8534 // And the same before any instance buffer exists at all.
8535 for merge in [true, false] {
8536 let mut passes = Vec::new();
8537 scratch.plan_passes(
8538 &PassShape {
8539 merge,
8540 instances: false,
8541 opaque: false,
8542 punch: false,
8543 },
8544 &mut passes,
8545 );
8546 assert_eq!(passes.len(), 1, "one clean clear: {passes:?}");
8547 }
8548 }
8549
8550 #[test]
8551 fn a_surface_only_frame_folds_its_clear_and_opaque_strips_into_one_pass() {
8552 let (scratch, _) = plan_of(&plan_scene(|builder| {
8553 plan_backdrop(builder);
8554 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8555 }));
8556 assert!(!scratch.opaque.is_empty(), "the backdrop is opaque strips");
8557
8558 let (merged, apart) = pass_lists(&scratch);
8559 assert_eq!(merged, [FramePass::Opening]);
8560 assert_eq!(
8561 apart,
8562 [FramePass::Clear, FramePass::Opaque, FramePass::Surface]
8563 );
8564 }
8565
8566 #[test]
8567 fn without_depth_the_opening_is_the_clear_and_the_surface_round() {
8568 let scene = plan_scene(|builder| {
8569 plan_backdrop(builder);
8570 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8571 });
8572 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8573 .compile(&scene, Affine::IDENTITY, PLAN_VIEWPORT)
8574 .expect("an in-range scene compiles");
8575 let rounds = Schedule::build(
8576 &frame.recorder,
8577 &TierCaps::fake(DownlevelProfile::Full),
8578 &PageConfig::default(),
8579 )
8580 .expect("a scene of solid fills schedules");
8581 let mut scratch = Scratch::default();
8582 scratch.build(&frame, &rounds, false, false, &[], &[]);
8583 assert!(scratch.opaque.is_empty(), "no depth, no opaque strips");
8584
8585 let (merged, apart) = pass_lists(&scratch);
8586 assert_eq!(merged, [FramePass::Opening]);
8587 assert_eq!(apart, [FramePass::Clear, FramePass::Surface]);
8588 }
8589
8590 #[test]
8591 fn a_page_round_waits_for_the_opening_rather_than_splitting_it() {
8592 let (scratch, _) = plan_of(&plan_scene(|builder| {
8593 plan_backdrop(builder);
8594 builder.push_layer(PLAN_CHIP, 0.5);
8595 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8596 builder.pop_layer();
8597 }));
8598 let (merged, apart) = pass_lists(&scratch);
8599 // The page is rendered first either way — a layer round writes a
8600 // pooled page and reads neither the surface nor its depth — and the
8601 // surface round that composites it carries the opening.
8602 assert_eq!(merged, [FramePass::Page, FramePass::Opening]);
8603 assert_eq!(
8604 apart,
8605 [
8606 FramePass::Clear,
8607 FramePass::Opaque,
8608 FramePass::Page,
8609 FramePass::Surface
8610 ]
8611 );
8612 }
8613
8614 #[test]
8615 fn a_punch_cut_ends_the_opening_where_the_first_surface_pass_ended() {
8616 let (scratch, _) = plan_of(&plan_scene(|builder| {
8617 plan_backdrop(builder);
8618 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8619 builder.clear_rect(PLAN_SLOT);
8620 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
8621 }));
8622 let (merged, apart) = pass_lists(&scratch);
8623 assert_eq!(
8624 merged,
8625 [FramePass::Opening, FramePass::Punch, FramePass::Surface]
8626 );
8627 assert_eq!(
8628 apart,
8629 [
8630 FramePass::Clear,
8631 FramePass::Opaque,
8632 FramePass::Surface,
8633 FramePass::Punch,
8634 FramePass::Surface
8635 ]
8636 );
8637 }
8638
8639 #[test]
8640 fn a_punch_ahead_of_every_surface_draw_takes_the_opening_alone() {
8641 // Nothing is recorded before the clear, so the plan the punch cuts
8642 // draws nothing: the opening — the backdrop's opaque strips inside it —
8643 // is recorded ahead of the erase on its own, and everything blended
8644 // after the cut loads.
8645 let (scratch, _) = plan_of(&plan_scene(|builder| {
8646 builder.clear_rect(PLAN_SLOT);
8647 plan_backdrop(builder);
8648 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE.with_alpha(0.5)));
8649 }));
8650 let (merged, apart) = pass_lists(&scratch);
8651 assert_eq!(
8652 merged,
8653 [FramePass::Opening, FramePass::Punch, FramePass::Surface]
8654 );
8655 assert_eq!(
8656 apart,
8657 [
8658 FramePass::Clear,
8659 FramePass::Opaque,
8660 FramePass::Punch,
8661 FramePass::Surface
8662 ]
8663 );
8664 }
8665
8666 #[test]
8667 fn a_punch_recorded_last_follows_the_opening() {
8668 let (scratch, _) = plan_of(&plan_scene(|builder| {
8669 plan_backdrop(builder);
8670 builder.fill_rect(PLAN_SOFT, Brush::Solid(BLUE));
8671 builder.clear_rect(PLAN_SLOT);
8672 }));
8673 let (merged, apart) = pass_lists(&scratch);
8674 assert_eq!(merged, [FramePass::Opening, FramePass::Punch]);
8675 assert_eq!(
8676 apart,
8677 [
8678 FramePass::Clear,
8679 FramePass::Opaque,
8680 FramePass::Surface,
8681 FramePass::Punch
8682 ]
8683 );
8684 }
8685
8686 #[test]
8687 fn a_filter_frame_records_its_filter_passes_before_the_opening() {
8688 use crate::filters::{LayerFilter, push_filter_layer};
8689 use vello_common::peniko::BlendMode;
8690 use vello_common::record::{CommandRecorder, LayerProps};
8691
8692 // Compiled as a backdrop and a chip, then re-recorded with the chip
8693 // inside a blurred layer under the surface: no `Scene` command
8694 // records a filter layer, so this is the only way to plan one.
8695 let mut frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
8696 .compile(
8697 &plan_scene(|builder| {
8698 plan_backdrop(builder);
8699 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
8700 }),
8701 Affine::IDENTITY,
8702 PLAN_VIEWPORT,
8703 )
8704 .expect("an in-range scene compiles");
8705 let draws = frame.recorder.draws.clone();
8706 assert_eq!(draws.len(), 2, "the backdrop and the chip");
8707 let mut recorder = CommandRecorder::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1);
8708 let strips_of = |draw: &crate::EngineDraw| &frame.strip_buf()[draw.strip_range.clone()];
8709 recorder.push_draw(draws[0].clone(), strips_of(&draws[0]));
8710 push_filter_layer(
8711 &mut recorder,
8712 LayerProps {
8713 blend_mode: BlendMode::default(),
8714 opacity: 1.0,
8715 mask: None,
8716 clip_path: None,
8717 },
8718 LayerFilter::Blur { sigma: 2.0 },
8719 Affine::IDENTITY,
8720 )
8721 .expect("a finite σ under the ceiling records");
8722 recorder.push_draw(draws[1].clone(), strips_of(&draws[1]));
8723 recorder.pop_layer();
8724 frame.recorder = recorder;
8725
8726 let rounds = Schedule::build(
8727 &frame.recorder,
8728 &TierCaps::fake(DownlevelProfile::Full),
8729 &PageConfig::default(),
8730 )
8731 .expect("a blurred layer under the surface schedules");
8732 let mut scratch = Scratch::default();
8733 scratch.build(&frame, &rounds, true, false, &[], &[]);
8734 assert!(scratch.filter_passes > 0, "the layer is filtered");
8735
8736 let (merged, apart) = pass_lists(&scratch);
8737 let filters = |passes: &[FramePass]| {
8738 passes
8739 .iter()
8740 .filter(|pass| **pass == FramePass::Filter)
8741 .count()
8742 };
8743 assert_eq!(filters(&merged), scratch.filter_passes as usize);
8744 assert_eq!(filters(&apart), filters(&merged));
8745 assert_eq!(
8746 merged.len() + 2,
8747 apart.len(),
8748 "the clear and the opaque strips fold into the surface round: \
8749 {merged:?} against {apart:?}"
8750 );
8751 assert_eq!(merged.first(), Some(&FramePass::Page));
8752 assert_eq!(
8753 merged.last(),
8754 Some(&FramePass::Opening),
8755 "every filter pass runs before the surface opens: {merged:?}"
8756 );
8757 }
8758
8759 // -----------------------------------------------------------------
8760 // A banded layer's column pages hold their own column, and only it
8761 //
8762 // The scheduler hands every band of a layer the same op list, so the
8763 // instances a band emits are where a column split is made or lost. Every
8764 // case here is host-only: the plan is built with no device, and the pixel
8765 // half of the same contract is `tests/desktop_stress.rs`'s 5K cases.
8766 // -----------------------------------------------------------------
8767
8768 /// Viewport the band plans below are built against: wide enough that one
8769 /// layer over all of it needs several column pages under [`BAND_PAGES`],
8770 /// and one tile row tall, so a draw costs one strip row per column.
8771 const BAND_VIEWPORT: (u16, u16) = (200, 8);
8772
8773 /// A page ceiling small enough to band a test-sized layer, so no case here
8774 /// has to allocate — or even name — a 5K one.
8775 const BAND_PAGES: PageConfig = PageConfig {
8776 min_page_size: 64,
8777 max_page_size: 64,
8778 };
8779
8780 /// The plan `scene` produces under `config`, alongside the rounds it was
8781 /// scheduled into.
8782 ///
8783 /// Depth is off, so every instance of the frame lands in the one blended
8784 /// buffer in painter order and a case can read the whole plan out of it;
8785 /// the split into the depth-writing pass is a surface-round decision and a
8786 /// page round never takes it.
8787 fn band_plan_of(scene: &Scene, config: &PageConfig) -> (Scratch, Vec<Round>) {
8788 let frame = SceneCompiler::new(BAND_VIEWPORT.0, BAND_VIEWPORT.1)
8789 .compile(scene, Affine::IDENTITY, BAND_VIEWPORT)
8790 .expect("an in-range scene compiles");
8791 let rounds = Schedule::build(
8792 &frame.recorder,
8793 &TierCaps::fake(DownlevelProfile::Full),
8794 config,
8795 )
8796 .expect("a layer wider than the ceiling bands rather than refusing");
8797
8798 let mut scratch = Scratch::default();
8799 scratch.build(&frame, &rounds, false, false, &[], &[]);
8800 assert_eq!(
8801 scratch.rounds.len(),
8802 rounds.len(),
8803 "a frame with no clear is cut nowhere, so the plans and the rounds \
8804 line up one for one"
8805 );
8806 (scratch, rounds)
8807 }
8808
8809 /// A layer at half opacity — so it cannot be inlined and has to take a page
8810 /// — holding one solid rectangle per entry of `rects`.
8811 fn band_scene(rects: &[(Rect, Color)]) -> Scene {
8812 let mut scene = Scene::new();
8813 let mut builder = SceneBuilder::new(&mut scene);
8814 builder.push_layer(
8815 Rect::new(
8816 0.0,
8817 0.0,
8818 f64::from(BAND_VIEWPORT.0),
8819 f64::from(BAND_VIEWPORT.1),
8820 ),
8821 0.5,
8822 );
8823 for (rect, color) in rects {
8824 builder.fill_rect(*rect, Brush::Solid(*color));
8825 }
8826 builder.pop_layer();
8827 scene
8828 }
8829
8830 /// The bounds of every page round, in order — one band's column each.
8831 fn band_bounds(rounds: &[Round]) -> Vec<RectU16> {
8832 rounds
8833 .iter()
8834 .filter_map(|round| round.page().map(|page| page.bounds))
8835 .collect()
8836 }
8837
8838 /// The strip instances one round plan issues, in execution order.
8839 fn plan_strips(scratch: &Scratch, plan: &RoundPlan) -> Vec<GpuStrip> {
8840 scratch.segments[plan.segments.clone()]
8841 .iter()
8842 .filter_map(|segment| match *segment {
8843 Segment::Strips(first, count, _)
8844 | Segment::External(first, count, _, _)
8845 | Segment::Backdrop(first, count, _) => Some((first as usize, count as usize)),
8846 Segment::Composite(..) => None,
8847 })
8848 .flat_map(|(first, count)| scratch.alpha[first..first + count].iter().copied())
8849 .collect()
8850 }
8851
8852 /// Every pixel column the frame's *page* rounds paint, mapped back out of
8853 /// the pages they were shifted into, and valued by what the shader reads
8854 /// there: the instance's paint payload, plus the exact alpha column that
8855 /// pixel samples (`None` for a sparse instance, which samples none).
8856 ///
8857 /// Keyed by the draw's own depth as well as the position, so a page painted
8858 /// by two different draws is not conflated — and so a *second* instance of
8859 /// one draw covering a pixel it already covered, which is precisely what a
8860 /// clamped band replay produces, is caught here rather than silently
8861 /// overwriting the first.
8862 fn painted_pixels(
8863 scratch: &Scratch,
8864 rounds: &[Round],
8865 ) -> BTreeMap<(u32, u16, u16), (u32, Option<u32>)> {
8866 let mut painted = BTreeMap::new();
8867
8868 for (plan, round) in scratch.rounds.iter().zip(rounds) {
8869 let Some(page) = round.page() else {
8870 continue;
8871 };
8872 for span in plan_strips(scratch, plan) {
8873 for offset in 0..span.width {
8874 let x = page.bounds.x0 + span.x + offset;
8875 let y = page.bounds.y0 + span.y;
8876 let column = (offset < span.dense_width_or_rect_height)
8877 .then(|| span.col_idx_or_rect_frac + u32::from(offset));
8878 assert!(
8879 painted
8880 .insert((span.depth_index, y, x), (span.payload, column))
8881 .is_none(),
8882 "one draw covers a device pixel at most once, however the \
8883 layer holding it was split"
8884 );
8885 }
8886 }
8887 }
8888
8889 painted
8890 }
8891
8892 #[test]
8893 fn a_banded_layer_paints_exactly_what_one_page_would_have() {
8894 // The whole point of the split, asserted as an equality rather than as
8895 // a rectangle property: the same scene planned onto one page and onto
8896 // column bands has to paint the same device pixels, from the same
8897 // paints, sampling the same alpha columns. A band replay that clamped
8898 // a strip left of its own column onto the page's edge fails here twice
8899 // over — once on the ghost pixel, once on the column it would sample.
8900 let scene = band_scene(&[
8901 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8902 // Straddles a band edge on a tile that is only partly covered, so
8903 // the case exercises an alpha-sampled instance cut in two, not
8904 // just a solid one.
8905 (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
8906 (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
8907 ]);
8908
8909 let (one_page, one_page_rounds) = band_plan_of(&scene, &PageConfig::default());
8910 let (banded, banded_rounds) = band_plan_of(&scene, &BAND_PAGES);
8911
8912 assert_eq!(
8913 band_bounds(&one_page_rounds).len(),
8914 1,
8915 "the reference plan really does hold the layer on one page"
8916 );
8917 assert!(
8918 band_bounds(&banded_rounds).len() > 1,
8919 "and the case only means anything while the other one is banded"
8920 );
8921
8922 assert_eq!(
8923 painted_pixels(&banded, &banded_rounds),
8924 painted_pixels(&one_page, &one_page_rounds),
8925 "a banded layer paints what one whole-layer page would have"
8926 );
8927 }
8928
8929 #[test]
8930 fn a_band_holds_no_instance_of_a_strip_outside_its_own_column() {
8931 // The counterexample the equality above generalizes: content confined
8932 // to the outer columns, and nothing at all in the middle. A band whose
8933 // column the scene never drew in must render nothing — under a
8934 // saturating shift it would render the leftmost content clamped onto
8935 // its own edge instead.
8936 let scene = band_scene(&[
8937 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8938 (Rect::new(160.0, 0.0, 200.0, 8.0), BLUE),
8939 ]);
8940 let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
8941 let bands = band_bounds(&rounds);
8942 assert!(bands.len() > 2, "the layer bands: {bands:?}");
8943
8944 let mut empty = 0_usize;
8945 for (plan, band) in scratch
8946 .rounds
8947 .iter()
8948 .zip(&rounds)
8949 .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
8950 {
8951 let strips = plan_strips(&scratch, plan);
8952 if strips.is_empty() {
8953 empty += 1;
8954 }
8955 for span in strips {
8956 let end = span.x + span.width;
8957 assert!(
8958 end <= band.width(),
8959 "an instance of {span:?} reaches past the {band:?} band's own \
8960 column, which its composite never samples"
8961 );
8962 // Mapped back to the scene, every instance lands where one of
8963 // the two rectangles was actually drawn.
8964 let x = band.x0 + span.x;
8965 assert!(
8966 x < 44 || band.x0 + end > 160,
8967 "an instance covers device x {x}..{}, which neither \
8968 rectangle reaches",
8969 band.x0 + end
8970 );
8971 }
8972 }
8973
8974 assert!(
8975 empty > 0,
8976 "a band whose column holds nothing renders nothing: {bands:?}"
8977 );
8978 }
8979
8980 #[test]
8981 fn a_strip_straddling_a_band_edge_advances_its_alpha_column_with_its_geometry() {
8982 // A pixel-aligned rectangle's left edge lands on a tile of its own, so
8983 // a rectangle starting at 49 puts an alpha-sampled instance across
8984 // 48..52 — and a band edge falls inside it. The two pieces together
8985 // have to read the same coverage the whole instance would: neighbouring
8986 // pixels of one strip row sampling neighbouring alpha columns, with no
8987 // column repeated. The rectangles either side of it are what carries
8988 // the layer past the ceiling, so the middle one is banded at all.
8989 let scene = band_scene(&[
8990 (Rect::new(0.0, 0.0, 40.0, 8.0), RED),
8991 (Rect::new(49.0, 0.0, 99.0, 8.0), BLUE),
8992 (Rect::new(160.0, 0.0, 200.0, 8.0), RED),
8993 ]);
8994 let (scratch, rounds) = band_plan_of(&scene, &BAND_PAGES);
8995 let edges: Vec<u16> = band_bounds(&rounds)
8996 .iter()
8997 .map(|bounds| bounds.x0)
8998 .collect();
8999
9000 // Every alpha-sampled pixel of every band, as (strip row, device x,
9001 // the alpha column it samples) — a row at a time, because two rows of
9002 // one draw sample different columns at the same x by construction.
9003 let mut dense: Vec<(u16, u16, u32)> = Vec::new();
9004 for (plan, band) in scratch
9005 .rounds
9006 .iter()
9007 .zip(&rounds)
9008 .filter_map(|(plan, round)| round.page().map(|page| (plan, page.bounds)))
9009 {
9010 for span in plan_strips(&scratch, plan) {
9011 for offset in 0..span.dense_width_or_rect_height {
9012 dense.push((
9013 band.y0 + span.y,
9014 band.x0 + span.x + offset,
9015 span.col_idx_or_rect_frac + u32::from(offset),
9016 ));
9017 }
9018 }
9019 }
9020 dense.sort_unstable();
9021
9022 let neighbours = || {
9023 dense
9024 .windows(2)
9025 .map(|pair| (pair[0], pair[1]))
9026 .filter(|((row, x, _), (next_row, next_x, _))| row == next_row && x + 1 == *next_x)
9027 };
9028 assert!(
9029 neighbours().any(|(_, (_, x, _))| edges.contains(&x)),
9030 "the case only means anything while an alpha-sampled instance \
9031 really is cut by a band edge {edges:?}: {dense:?}"
9032 );
9033
9034 for ((_, _, column), (_, x, next_column)) in neighbours() {
9035 assert_eq!(
9036 next_column,
9037 column + 1,
9038 "neighbouring pixels of one strip row sample neighbouring alpha \
9039 columns, band edge at {x} or not: {dense:?}"
9040 );
9041 }
9042 }
9043
9044 #[test]
9045 fn a_translucent_layer_over_a_clear_is_composited_after_the_punch() {
9046 let (scratch, rounds) = plan_of(&plan_scene(|builder| {
9047 plan_backdrop(builder);
9048 builder.clear_rect(PLAN_SLOT);
9049 builder.push_layer(PLAN_CHIP, 0.5);
9050 builder.fill_rect(PLAN_CHIP, Brush::Solid(BLUE));
9051 builder.pop_layer();
9052 }));
9053
9054 assert_eq!(
9055 scratch.rounds.len(),
9056 rounds.len() + 1,
9057 "the layer's page round, then the surface round cut in two"
9058 );
9059 let cut = scratch
9060 .rounds
9061 .iter()
9062 .position(|plan| plan.punch.1 > 0)
9063 .expect("the cut plan carries the punch");
9064 // The composite is what the cut has to fall before: a layer recorded
9065 // after the clear carries a deeper index than the punch, and the
9066 // composite writes no depth for the test to save it by.
9067 let composited = scratch.segments[scratch.rounds[cut + 1].segments.clone()]
9068 .iter()
9069 .any(|segment| matches!(segment, Segment::Composite(..)));
9070 assert!(
9071 composited,
9072 "the layer composites in the plan AFTER the punch, not before it"
9073 );
9074 assert!(
9075 !scratch.segments[scratch.rounds[cut].segments.clone()]
9076 .iter()
9077 .any(|segment| matches!(segment, Segment::Composite(..))),
9078 "and nothing composites into the plan the punch closes"
9079 );
9080 }
9081
9082 // -----------------------------------------------------------------
9083 // Backdrop blurs: the copy round, the sampled page, the opaque limit
9084 //
9085 // Host-only, over the same plan the frame path records from: a
9086 // backdrop's region copy is the one round besides a punch that must
9087 // follow the opening, and its rectangle the one strip run that samples
9088 // a page rather than a paint.
9089 // -----------------------------------------------------------------
9090
9091 /// The rectangle every backdrop case below blurs behind.
9092 const PLAN_PANEL: Rect = Rect::new(8.0, 8.0, 40.0, 36.0);
9093
9094 /// [`plan_of`] for a frame whose backdrops are served — scheduled through
9095 /// [`Schedule::build_frame`] and planned with the frame's own backdrop
9096 /// records, as `encode` does on a readable target.
9097 fn backdrop_plan_of(scene: &Scene, punches: bool) -> (Scratch, Vec<Round>, CompiledFrame) {
9098 let frame = SceneCompiler::new(PLAN_VIEWPORT.0, PLAN_VIEWPORT.1)
9099 .compile(scene, Affine::IDENTITY, PLAN_VIEWPORT)
9100 .expect("an in-range scene compiles");
9101 let rounds = Schedule::build_frame(
9102 &frame.recorder,
9103 &TierCaps::fake(DownlevelProfile::Full),
9104 &PageConfig::default(),
9105 &RetainedLayers::new(),
9106 &frame.backdrops,
9107 )
9108 .expect("a backdrop under the surface schedules");
9109 let mut scratch = Scratch::default();
9110 scratch.build(&frame, &rounds, true, punches, &[], &frame.backdrops);
9111 (scratch, rounds, frame)
9112 }
9113
9114 /// An opaque field, the backdrop, and a chip painted over the panel.
9115 fn backdrop_scene() -> Scene {
9116 plan_scene(|builder| {
9117 builder.fill_rect(
9118 Rect::new(
9119 0.0,
9120 0.0,
9121 f64::from(PLAN_VIEWPORT.0),
9122 f64::from(PLAN_VIEWPORT.1),
9123 ),
9124 Brush::Solid(BLUE),
9125 );
9126 builder.backdrop_blur(PLAN_PANEL, 2.0);
9127 builder.fill_rect(Rect::new(12.0, 12.0, 20.0, 20.0), Brush::Solid(RED));
9128 })
9129 }
9130
9131 #[test]
9132 fn a_backdrop_ends_the_opening_before_its_copy_and_draws_after_its_passes() {
9133 let (scratch, _, frame) = backdrop_plan_of(&backdrop_scene(), false);
9134 assert_eq!(frame.backdrops.len(), 1);
9135 let (merged, apart) = pass_lists(&scratch);
9136 // σ 2 convolves directly: one horizontal and one vertical pass.
9137 assert_eq!(
9138 merged,
9139 [
9140 FramePass::Opening,
9141 FramePass::Copy,
9142 FramePass::Filter,
9143 FramePass::Filter,
9144 FramePass::Surface,
9145 ],
9146 "the opening — clear, opaque strips and the field — lands before the copy reads \
9147 the surface back, and the panel and chip draw after the blur"
9148 );
9149 assert_eq!(
9150 apart,
9151 [
9152 FramePass::Clear,
9153 FramePass::Opaque,
9154 FramePass::Surface,
9155 FramePass::Copy,
9156 FramePass::Filter,
9157 FramePass::Filter,
9158 FramePass::Surface,
9159 ],
9160 "unmerged, the same order with the clear and the opaque strips apart"
9161 );
9162 }
9163
9164 #[test]
9165 fn a_backdrops_rectangle_samples_the_page_its_round_releases() {
9166 let (scratch, rounds, frame) = backdrop_plan_of(&backdrop_scene(), false);
9167 let backdrop = frame.backdrops[0];
9168 let sampling = rounds
9169 .iter()
9170 .position(|round| !round.samples.is_empty())
9171 .expect("one surface round samples the blurred page");
9172 let sample = rounds[sampling].samples[0];
9173 assert_eq!(sample.layer, backdrop.layer);
9174 assert!(
9175 rounds[sampling].released.contains(&sample.parity),
9176 "the round that samples the page hands it back"
9177 );
9178
9179 let plan = &scratch.rounds[sampling];
9180 let segments = &scratch.segments[plan.segments.clone()];
9181 let (first, count, parity) = segments
9182 .iter()
9183 .find_map(|segment| match *segment {
9184 Segment::Backdrop(first, count, parity) => Some((first, count, parity)),
9185 _ => None,
9186 })
9187 .expect("the panel is drawn as a backdrop segment");
9188 assert_eq!(parity, sample.parity);
9189 assert!(count > 0);
9190 for instance in &scratch.alpha[first as usize..(first + count) as usize] {
9191 assert_eq!(
9192 instance.paint_and_rect_flag & !crate::gpu::strips::RECT_STRIP_FLAG,
9193 LAYER_PAINT_SOURCE | u32::from(u8::MAX),
9194 "every panel instance reads the layer input at full opacity"
9195 );
9196 let (page_x, page_y) = (instance.payload & 0xffff, instance.payload >> 16);
9197 assert_eq!(
9198 (page_x, page_y),
9199 (
9200 u32::from(instance.x - backdrop.region.x0),
9201 u32::from(instance.y - backdrop.region.y0)
9202 ),
9203 "and samples the page texel under its own device origin"
9204 );
9205 }
9206 }
9207
9208 #[test]
9209 fn draws_after_a_backdrop_never_take_the_depth_writing_pass() {
9210 // The field recorded *after* the backdrop: its opaque interior must
9211 // not be drawn in the opening, ahead of the copy that reads the
9212 // surface back.
9213 let after = plan_scene(|builder| {
9214 builder.backdrop_blur(PLAN_PANEL, 2.0);
9215 builder.fill_rect(
9216 Rect::new(
9217 0.0,
9218 0.0,
9219 f64::from(PLAN_VIEWPORT.0),
9220 f64::from(PLAN_VIEWPORT.1),
9221 ),
9222 Brush::Solid(BLUE),
9223 );
9224 });
9225 let (scratch, _, _) = backdrop_plan_of(&after, false);
9226 assert!(
9227 scratch.opaque.is_empty(),
9228 "nothing at or past the backdrop's depth is routed opaque"
9229 );
9230
9231 // The same field before the backdrop keeps its opaque spans.
9232 let (before, _, _) = backdrop_plan_of(&backdrop_scene(), false);
9233 assert!(!before.opaque.is_empty());
9234 }
9235
9236 #[test]
9237 fn a_punch_ahead_of_a_backdrop_is_erased_before_the_copy() {
9238 let scene = plan_scene(|builder| {
9239 builder.fill_rect(PLAN_SLOT, Brush::Solid(BLUE));
9240 builder.clear_rect(PLAN_SLOT);
9241 builder.backdrop_blur(PLAN_PANEL, 2.0);
9242 });
9243 let (scratch, _, _) = backdrop_plan_of(&scene, true);
9244 let (merged, _) = pass_lists(&scratch);
9245 let punch = merged
9246 .iter()
9247 .position(|pass| *pass == FramePass::Punch)
9248 .expect("the punch is recorded");
9249 let copy = merged
9250 .iter()
9251 .position(|pass| *pass == FramePass::Copy)
9252 .expect("the copy is recorded");
9253 assert!(
9254 punch < copy,
9255 "the backdrop reads the erased surface, not the field the punch erases: {merged:?}"
9256 );
9257 }
9258
9259 #[test]
9260 fn a_target_that_cannot_be_read_back_serves_no_backdrop() {
9261 // `encode` hands the plan no backdrop records on such a target: the
9262 // panel is the transparent draw it was recorded as, scheduled as the
9263 // ordinary layer it is — no copy, no filter pass, no sampled page.
9264 let (scratch, rounds) = plan_of(&backdrop_scene());
9265 assert!(
9266 rounds
9267 .iter()
9268 .all(|round| round.copy.is_none() && round.samples.is_empty())
9269 );
9270 let (merged, _) = pass_lists(&scratch);
9271 assert_eq!(merged, [FramePass::Opening]);
9272 assert_eq!(scratch.copies, 0);
9273 }
9274
9275 #[test]
9276 fn the_region_copy_mode_prefers_sampling_and_needs_a_single_sample() {
9277 use wgpu::TextureFormat::{Bgra8Unorm, Rgba8Unorm, Rgba8UnormSrgb};
9278 use wgpu::TextureUsages as U;
9279 let attach = U::RENDER_ATTACHMENT;
9280 assert_eq!(
9281 RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8Unorm, 1),
9282 RegionCopyMode::Texture,
9283 "a headless readback target in the pages' own format is copied"
9284 );
9285 assert_eq!(
9286 RegionCopyMode::for_usage(attach | U::COPY_SRC | U::TEXTURE_BINDING, Rgba8Unorm, 1),
9287 RegionCopyMode::Fragment,
9288 "sampling costs the one pass alone, so it wins whenever it is allowed"
9289 );
9290 assert_eq!(
9291 RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING, Bgra8Unorm, 1),
9292 RegionCopyMode::Fragment,
9293 "any format can be sampled across into a page"
9294 );
9295 assert_eq!(
9296 RegionCopyMode::for_usage(attach | U::COPY_SRC, Rgba8UnormSrgb, 1),
9297 RegionCopyMode::Unavailable,
9298 "a copy would move encoded bytes into a linear page"
9299 );
9300 assert_eq!(
9301 RegionCopyMode::for_usage(attach, Bgra8Unorm, 1),
9302 RegionCopyMode::Unavailable,
9303 "a swapchain is RENDER_ATTACHMENT only"
9304 );
9305 assert_eq!(
9306 RegionCopyMode::for_usage(attach | U::TEXTURE_BINDING | U::COPY_SRC, Rgba8Unorm, 4),
9307 RegionCopyMode::Unavailable,
9308 "a multisampled target is neither"
9309 );
9310 }
9311}