# Performance improvement plan for v2.6.0
Goal: v2.6.0 is a performance release. This file is the backlog. It came
from two rounds of work on `main` at `0291d2a` (v2.5.7):
- Round 1: five code audits (render CPU, GPU passes and shaders, CPU
simulation, loading/startup/editor) and a measured baseline on an RTX 3060.
- Round 2: an Nsight Systems profile, a bisect of 8 history points, stress
and load measurements, and two more audits (uncovered subsystems and
codebase-wide patterns). See "Round 2 findings".
Line numbers are at that commit and drift as items land.
Item ID prefixes: M measure, G GPU physics and passes, R render CPU, S CPU
simulation, L load and editor (round 1); N profiler, B bisect, T measurement,
U subsystem audit, X codebase-wide audit (round 2).
Work it like `roadmap.md`: claim an item (`scripts/lane.sh claim "<ID>"`),
one item per commit, and record before/after numbers next to the checkbox.
## Ground rules
- Every item needs a before and after number from the same machine and
driver, or a test that proves the work went away (counter, call count,
allocation count). "Feels faster" is not evidence.
- Physics changes keep the AGENTS.md determinism rules. Hashes must stay
bit-identical, except where an item says it deliberately re-records them
once (S5).
- Bit-identical guards for simulation work:
- the `physics_suite::tests::regression_*` tests (one per scene since T13)
- `cpu_physics_is_deterministic_and_ignores_gpu_bodies`
- `cpu_physics_matches_bit_for_bit_whatever_the_render_frame_rate`
- `broad_phase_finds_the_same_pairs_as_testing_every_pair`
- `moving_part_of_a_hierarchy_matches_a_full_rebuild_bit_for_bit`
- the GPU determinism and tick-hash gpu-tests
- Render changes keep the golden-image gpu-tests unchanged unless the item
is a deliberate quality trade (G6).
- `scene_renderer.rs` is 27k lines. Keep render items in the render lane,
in sequence, to avoid merge conflicts.
- GPU benchmarks run through `flock /tmp/rusting-gpu.lock`, one at a time,
on an otherwise idle GPU.
## Baseline (RTX 3060, NVIDIA 615.78.08, 1080p, release, 2026-10-16)
Raw files from this run: the session scratchpad `perf/` folder. M1 moves
them into `benchmarks/`.
### render_bench (600 frames, p95 ms)
| Quality | Frame | CPU | GPU | Physics pass | Scene pass | Draws | Dispatches | Upload bytes |
|---|---|---|---|---|---|---|---|---|
| Eco, v1.4.0 stored baseline (driver 615.71.09) | 3.0 | 0.23 | 2.2 | 0.76 | 1.02 | 21 | 4 | 0.74 MB |
| Eco, now | 8.7 | 1.6 | 5.2 | 2.27 | 1.51 | 13 | 15 | 43.0 MB |
| Balanced, now | 11.5 | 1.75 | 6.2 | 2.33 | 1.64 | 19 | 16 | 43.0 MB |
| High, now | 11.7 | 1.72 | 6.4 | 2.35 | 1.66 | 23 | 16 | 43.0 MB |
On the Balanced and High rows, the passes not shown are SceneColor (0.27 ms)
and Transparent (0.57 ms).
**Correction from round 2: the stored v1.4.0 row is not a real baseline.**
The bisect rebuilt `5d9f7b4` (v1.4.0) and measured Physics 5.54 ms, 5
dispatches and frame p95 9.21 ms. The stored row says 0.76 ms and 4
dispatches, so it was recorded before that commit's final shaders. Its CPU
p95 of 0.23 ms does not reproduce either: the same commit measures 0.66 ms
today. Compare against the "now" rows, not against v1.4.0.
What really changed since v1.4.0 (bisect, Eco):
- Physics: 5.5 ms at v1.4.0 (correctly rounded `sim_math`), 23 ms at v2.0.2
(`CONTACT_ROUNDS = 4`), 1.6 ms after fb4f83d (oversized pass), 1.95 ms now.
Both cost increases were made for correctness.
- Upload: 0.74 MB to 43 MB came from clustered lighting (`5790146`), not from
instances. `--no-bodies` still uploads 71 KB a frame (T1). Moving instances
is a separate cost (T2).
- Triangles and draws follow features (point shadows, Eco light budget).
### physics_bench (windowed, GPU bodies, per 60 Hz tick)
The window runs at 600-900 fps with vsync off, so only about 1 frame in 10
runs a tick. Its printed "ms/frame" is mostly render-loop cost. The
numbers below come from `RUSTING_PERF=1` lines that contained a Physics
pass.
| Scene | Bodies | GPU Physics per tick | CPU fixed per tick | Extract per frame | First frame |
|---|---|---|---|---|---|
| falling, debris | 10,000 | 2.0-2.5 ms | ~0 | 0.6-0.8 ms | 2.5 s |
| debris | 100,000 | ~30 ms | ~2.4 ms | 4.8-8.7 ms | 13-17 s |
| mixed | 100,000 | ~35 ms | ~7.3 ms | 7.2-7.7 ms | |
- **GPU physics has a floor of about 2 ms per tick** that barely moves
from 1K to 10K bodies. This floor is the top GPU cost of the render
bench. Round 2 explained it (M4): the contacts dispatch is about 90% of
the pass, and it runs once per contact round (4 rounds, about 0.37 ms
each). Correctly rounded `sim_math` makes each round about 30% slower.
The dispatch count and the gaps between dispatches cost almost nothing.
- At 100K, each frame hits the cap of 8 ticks (`MAX_PHYSICS_STEPS_PER_FRAME`)
and takes about 250 ms. The game therefore runs in slow motion at about
0.5x real time. The slow-motion warning in `runtime/mod.rs` never printed.
- Per tick, 100K is close to the 34-42 ms the roadmap recorded. The 100K
problem is the per-tick cost multiplied by the cap, not a 100K regression.
- Rendering stays cheap at 100K: the Scene pass takes 0.6-1.9 ms for 100K
cubes.
- Extraction costs about 70 ns per entity, and it is the largest CPU phase
in a steady scene (R11, R13).
### Other findings from the measurement
- Eco mean frame time is 6.84 ms, but CPU plus GPU is only 4.92 ms. The
Nsight trace explains the missing 1.9 ms:
- The bench waits on a fence at the end of every frame
(`rendering/benchmark.rs:345-360`). Frame time is therefore CPU + GPU,
not the larger of the two.
- 0.6-1.0 ms of GPU work has no timestamp: render-pass clears and
resolves, layout transitions, and skinning and particle
`record_compute` (`scene_renderer.rs:4652-4661`).
- About 0.2 ms of CPU is outside `cpu_time`: `render_screens`,
`take_completed_*`, and flush and submit.
- Submit latency is 0.13-0.38 ms.
- The first frame takes 313 ms (one `vkAllocateMemory` of 251 ms), which
adds about 0.5 ms to the 600-frame mean.
- On High, `--no-bodies` leaves about 6 ms per frame that no measured phase
covers (T5).
- **GPU contention.** Jobs outside our lock (strata CUDA, llama-server,
Steam, firefox) kept the GPU 6-100% busy during round 2. Identical Eco
runs gave frame p95 6.33, 13.34 and 13.50 ms. Counters are exact; GPU
times are not. Record new baselines only on an idle GPU (T11).
### v2.6.0 acceptance gates
These are milestone gates, not tasks (roadmap "Performance numbers").
- render_bench Eco, RTX 3060: frame p95 ≤ 3.0 ms (from 8.7). Physics pass
≤ 0.5 ms (from 2.27). Upload ≤ 1 MB per run (from 43 MB). The upload
gate needs T1, because the bench camera moves every frame. Eco
`--no-bodies` already reaches 2.40 ms, so the frame gate depends mostly
on physics.
- render_bench High: frame p95 ≤ 4.0 ms (from 11.7).
- GPU physics: ≤ 0.5 ms per tick at 10K (from about 2.3 ms), and ≤ 8 ms per
tick at 100K (from about 30 ms), so 100K runs in real time at 60 Hz
without hitting the 8-tick cap. First frame at 100K ≤ 2 s (from 13-17 s).
- physics_suite (CPU solver): mean step at least 2x faster on every scene.
- Stress scenes from M3 get a baseline first. Each must then improve at
least 3x, or be shown to be already bound by the hardware.
- Editor on a 10K-entity scene:
- idle frame ≤ 4 ms
- undo ≤ 50 ms
- save ≤ 200 ms
- history memory ≤ 10x one snapshot
- Round 2 gates:
- Windowed 50K static entities: CPU ≤ 1 ms per frame (from 5.47 ms).
- render_bench `--bears 50000 --bear-updates 50`: upload ≤ 100 KB per frame
(from 12.3 MB).
- Peak RSS of render_bench Eco ≤ 400 MiB (from about 1.2 GB).
- `rusting validate` on 50K entities ≤ 700 ms (from 2.15 s).
- Engine library test suite ≤ 90 s wall time (from 209 s).
## Round 2 findings (2026-10-16)
Raw output is in the session scratchpad (`perf/nsys`, `perf/bisect`,
`perf/more`, `perf/audit`). M1 moves what is worth keeping into
`benchmarks/`. GPU times in this round were measured on a contended GPU, so
read them as ranges. Counters and CPU profiles are reliable.
### Where the CPU time goes (Nsight, frame-pointer build)
render_bench Eco, main thread, about 1.1 ms CPU per frame (inclusive %):
| % | Function | Item |
|---|---|---|
| 15.6 | `record_state_hash` (`state_hash.rs:354`) | S1 |
| 11.4 | vulkano `AutoCommandBufferBuilder::build` (auto-sync tracking) | N8 |
| 11.4 | `extract_render_world` (`render_world.rs:384`) | R13 |
| 7.8 | `QueryState::new`, about 35 per frame | R13 |
| 4.3 | `DescriptorSet::new` and update | X7 |
| 3.8 | `render_screens` with zero screens | N6 |
| 3.45 | `propagate_transforms` | S4 |
| 3.3 | `simple_gpu_physics_signature` | N5 |
| 2.5 | `step_cpu_physics` / `gather_bodies` with no CPU bodies | N4 |
| 2.2 | `renderables_changed` | R11 |
| 1.95 | `stream_gpu_bodies` | N3 |
| 1.8 | `PreparedGpuPhysics::clone`; `build_light_clusters` | R12; R10 |
| 1.6 | `step_soft_bodies` with no soft bodies | N2 |
| 1.5 | `update_skins` with no skins | N7 |
libc malloc, free and memcpy take 18% of the samples, and the Bevy
multithreaded executor about 17%.
physics_bench debris 100K, steady state, about 36 ms CPU per frame:
| ms/frame | Function | Item |
|---|---|---|
| 10.8 | `stream_gpu_bodies` sorts and writes every body, with no budget | N3 |
| 9.0 | `step_soft_bodies` builds and sorts a collider per body, with no soft bodies | N2 |
| 6.3 | `extract_render_world` | R13 |
| 4.2 | `gather_bodies` mirrors every GPU body into the CPU world | N4 |
| 4.7 | `simple_gpu_physics_signature` plus its hasher | N5 |
| 2.1 | `render_screens` clones `PreparedGpuPhysics` with zero screens | N6 |
| 1.7 | `renderables_changed` | R11 |
| 1.25 | `update_skins` with no skins | N7 |
The state hash is off in the game runner, so S1 does not apply here. Of
these, N2-N7 are about 30 ms per frame, and most are early-outs.
**The 100K spawn (13-17 s) is one O(n²) name check.** 98% of the first
frame is `find_named_entity` (`project_runner.rs:4127-4153`), called from
`spawn_renderable` (`:1060`). A new name always misses `SceneNameIndex`, and
the miss falls back to a scan of every `Name` (N1).
### GPU physics per dispatch (patched timestamps)
| Scene | Contacts per round | Oversized per round | Grid per round | Physics | Hash | Pass |
|---|---|---|---|---|---|---|
| Eco, about 1K bodies | 0.37-0.66 ms | 0.015-0.023 ms | 0.006-0.008 ms | 0.013-0.018 ms | 0.005 ms | 1.90-2.01 ms |
| debris 100K | 49.6-51.0 ms | 7.75-8.15 ms | 2.06-2.42 ms | 1.05 ms | 2.94 ms | 253.6 ms |
Experiments, all in scratch copies:
| Change | Result |
|---|---|
| Remove the two `atomicAdd` on `event_header` (`physics_contacts.comp:308-309`), 100K | contacts 50 to 33.6 ms per round; GPU frame 255 to 175.5 ms (-31%) |
| 1, 2, 4 contact rounds, Eco | 0.46, 0.81, 1.56-1.83 ms |
| Native `sim_recip`/`sim_rsqrt`/`sim_sqrt`, 4 rounds, Eco | 1.17-1.19 ms (-30%) |
| Native math, 1 round, Eco | 0.33 ms |
At HEAD the contact pass sees 71,512 hash-collision candidates a frame in
Eco, about 18 wasted neighbour tests per body per round (B4). Hardware GPU
counters need admin rights here (`RmProfilingAdminOnly`), so occupancy and
memory stalls are not split.
### Stress and scaling (render_bench, 1080p, 600 frames)
| Run | Frame mean | CPU mean | GPU p95 | Draws | Triangles | Upload per frame | Peak RSS |
|---|---|---|---|---|---|---|---|
| Eco | 5.33 ms | 0.77 ms | 4.39 ms | 13 | 142,768 | 72 KB | 1.2 GB |
| Eco `--no-bodies` | 1.85 ms | 0.43 ms | 1.20 ms | 13 | 112,960 | 71 KB | 1.16 GB |
| Eco 50K bears | 14.1-17.7 ms | 3.1-3.7 ms | 9.9-14.4 ms | 241 | 11.8M | 97 KB | 1.23 GB |
| Eco 50K, 1 or 50 or 5000 updates | 14.2-16.2 ms | 4.3-4.9 ms | 8.9-9.9 ms | 241 | 11.8M | 12.3 MB | 1.26 GB |
| Eco 100K, 100 updates | 20.5 ms | 7.0 ms | 11.3 ms | 241 | 16.8M | 24.3 MB | 1.28 GB |
| Eco `--screens 6` | 14.1 ms | 0.86 ms | 11.8 ms | 27 | 142,924 | 72 KB | 1.36 GB |
| High | 11.3 ms | 0.85 ms | 15.3 ms | 23 | 239,648 | 72 KB | 1.24 GB |
| High `--no-bodies` | 9.42 ms | 1.03 ms | 2.20 ms | 23 | 212,440 | 71 KB | 1.17 GB |
| High 50K bears | 82 ms | 4.1 ms | 92.6 ms | 376 | 25.8M | 97 KB | 1.39 GB |
- Each static bear costs about 47 ns of CPU.
- Moving any number of bears re-uploads all of them: 240 B times the
instance count, plus a flat 1.1 ms of CPU at 50K (T2).
- The High shadow pass at 50K bears took 56.2, 60.4, 5.5 and 3.2 ms in four
runs with identical counters (T4).
- `--screens 6` reports almost the same triangles as no screens, and
`cpu_time` misses the screen work (T6).
### CPU solver (physics_suite, mean step)
| Scene | Step | Scene | Step |
|---|---|---|---|
| stacking_stability (20) | 153.5 µs | character_stairs (10) | 86.3 µs |
| joint_chain (20) | 126.8 µs | **buoyancy_sort (16)** | **46.8 ms** |
| ccd_bullets (400) | 2.27 ms | sheet_catch (16) | 7.93 ms |
| cloth_hang (40) | 9.97 ms | **jelly_raft (16)** | **40.4 ms** |
| fluid_drop (1000) | 2.17 ms | stacking 2d (20) | 96.2 µs |
| vehicle_stress (25) | 127.3 µs | joint_chain 2d (20) | 230.3 µs |
| fracture_drop (50) | 1.09 ms | ccd_bullets 2d (400) | 2.23 ms |
| ragdoll_pile (100) | 1.69 ms | articulated_chain (20) | 591.4 µs |
stacking_stability shows a drift of 66.27 m (the stack falls over). This is a
known issue noted in its test.
### Startup, load and memory
| Measure | 10K entities | 50K entities |
|---|---|---|
| `validate` | 372.8 ms | 2.15 s |
| `lint` | 95.4 ms | 504.7 ms |
| `cook` | 461 ms | 2.41 s |
| warm `run --ticks 1` | 852 ms | 3.51 s |
| headless first playable frame | 107-118 ms | 485-494 ms |
| headless steady tick | 2.0 ms | 12.0 ms |
| windowed CPU p50 (0-552 triangles drawn) | 1.34 ms | 5.47 ms |
| headless RSS | 47 MiB | 137 MiB |
- A new project's first `rusting run` takes 131 s and 902 s of CPU to build
the engine. The first `--release` run adds 105 s. A scene-only edit
rebuilds the game in 7.1 s (T9).
- Any windowed run uses about 1.2 GB RSS, because each of about 9
`StandardMemoryAllocator`s maps 256 MiB blocks (T3).
- No leak was found over 90 s runs at 10K and 100K bodies.
- One test, `regression_scenes_repeat_their_recorded_results`, takes 191 s
of the 209 s library suite. Tests build at opt-level 0 (T13, X1).
## Phase 0: measure first (lane: tools, M4 in physics)
- [-] **M1 Refresh the baselines.** Store
`benchmarks/rtx3060-linux-{eco,balanced,high}.json` from a fresh run on
615.78.08. Add the table above to `benchmarks/README.md` and keep the
v1.4.0 row for history. Today the stored Eco baseline fails the compare.
Record on an idle GPU: stop strata, llama-server and games first, and note
the driver (B5). Mark the stored v1.4.0 row as not reproducible.
- Blocked (loop, 2026-10-17): the loop writes only to `target/perf-loop/`,
never `benchmarks/`, and needs an idle GPU; a game held it (39-98% busy)
all evening. The owner records these: `render_bench <quality> --repeat 3
benchmarks/rtx3060-linux-<quality>.json` with the old file moved away,
and checks that M2l prints no busy or spread warning.
- [x] **M2 Fix the benchmark tooling.** Split into M2a-M2n below, one
commit each.
- [x] **M2b** `physics_bench` reports ms per frame at 600-900 fps, where 9 frames in
10 run no tick. Report ms per tick: GPU Physics pass time divided by the
ticks run, plus CPU fixed time divided by the ticks run.
- [x] **M2a** The stored baseline can never be compared. The example always sets
`screen_size` to [320,180]. The old JSON has no such field and reads as
[0,0]. `regressions()` compares that field even with 0 screens, so every
compare fails with "does not match baseline".
- [x] **M2a** Times are compared only when the driver string matches exactly, so any
driver update hides timing drift. Keep the rule, but print the drift as
a warning.
- [x] **M2a** `render_bench` prints "Selected headless Vulkan device: ..." to stdout
before the JSON, so `> report.json` is invalid. Send that line to stderr.
- [x] **M2c** The windowed bench line counts extraction twice: `record_bench_work`
adds the update wall time, which already includes `render_extract`.
The line then claims "bound: cpu" with only 0.27 ms of GPU.
- [x] **M2d** Add the CPU phase split (fixed, update, post_update, extract, prepare,
record) to `RenderBenchmarkReport`.
- [x] **M2e** `gpu_memory_bytes` reads 1 GiB on every run: it measures allocator
blocks, not use. Report the bytes actually allocated.
- [x] **M2f** Print the slow-motion warning when the fixed tick hits the 8-step cap.
It did not print at 100K.
- [x] **M2g** Fix the stale `benchmarks/README.md`: it says 24 lights exceed Eco's
budget (the budget is now 256) and lists 21 draws (Eco now makes 13).
- [x] **M2h** Recook `testGame/build/main.rscene.bin`, or make physics_bench stop
loading testGame. Every run warns that the cooked file came from another
engine build.
- [x] **M2ia N10** Per-dispatch physics timestamps on the `RUSTING_PERF` line.
Raise `PART_MARKS` (`scene_renderer.rs:315`) from
64 and mark each physics dispatch (`:4426-4571`). The round-2 patch is
in the scratchpad `perf/nsys/src-patch`. Verify: a gpu-test that the
parts sum to the Physics pass.
- [x] **M2ib N10** The contact counters: pairs tested and contacts found per
tick, from a shader counter (an atomic counter is allowed; it does not
feed the simulation), on the `RUSTING_PERF` line.
- [x] **M2j T5** Add `untimed_ms` (frame minus measured phases) and timestamps
around the fence wait and the High-only passes. High `--no-bodies` has
about 6 ms per frame that no phase covers (`benchmark.rs:328-400`).
- [x] **M2k T6** `cpu_time` leaves out `render_screens` and the bear update, and
the counters leave out screen draws (`benchmark.rs:366-372`). Add
`screens_cpu_ms`, `bear_update_ms`, and screen draws and triangles.
- [x] **M2l T11** Record GPU busy state before and after the run
(`/sys/class/drm/*/device/gpu_busy_percent` or `nvidia-smi`). Report the
minimum and median of N repeats, and warn when the spread is over 20%.
- [x] **M2m T1 (counters part)** Split `upload_bytes` by category: instances,
clusters, uniforms, other. A unit test checks that they sum to the
total.
- [x] **M2n** render_bench enables `ext_debug_utils` in release whenever the driver
has it (`rendering/mod.rs:228-243`). Turn it on only with the
`validation` feature, so the bench measures what games run.
- M2n Done: `try_init_vulkan_headless` now enables `ext_debug_utils` only in debug builds or with the `validation` feature (`headless_debug_labels`), the same rule as the windowed instance, so release render_bench no longer records debug labels or object names. Debug test builds keep them, so the gpu-tests still run the label scopes. `render_bench eco --repeat 3`, a game on the GPU (39-92% busy) in both sets: before (`target/perf-loop/m2n-before.*`), record 0.220 ms, CPU 0.595 ms, frame median 5.72 ms (spread 23%); after (`target/perf-loop/m2n-after.*`), record 0.208 ms (-5%), CPU 0.568 ms (-5%), frame median 5.40 ms (spread 6%). The frame change is within the GPU noise; the CPU change is the labels. No new test: a test build is a debug build, so it cannot see the release choice. Full check passed (2684 tests).
- M2m Done: `RenderCounters` gains `upload_split` (`UploadSplit`: instances, clusters, uniforms, physics, assets, other); all 30 upload sites in `scene_renderer.rs` now count through `RenderCounters::upload(kind, bytes)`, which adds to the part and the total. Physics and assets got their own parts so `other` stays small. render_bench reports `upload_split_bytes`, totals over the run. `render_bench eco --repeat 3`: `upload_bytes` 43,004,456 before and after (the split does not change what is uploaded); after (`target/perf-loop/m2m-after.{json,err}`): clusters 40,312,096 (93.7%, 67 KB per frame), uniforms 1,970,000 (4.6%), instances 380,948 (0.9%), physics 291,392 (0.7%), assets 49,892, other 128. The light cluster list is nearly all of the per-frame upload, but at 67 KB a frame it is under the 1 MB sweep bar. Times were not compared: a game held the GPU (36-98% busy, the M2l warnings fired). Tests: the gpu-tests `frames_record_the_declared_passes_with_their_layouts` and `benchmark_records_every_metric_along_the_camera_path` check that the parts sum to `upload_bytes`, so an upload site without a part fails them; a no-GPU unit test could only check the adder, so none was added. Full check passed (2684 tests).
- M2l Done: render_bench records the busiest GPU's load before and after each run (`gpu_busy_percent`, from the DRM `gpu_busy_percent` files and `nvidia-smi`; the value after includes this run's last second) and warns when the GPU is over 10% busy before a run. `--repeat N` runs N times, prints the minimum and median mean frame time and their spread, warns when the spread is over 20%, and keeps the median run as the report, with every repeat's mean in `repeat_frame_means_ms`. Before: sweep 2 runs with another engine (RustingStrata) on the GPU came out about 8x slow with no sign in the report. After, `render_bench eco --repeat 3` with RustingStrata benchmarking on the GPU (`target/perf-loop/m2l-after.{json,err}`): `warning: the GPU was 44% busy before the run` (then 86% and 93%), `3 runs: mean frame min 4.37 ms, median 10.89 ms, spread 185%` and the spread warning. Test: `repeats_report_minimum_median_and_spread` (minimum, lower-middle median and spread for odd, even and single counts). Full check passed (2684 tests).
- M2k Done: render_bench `cpu_phases_ms` gains `bears` (the bear transform update) and `screens` (CPU wall time of `render_screens`) before `wait`; `untimed` now leaves them out. The report gains `screen_draws` and `screen_triangles` (most per frame, summed over the feed renderers by `SceneRenderer::screen_counters`); `draws` and `triangles` still mean the main view, and `cpu_time` keeps its definition so baselines still compare. `render_bench eco --screens 6 --bears 5000 --bear-updates 500`, median of 3: before, frame 14.6 ms, `untimed` 4.5 ms, 72 draws, no screen counts. After, frame 15.05 ms: `bears` 0.01, `screens` 5.20, `wait` 7.83, `untimed` 0.11; 399 screen draws and 10.9M screen triangles against 72 draws and 2.7M triangles in the main view. So the six feeds cost about 3x the main view's CPU (record 0.53 ms) and 4x its triangles: candidate for sweep 3. Outputs `target/perf-loop/m2k-{before,after}-*.json`. Test: `benchmark_counts_screen_feeds_and_bears_apart` (gpu-tests) runs 2 screens and 8 moving bears and checks screen draws, screen triangles and both phases are above zero. Full check passed (2682 tests).
- M2j Done: render_bench `cpu_phases_ms` gains `wait` (submit plus fence wait) and `untimed` (frame time minus every phase and the wait), so the phases now add up to `frame_time`. The High-only passes (`SceneColor`, `Transparent`) already had pass timestamps, so no new GPU marks were needed. `render_bench high --no-bodies`, median of 3 (one run of each set doubled under another engine's GPU load and is excluded): before, frame 3.07 ms, CPU 0.40, GPU passes 1.94, so 0.73 ms was unaccounted (the 6 ms gap in T5 is gone since the M2c-M2e fixes). After, frame 3.11 ms: phases 0.41, `wait` 2.60, `untimed` 0.02. The gap sits in the wait: 0.66 ms of submit and GPU time outside the timed passes. Outputs `target/perf-loop/m2j-{before,after}-*.json`. Test: `benchmark_records_every_metric_along_the_camera_path` checks the new phase names, that each is above zero, and that the phases sum to the mean frame time within 0.01 ms. Full check passed (2681 tests).
- M2ib Done: `physics_contacts.comp` already counts grid overflow, oversized bodies, hash collisions and fallback tests into `event_header.contacts`, and `RenderCapacityDiagnostics` already reads them back (tested by the fallback gpu-test), so no shader change was needed. The `RUSTING_PERF` line now prints them after the per-tick physics times. physics_bench falling 100000 (`target/perf-loop/m2ib-100000.out`): `contacts grid overflow 0 oversized 32 hash collisions 88824640 fallback tests 6400032` per frame (8 ticks x 4 rounds), with `Physics 216.45 ... contacts 178.53 oversized 27.95`. That is about 28 hash-collision visits per body per round. (Corrected in sweep 2: `contact_hash_budget` is about 512 MB here and does not cap the table; it has 2 cells per body. See P2.) At 10K: 993704 hash collisions, 80004 fallback tests. Candidate for sweep 2: a bigger contact hash table. No new test: the line only formats counters that a gpu-test already covers. Full check passed (2681 tests).
- M2ia Done: `PART_MARKS` 64 to 256 (8 ticks of 4 rounds x 3 dispatches plus step, hash and readback fit), and the physics pass marks `grid` (grid clear and build), `contacts`, `oversized`, `step`, `custom`, `hash` and `readback`, summed over rounds and ticks. The `RUSTING_PERF` line prints them after `Physics`. physics_bench falling 100000 (`target/perf-loop/m2i-100000.out`): `Physics 445.40 grid 9.02 contacts 365.98 oversized 67.69 step 1.01 hash 1.70`, so the contact pass is 82% of physics and the solver step 0.2%. At 10K: `Physics 4.48 grid 0.09 contacts 3.98 oversized 0.38 step 0.02 hash 0.01`. The parts sum to the pass within 0.01 ms. Another engine (RustingStrata) held the GPU during the run, so treat these times as shares, not absolutes. Test: `physics_events_read_back_from_transient_arenas_every_tick` now asserts that each tick has the parts `grid, contacts, oversized, step, hash` and that they sum to at most the Physics pass (a forced wrong name made it fail). The contact counters are split off as M2ib. Full check passed (2681 tests).
- M2h Done: physics_bench now loads the tracked source scene `testGame/scenes/main.rscene`, not the gitignored cooked `testGame/build/main.rscene.bin`. Before: without the cooked file the bench exits with `Error: Io(... NotFound ...)`, and a cooked file from another engine build warns every run. After: `physics_bench falling 10000` runs with the cooked file removed and prints no warning. First playable frame, median of 3: 461 ms before (cooked), 497 ms after (source parse, +36 ms). No test: the change is the scene path of one example. The other nine examples that load testGame still use the cooked file. Full check passed (2681 tests).
- M2g Done: `benchmarks/README.md` now says the 24 point lights fit every budget (Eco 256), adds v2.5.7 rows from sweep 1 (Eco 4.5/0.55/3.8 ms, High 6.1/0.68/4.8 ms, 0 dropped lights; 13 and 23 draws), keeps the v1.4.0 rows labelled as such (Eco then took 16 lights), documents the `times not compared` and `warning:` output from M2a, and says the stored baseline predates the current engine (M1 re-records it). Docs only; no code changed since the full check of b664a6d, and no test reads the README.
- M2f Done: `physics_bench falling 100000`, release, 40 s, `RUSTING_PERF=1`: before, no warning (frames of 218 ms run the 8-step cap, but CPU fixed is 18-19 ms, under the 133 ms simulated, so `slow` never held). After: `warning: frames run 8 fixed steps (CPU 19ms) and still fall behind the 133ms they simulate; the game now runs in slow motion.` printed once. The warning now also fires after 4 frames in a row that hit the cap with a step still owed; stepping is unchanged (the slow-motion clamp still needs slow CPU steps). Test `capped_frames_that_fall_behind_warn_even_with_fast_steps`: one hitch does not warn, sustained capped frames do. Log: `target/perf-loop/m2f-after.log`.
- M2e Done: `render_bench eco` `gpu_memory_bytes` was 1073741824
(1 GiB of reserved blocks) every run. After: 103.5 MiB suballocated,
on Eco and High alike. Large images that get dedicated allocations are
still not counted (existing ponytail note). Test:
`frames_record_the_declared_passes_with_their_layouts` (gpu-tests,
in-use below reserved).
- M2d Done: `render_bench eco` reports had no phase split. After,
`cpu_phases_ms` (mean ms): fixed 0.179, update 0.022, post_update
0.049, extract 0.099, prepare 0.092, record 0.200; fixed + extract +
prepare + record = 0.570, the `cpu_time` mean. Test:
`benchmark_records_every_metric_along_the_camera_path` (gpu-tests).
- M2c Done: `RUSTING_BENCH_FRAMES=600 physics_bench falling 10000`,
medians of 3 (strata CUDA job on the GPU): `cpu_p50_ms` 1.15 before,
above the 0.83 ms frame p50, so impossible; 0.69 after, below the
0.88 ms frame p50. GPU p50 0.25 ms, so "bound: cpu" is now real. Test:
`bench_cpu_time_counts_extraction_once`.
- M2b Done: `RUSTING_PERF=1 RUSTING_QUIT_AFTER_MS=10000 physics_bench
falling 10000` before printed only per-frame times (GPU 0.25 ms a frame
at 1130 fps, since 19 frames in 20 run no tick). After, every line adds
`per tick: CPU fixed 0.26 GPU Physics 1.92 ms (60 ticks, 60 timed)`.
`SceneRenderer::take_physics_tick_totals` sums the Physics pass over
every completed frame. Test:
`physics_events_read_back_from_transient_arenas_every_tick` (gpu-tests).
The lane had no `testGame/build/main.rscene.bin`; `rusting cook
testGame` made it (gitignored).
- M2a Done: `render_bench eco benchmarks/rtx3060-linux-eco.json` before
printed only "run [1920, 1080] Eco 600 frames does not match baseline"
and stdout started with the device line. After: stdout parses as JSON,
the compare runs (dispatches 4 to 15, triangles, upload 0.74 to 43 MB:
the real regressions M1 re-records), and the driver change
615.71.09/615.78.08 prints the time drift as warnings (frame p95 3.04
to 5.15 ms, GPU 2.20 to 4.16 ms). Test:
`regressions_flag_material_growth_only` (screen size without screens,
`same_machine`, `time_growth`).
- [x] **M3 Add stress modes, then baseline them.** The current bench has
2-3 meshes, 5 untextured materials, about 1K instances (almost all GPU
bodies), 24 lights and one shadow. It cannot show per-draw cost, texture
bandwidth, extraction at scale or light scaling. Add:
- **City:** 10K static objects from 1-2K meshes, 200 materials over 64
1K textures, LOD groups, and a street-level camera with heavy
occlusion. Optionally a 100K-instance mode with 50 meshes.
- **Night street:** 200-500 point and spot lights, 8 shadowed point lights
and 4 shadowed spots.
- **Effects:** 50K GPU particles in 20 blended emitters, 2K transparent
instances, 300 skinned characters.
- The smaller modes below:
- render_bench `--bears 50000 --bear-updates 50` (the flags exist):
exercises instance upload, extract and culling.
- render_bench on High with a `LightProbeGrid`: covers R1.
- render_bench with 256 point lights: covers clustering.
- render_bench with 50K particles, plus a heightfield with blockers:
covers S3.
- `physics_suite` stored as a JSON baseline with the same regression rules
as render_bench.
- An editor timing test on a generated 10K-entity scene. It records load,
save, one inspector edit plus undo, the idle frame time
(`CpuFrameTimings.editor`), and history bytes after 100 edits.
- A load test: 50 textures and 200 instances of one prefab, timed with
`FIRST_FRAME_MARKER`.
Split into M3a-M3h; each one adds a mode and its measured numbers. The
baselines in `benchmarks/` stay with the owner (see M1).
- [x] **M3a** Night street: render_bench `--lights N`.
- [x] **M3b** City.
- [x] **M3c** Effects (particles, transparent instances, skinned).
- [x] **M3d** High with a `LightProbeGrid` (R1).
- [x] **M3e** 50K particles plus a heightfield with blockers (S3).
- [x] **M3f** `physics_suite` JSON baseline with the render_bench rules.
- [x] **M3g** Editor timing test on a generated 10K-entity scene.
- [x] **M3h** Load test: 50 textures, 200 prefab instances.
- M3a Done: `RenderBenchmarkExtras::lights` (render_bench `--lights N`, report field `lights`) adds N lights on a grid 3 m up over the 72 m floor, alternating point lights and spots facing down; the first 8 points and 4 spots ask for shadows (the frame still shadows one light, and the sun wins). The `--bears 50000` mode already existed. `render_bench eco --lights N --repeat 3`, a game on the GPU (36-95% busy) in every set, medians, spread 1-2%: N=0 (`target/perf-loop/m3a-lights0.*`) frame 5.13 ms, GPU 3.49, Scene 1.51, prepare 0.10, cluster upload 67 KB a frame, 0 dropped lights. N=256: frame 9.37 ms, GPU 7.10, Scene 5.01 (3.3x), prepare 0.27 (2.6x), cluster upload 375 KB a frame (5.6x), 25 lights dropped over the Eco budget of 256. N=500: frame 9.28 ms, Scene 4.86, 269 dropped, so the budget caps the cost. The Scene pass grows about 14 us per clustered light. Test: `extra_lights_alternate_points_and_spots_with_few_shadows` (151 points, 150 spots, 8 and 4 shadowed for N=301). Candidate for sweep 3: Scene light loop cost and the per-frame cluster upload of an unchanged light set. Full check passed (2686 tests).
- M3b Done: `RenderBenchmarkExtras::city` (render_bench `--city N`, report field `city`) spawns N static buildings in blocks of 5 by 5 lots with 8 m streets. `render_benchmark_city_assets` gives 1000 building meshes, each a scaled cylinder with a box LOD level from 60 m, 200 materials and 64 1024x1024 facade textures at 10K buildings (fewer below). Building `i` uses mesh `i % 1000` and material `i % 200`, so 1000 mesh-material pairs. With a city the camera walks a square of streets at 1.7 m, facing the way it goes (`render_benchmark_city_camera`, closed). The optional 100K-instance mode is not added; `--city 100000` gives 100K buildings over the same 1000 meshes. `render_bench eco --city N --repeat 3`, a game on the GPU (39-100% busy), medians: N=0 (`target/perf-loop/m3b-city0.*`) frame 4.75 ms, CPU 0.60, GPU 3.18, record 0.22 ms, 13 draws, 1079 visible instances, GPU memory 113 MB. N=10000 (spread 17%): frame 39.15 ms, CPU 23.46, GPU 12.75 (Scene 6.61, Shadow 0.14), record 22.40 ms, 2214 draws, 5989 visible instances, 178K triangles, GPU memory 502 MB, asset upload 284 MB on the first frames, instance upload 27.5 MB over the run (46 KB a frame for static buildings). About 10 us of CPU record per draw: candidate for the next sweep. Occlusion barely helps: 60% of buildings stay visible from street level. Test: `city_shares_lod_meshes_and_textures_and_its_streets_are_clear` (600 buildings: 60 meshes with LOD groups, 12 materials, 3 textures; closed walk at eye height that never enters a lot). Full check passed (2689 tests).
- M3c Done: `RenderBenchmarkExtras::effects` (render_bench `--effects`, report field `effects`) adds `RENDER_BENCHMARK_EFFECTS`: 50K GPU particles in 20 prewarmed emitters on a 10 m ring (alternating alpha and additive, 2500 each), 2000 see-through spheres (4 `AlphaMode::Blend` materials) 6 m up, and 300 skinned 1.8 m characters, each bending at the hips by an autoplayed looping clip on `Hip/Spine`, deformed on the GPU (`GpuSkinning`) from cooked two-joint weights. `render_bench eco --repeat 3`, GPU 31-81% busy from other processes, medians: plain (`target/perf-loop/m3c-plain.*`) frame 3.62 ms, CPU 0.55, GPU 2.85 (Physics 1.51, Scene 1.27), record 0.20 ms, 13 draws, 1079 visible instances, GPU memory 113 MB, upload 43 MB over the run. Effects (`m3c-effects.*`, spread 10%): frame 49.83 ms, CPU 28.46, GPU 18.28 (Scene 11.90, Physics with the particle step 5.66, Shadow 0.41, ToneMap 0.29), record 26.97 ms, prepare 0.70 ms, 2933 draws, 3379 visible instances, 870K triangles, GPU memory 130 MB, upload 110 MB over the run. Each blended sphere and each skinned copy is its own draw: about 9 us of CPU record per draw, as in M3b; candidate for the next sweep. Test: `effects_fill_gpu_emitters_and_skin_animated_characters` (20 GPU emitters totalling 50K, 2000 blended renderers, 300 skin jobs with a bent spine after 10 ticks). Full check passed (2691 tests; fmt fixed and rechecked, clippy rerun in all three feature sets).
- M3d Done: `RenderBenchmarkExtras::probes` (render_bench `--probes`, report field `probes`) adds one `LightProbeGrid` of `RENDER_BENCHMARK_PROBES` = 4x4x4 (the renderer's cap of 64) over the scene, extents 24x6x24 m round (0, 6, 0). `render_bench high --repeat 3`, GPU 58-82% busy from other processes, medians: plain (`target/perf-loop/m3d-plain.*`) frame 4.97 ms (one repeat 9.10 ms, noise), CPU 0.67, GPU 3.77, record 0.30, untimed 0.01 ms, 23 draws. Probes (`m3d-probes.*`, repeats 11.54-12.01): frame 11.67 ms (+6.70 ms, +135%), CPU 0.63, GPU 3.86, GPU passes unchanged, untimed 6.71 ms, GPU memory +3.1 MB. The whole cost is `relight_light_probes` (two probes, 12 cube faces, every High frame), called before the preparation timer so it shows only as untimed: this is the R1a/R1b target, and R1a's gate measures against it. Test: `probes_add_one_full_grid_over_the_scene`. Full check passed (2693 tests).
- M3e Done: `RenderBenchmarkExtras::rain` (render_bench `--rain`, report field `rain`) adds `RENDER_BENCHMARK_RAIN`: 16 CPU emitters with `blockers` on, 3125 drops each (50K cap), falling from 20 m over a 4x4 grid onto a drawn 257x257 `Heightfield` (64 m, low hills) under 16 roofs that are `ParticleBlocker`s. `render_bench eco --repeat 3` medians: plain (`target/perf-loop/m3e-plain.*`) frame 3.74 ms, CPU 0.68, GPU 2.71, fixed 0.21, record 0.25 ms, 13 draws, upload 43 MB over the run. Rain (`m3e-rain.*`, repeats 14.25-20.88 ms, GPU 30% then 99% busy from another process): frame 19.04 ms, CPU 12.26, GPU 6.18 (Scene 3.55, Physics 2.49), fixed 8.23 ms (the CPU particle step with blockers: the S3 target), extract 0.90, record 3.01 ms, 32 draws, 632K triangles, GPU memory 133 MB, upload 1.35 GB over the run (2.2 MB a frame of CPU particle instances: over the 1 MB a frame bar, candidate for the next sweep). A debug-build test that stepped the rain 180 ticks took 118 s, so the test only checks the scene. Test: `rain_comes_from_blocked_cpu_emitters_over_ground_and_roofs`. Full check passed (2695 tests).
- M3f Done: `SuiteReport` (`runtime/physics_suite.rs`: the CPU name plus each scene's name, size, steps, mean step and error) serializes to JSON, and `physics_suite <path>` writes it as the baseline when the file is missing or compares against it and exits 1 on a regression, like render_bench. `SuiteReport::regressions` reports a changed scene list; an error growing past `REGRESSION_TOLERANCE` (10%) and `SUITE_ERROR_NOISE` (1e-4); and, only on the same CPU, a step time growing past 10% and `REGRESSION_NOISE_MS` (0.5 ms). A run takes 29.6 s; the first wrote `target/perf-loop/m3f.json`, and the next two found no regressions. Against a copy with cloth_hang's error and jelly_raft's step halved (`m3f-tight.json`) it exits 1 on both. That run, with the host loaded (load average 6.5), also flagged sheet_catch at 7.11 to 8.56 ms and ccd_bullets 2d at 2.06 to 8.02 ms, so like render_bench it wants a quiet machine. Errors match sweep 5 exactly. Test: `suite_report_flags_error_growth_and_same_machine_step_growth`. Full check passed (2697 tests).
- M3g Done: `editor_timings(count)` (`editor/tests.rs`) builds an editor (asset, extract and editor plugins) with `count` saved built-in cubes on a grid, then times `save_editor_scene`, `load_editor_scene`, the median of 20 idle frames (`draw_editor_view` plus tessellation, the span `CpuFrameTimings::editor` covers in the binary), one Inspector edit (its snapshot, a Transform write and `push_undo`, called directly rather than through a drag), the frame that runs the queued `EditorAction::Undo`, and the JSON size of the Undo snapshots after 100 more edits. The 10K run is an ignored test (`cargo test --release --lib editor_timings_on_10k -- --ignored --nocapture`); 3 runs (`target/perf-loop/m3g-*.out`, load average 2.7), medians, spread under 4%: save 44.1 ms, load 34.5 ms, idle frame 2.93 ms, edit 8.61 ms, Undo frame 18.73 ms, history 499.7 MB of JSON (about 5.0 MB per snapshot, so 100x one snapshot). Against the goals: idle frame (4 ms), Undo (50 ms) and save (200 ms) pass; history memory is 10x over its 10x-one-snapshot goal, the L14a target. Test: `editor_timings_cover_save_load_edit_undo_and_history` (50 entities, checks the Undo restores the position and the history holds 100 steps). Full check passed (2699 tests).
- M3h Done: `write_load_test_project(folder, textures, instances)` (`project_runner.rs` tests) writes a project: `textures` 1024x1024 PNG files (64 MB for 50) on as many cubes, a prefab `assets/crate.rscene` of two textured cubes, `instances` copies of it in `scenes/main.rscene`, and the cooked `build/main.rscene.bin`. The ignored test `write_m3h_load_test_project` writes the 50-texture, 200-copy project to `target/perf-loop/m3h`. Timed with `RUSTING_HEADLESS_TICKS=1 target/release/game <scene>`, which prints `FIRST_FRAME_MARKER` (`target/perf-loop/m3h.out`, load average 6.1, a GPU job running): cooked 330, 320, 320 ms, median 320 ms; source scene 326, 326, 328 ms, median 326 ms, so parsing the text scene and its 200 instance links adds about 6 ms. `game --headless` calls `run_project_headless` without starting the timer, so its marker reads about 0 ms; the env route goes through `run_project`. Candidate for the next sweep: the 50 decodes run one after another on the loading thread. Test: `load_test_project_places_its_textures_and_prefab_copies` (3 textures and 4 copies load as 4 textures and 11 mesh renderers). Full check passed (2701 tests; clippy precedence and fmt fixed, rechecked).
- [x] **M4 Bisect the GPU physics floor.** Done in round 2 (bisect of 8
history points plus 3 experiments, render_bench Eco, medians of 5).
- `5d9f7b4` (v1.4.0) made `sim_recip`, `sim_rsqrt` and `sim_sqrt`
correctly rounded: 0.92 ms with native math against 5.54 ms. This is
needed for `DeterminismMode::CrossPlatform`.
- `8aa76fc` (v2.0.2) added `CONTACT_ROUNDS = 4`: 5.7 to 23 ms. This is
needed: in the deep ball pit, rest went from 646 to 3,045 balls and
escapes from 17 to 0.
- `fb4f83d` (oversized pass) is the fix: 23 to 1.6 ms. HEAD is 1.95 ms,
flat within noise.
- The `fill_buffer` calls and dispatch gaps cost almost nothing.
- Follow-up items: B1, B2, B3, B4, G1, N9.
- [x] **M5 Explain the 100K CPU cost.** Done in round 2 (Nsight).
- The spawn is the O(n²) name check (N1).
- The fixed tick is N2 + N3 + N4, not the state hash, which is off in the
game runner.
- Extraction is R13, N5 and N6.
## Phase 1: quick wins (small effort, large gain)
### GPU physics (lane: physics; shader files plus the physics block of `scene_renderer.rs`)
- [ ] **G4a Static bodies cause no oversized pass.** When appending
`big_bodies`, skip Fixed bodies (inverse mass 0)
(`physics_contacts.comp:86`) and keep the "touched" semantics. In the bench
every body tests the 80 m ground, so every thread takes the fallback path
in all 4 rounds. Verify: Physics pass time; determinism gpu-tests.
- [ ] **G1 Physics counters off PCIe.**
- The problem: `event_header`, `tick_hashes` and `events` live in the
host-random `transient` arena (`scene_renderer.rs:2206`, `:4126-4214`).
Physics then does more than 6000 atomics a tick on single addresses over
PCIe (`physics_contacts.comp:84,95,272,308-309`,
`physics_hash.comp:58-59`, `physics_abi.glsl:89`).
- Fix part 1: reduce the counters per workgroup with `subgroupAdd` or
shared memory, then do one atomic per group.
- Fix part 2: keep the buffers device-local, plus one copy to a readback
buffer.
- Integer sums are order-free, so this is deterministic.
- Round 2 measured it (N9): removing only the two `atomicAdd` at
`physics_contacts.comp:308-309` cut the GPU frame at 100K from 255 to
175.5 ms (-31%). Every body runs them, because the fixed oversized
bodies put every body in the fallback list.
- Also drop the `cell_of` reload in `touch_in_cell` (`:180-183`). It
exists only for the hash-collision counter. Put the debug counters
behind a define.
- [ ] **B1 Compute loop-invariant `sim_math` once per thread.** Same
functions on the same inputs give the same bits, so CrossPlatform
determinism holds.
- The problem: in `physics_contacts.comp`, each candidate recomputes
`sim_recip(pc.grid_cell_size)` in `cell_of` (l.54), `bounding_radius` of
both bodies (`physics_shapes.glsl:29`; a `sim_sqrt` for boxes),
`inverse_mass(other)` and `sim_normalize(gravity)`.
- Fix: compute the inverse cell size, own radius and gravity direction
once per invocation. Have the grid pass store each body's radius and
inverse mass next to the snapshot.
- Expected: 0.2-0.4 ms at 4 rounds in Eco.
- Verify: `shader_sim_math_matches_the_rust_reference_bit_for_bit`,
`benchmark_scenes_run_on_the_gpu_and_repeat_exactly` and
`gpu_bodies_collide_with_each_other_through_grid_and_fallback` give
byte-identical poses.
- [ ] **B4 Bigger contact hash table.** `PhysicsContactGrid::hash_cells`
(`scene_renderer.rs:1361`) is `capacity * 2`, and Eco sees 71.5K
collision candidates a frame. Try `capacity * 8`, still capped by the
budget. Grid size does not change results (roadmap 5020), so poses must
stay identical, and `physics_grid_hash_collisions` must drop.
- [ ] **G3 Workgroup size 64** for the grid, contact, physics and hash
passes. 1000 bodies currently fill 4 of 28 SMs. Keep 256 for
OVERSIZED_PASS, which reduces in shared memory.
- [ ] **G5 Drop `visited[27]`** in `physics_contacts.comp:290-300`. It is a
dynamically indexed array, so it spills to local memory, and its scan is
O(27²). When walking cell c, skip entries whose own cell hash is not c.
The same item covers `bool stack[64]` in `physics.comp:195`: replace it
with a bitmask.
### Render (lane: render)
- [ ] **R1a High-quality probe relight only on change.**
- The problem: `relight_light_probes` runs every High frame
(`scene_renderer.rs:2938`). It clones the whole `RenderWorld` (`:6929`),
renders 12 cube faces with a `wait(None)` after each, then calls
`submit_and_wait` (`:6989-7001`). That is 13 CPU-GPU stalls a frame.
- Fix: key the relight on (lights_revision, renderables_revision,
sky/ambient). Stop when every probe is fresh.
- Test: a static scene relights 0 probes after the first sweep.
- [ ] **R2a/b Shadows skip non-casters and whole batches.**
- The problem: the `keep` closure (`:4760`) ignores the cast bit, so
non-casters still run the shadow vertex shader. Also, every instance is
frustum-tested once per cascade.
- Fix: return `None` when the cast bit is clear. Test the batch AABB per
cascade before testing instances.
- [ ] **R8/G2/G11 Per-frame GPU-read data into BAR memory.**
- The problem: the `transient` arena is `PREFER_HOST |
HOST_RANDOM_ACCESS` (`:2199-2210`). It holds data the GPU reads:
- `shadow_upload` and the light clusters, read per pixel by
`scene.frag:49,86`
- decals and probes
- sprite, particle and debug vertices
- skinning and cloth inputs
- occlusion parameters
- Fix part 1: give that arena `PREFER_DEVICE | HOST_SEQUENTIAL_WRITE`,
like `instance_allocator` (`:2154`). Keep a small host-random arena only
for readbacks.
- Fix part 2: declare `Shadow` as a `uniform` block.
- [ ] **R4 Material group computed once per material**, not inside the sort
comparator. `material_group` hashes the shader string per compare
(`:8798`, `:8822-8843`).
- [ ] **R5 Hash bits, not Debug strings.** `render_world.rs:1198` uses
`format!("{bounds:?}{foliage:?}")` per entity on every rebuild.
- [ ] **R6 Cheap `lightmap_key`.** `:9913` hashes every renderable and
`format!`s all lights every frame. Key it on the existing revisions.
- [ ] **R7 Foliage flag per batch.** `foliage_materials()` builds a HashSet
every frame only to label timings (`:4871`, `render_world.rs:303`).
- [ ] **R9 Gate `prepare_materials` on revisions, and build decals once.**
It runs every frame (`:7120`), and `frame_decals` is called twice
(`:3103`, `:3475`).
- [ ] **R14 No per-frame allocations in the "unchanged" check.**
`lod_signature` allocates 2-3 times a frame (`:7488-7535`), and so do the
wave and lean-field compares (`:3150-3213`).
- [ ] **N6 `render_screens` returns at once with zero screens.**
`scene_renderer.rs:2713-2740` clones and drops `PreparedGpuPhysics` and
submits an empty future every frame: 2.1 ms a frame at 100K, 3.8% of Eco
CPU, and one extra `vkQueueSubmit` a frame. Return when
`render_world.screens` and `screen_feeds` are both empty. R12 still
matters when screens exist. Verify: the submit count equals the frame
count; screen-feed tests.
- [ ] **X3 Shader pipelines keyed by hash, not source text.**
`surface_pipelines` (`:10915-10960`, called every frame at `:4835`),
`particle_pipelines` (`:10969`) and `condition_pipelines` (`:11555`) run
`retain` with a full string compare, then clone and SipHash the whole GLSL
source per material per frame. `render_world.rs:698-711` also clones
`GpuConditionShaders` every frame. Key by the material's u64 shader hash
(compare the source on a hit), and run `retain` only on a material
revision change. Pairs with R4.
- [ ] **X10 `lod_groups()` once per frame.** It builds a HashMap twice
(`:8782`, called at `:7033` and `:7582`).
- [ ] **X15 Cache the upscale framebuffer.** `:6074` calls
`Framebuffer::new` and `:6083` builds a descriptor set every frame. Cache
both per swapchain image and rebuild on resize.
### CPU simulation (lane: physics)
- [ ] **S1 State hash off where nobody reads it.**
- The problem: `state_hashing: true` is the default (`runtime/mod.rs:289`),
and editor play leaves it on (`bin/editor.rs:354`).
`state_hash::entity_places` runs 3 times a tick (`state_hash.rs:115`,
`:128`, `:194`). The code's own doc says this costs hundreds of ms a tick
at 1M bodies.
- Fix part 1: turn hashing off in editor play and in non-replay scenarios.
- Fix part 2: compute places once. Replace the `BTreeMap` with a sorted
`Vec`.
- Hashes must stay bit-identical.
- [ ] **S2 `refreeze_debris` stops cloning the heightfield.**
`fracture.rs:1441` clones it every tick before checking for debris. Return
early when there is none, borrow the field, and test bounds before
`world_points`.
- [ ] **S3 Particle `ground_height` columns computed once**, in
`Blockers::new` (`particles.rs:774`). Today `particles.rs:809` scans every
row for every particle.
- [ ] **S8 Sleepers cost nothing per substep.**
- Skip sleeping bodies in `write_back`.
- Read `Sleeping` and `gravity_scale` inside the gather query
(`cpu_physics.rs:1868`, `:1886`).
- Index wake-ups through broadphase pairs (`:2126`, today O(changed ×
sleepers)).
- [ ] **S12 GJK and EPA without heap.** Use `[Vector3; 4]` plus a length for
the simplex (`cpu_physics.rs:3855`, `:3927`), and reuse the EPA polytope.
- [ ] **S14 Emitters in parallel.** Each emitter owns its seeded stream, so
this stays deterministic (`particles.rs:883`). Take `command` only when it
is non-empty, so `ParticleEmitter` is not marked changed every tick.
- [ ] **N3 `stream_gpu_bodies` with no budget does nothing.**
- The problem: `hybrid_physics.rs:1097-1168` ranks, sorts and `get_mut`s
every GPU body every tick, also when `keep == usize::MAX`. That is
10.8 ms a frame at 100K.
- Fix: with no budget, remove `StreamedOut` through a
`With<StreamedOut>` query and return. With a budget, use
`select_nth_unstable_by` with the same comparator (slot as tie-break),
and write only bodies whose resident flag changes (U2).
- Verify: the streaming tests in `runtime/tests.rs`; a test that the
resident set equals the old sort on random inputs with ties.
- [ ] **N2 `step_soft_bodies` returns early with nothing to step.**
`soft_body.rs:1220-1272` builds and sorts a collider for every unparented
entity before it checks for volumes, cloths or fluids: 9.0 ms a frame at
100K, 1.6% of Eco. Return when all three are empty (U1). Verify: the
soft-body tests; `CPU fixed` on the perf line.
- [ ] **N7 `update_skins` visits only skinned or morphed meshes.**
`skinning.rs:296-305` costs 1.25 ms a frame at 100K with no skins. Query
`Or<(With<Skin>, With<Morph>)>`, and return when neither exists.
- [ ] **U4 No reverb probe without reverb zones, and a cheap
`overlap_sphere`.** `audio.rs:1013` calls
`physics.overlap_sphere(at, 0.01, u32::MAX)` every frame while a listener
exists. `overlap_sphere` (`cpu_physics.rs:1463`) runs GJK against every
body with no bounds test. Skip the probe when no `ReverbZone` exists, and
filter by bounding sphere or AABB before `collide()`, in place so the
result order is kept. Simulation code also calls `overlap_sphere`.
- [ ] **U5 State-machine `touching` from a per-tick contact index.**
`game_feel.rs:851-864` scans every contact for each `touching` edge.
Build an entity-to-partners index once per tick, only when some edge uses
`touching`. Edge order stays the same.
- [ ] **X13 Run conditions on feature systems.** No system uses `run_if`.
Ropes, fractures, soft bodies, fluid, tile maps, platformer, environment
maps and hybrid proxies run every tick in scenes without them. Add
`.run_if(any_with_component::<T>)`, but not to spawn systems that must
see the first component. FixedUpdate order stays the same. Includes the
early-outs in `collect_pickups` (`game_feel.rs:2309`) and
`allocate_auto_simulation` (`hybrid_physics.rs:1384`).
- [ ] **X11 `bind_input_actions` only on change.** `actions.rs:85-123` is
exclusive and clones and sorts every binding string every frame. Gate it
on a bindings revision.
- [ ] **X6 Single-threaded executor for PostUpdate and RenderExtract.**
Both hold only exclusive systems (`runtime/mod.rs:272-284`), so the
multithreaded executor adds wake-ups for no parallelism. The Bevy
executor costs about 17% of Eco main-thread samples.
### Load and editor (lane: tools for load and save, editor for UI)
- [x] **L1 O(N) folder-scene save.** `scene_file.rs:2430` scans every id
and allocates a `PathBuf` per compare, which is O(N²). Use a
`HashSet<Uuid>`.
- Done: `write_scene_folder` keeps the written file names
(`<id>.json`) in a `HashSet` and checks each file in `entities/`
against it, so the stale-file sweep is O(N). The set holds file names
rather than `Uuid`s, so a file the old compare deleted (an uppercase
or braced UUID name) is still deleted. Bench (temporary ignored release
test, removed): an unchanged save of a folder scene, median of 3 per
run, 2000 entities 388.6 / 391.9 / 388.9 ms before and 5.5 / 6.1 /
6.1 ms after; 10000 entities 6950 / 7092 / 7014 ms before and 31.7 /
32.5 / 31.6 ms after (median 7.0 s to 32 ms). Test:
`a_patch_on_a_folder_form_scene_rewrites_only_the_changed_entity` now
also checks that a kept entity keeps its file.
Full check passed (2731 tests).
- [x] **L2 Material dedup by hash map.** `scene_file.rs:3351` scans the
material pool for every renderer, on every load and every Undo.
Done: `prepare_assets` indexes the material pool once per call by name
and base color (with -0.0 folded into 0.0), and each inline material
compares only the candidates in its bucket, in pool order, so the first
equal material is still the one reused. Bench (release, `prepare_assets`
on 10K renderers, median of 3): 10K distinct materials 202.4 ms to
2.0 ms; 100 distinct materials 2.9 ms to 1.5 ms. Test
`equal_inline_materials_share_one_handle`. Full check passed (2733 tests).
- [ ] **L3 Big lists collapsed in the inspector.**
`editor/inspector/reflected.rs:139-160` and `:631` draw every element,
expanded. `Heightfield.heights` at 257² is about 66K widgets a frame. For
more than 32 items: collapse by default, show "N items", and use
`show_rows`.
- [ ] **L5a Single-entity capture is O(1).** `scene_entity_lenient` scans
the world (`scene_file.rs:2075`) once per scenario check per tick
(`scenario.rs:3236`). Use `query.get(world, only)` instead.
- [ ] **L6 Prefab sources read once per load.** `scene_instance.rs:496`
re-reads and re-parses the source file for every instance. Memoize per
call.
- [ ] **L8 Particle-preview check by query.** `editor/mod.rs:87-97`
iterates every entity on every winit event batch.
- [ ] **L9 One scene-object list per frame.** `reflected.rs:130` builds and
sorts every object per Entity field per frame. Build it once, or only
when the combo is open.
- [ ] **L10 No hot-reload polling in shipped games.** `assets/mod.rs:3416`
stats every asset every 500 ms on the main thread. Turn it on only in
editor and dev runs.
- [x] **N1 Spawning by name is O(1).** 98% of the 13-17 s first frame at
100K is `find_named_entity` (`project_runner.rs:4127-4153`) from
`spawn_renderable` (`:1060`). A new name misses `SceneNameIndex` and falls
back to scanning every `Name`. Keep the index exact with `Name` component
hooks (`on_insert`, `on_replace`, `on_remove`) and drop the scan. Verify:
a test that a duplicate name after a `world()` spawn or a rename is still
caught; first frame at 100K under 0.5 s.
- N1 Done: `ensure_scene_name_index` now also adds two observers once per world (`observe_scene_names`, their entities kept in `SceneNameObservers` so a scene reload adds no second pair): `On<Insert, Name>` writes the entry and `On<Discard, Name>` drops it when it points at that entity, so spawns, renames and despawns through `world()` keep `SceneNameIndex` exact. `find_named_entity` drops the scan of every `Name`; a hit is still checked against the live name, since a rename through `Mut<Name>` fires no observer (such a rename is found again only when reinserted). The manual index inserts in `spawn_renderable` and `copy_tree` are gone. `RUSTING_HEADLESS_TICKS=1 physics_bench debris 100000`, first playable frame, 3 runs (`target/perf-loop/n1-{before,after}-*`): before 10513, 10743, 10860 ms, median 10.74 s; after 355, 258, 260 ms, median 0.26 s (41x, under the 0.5 s goal); stdout and stderr otherwise the same. Tests: `the_name_index_follows_spawns_renames_and_despawns_through_world`, `a_name_spawned_through_world_after_the_index_blocks_a_spawn`. Full check passed (2705 tests).
- [x] **T7 `rusting validate` as fast as `lint`.** It is 4x slower (2.15 s
against 0.50 s at 50K). `dropped_field_diagnostics` (`cli.rs:1533`) parses
the file again, and `scene_patch.rs` `dropped_fields` clones the whole
document, then clones twice and runs `format!` for every entity key. Parse
once, remove `entities` before cloning, compare by reference, and format
only reported paths.
- T7 Done: `dropped_fields` no longer converts or clones the whole document. The settings are compared through a `SceneDocument` with no entities, and each entity is converted on its own with `to_value(entity)` and compared on the rayon pool; `par_extend` keeps the reports in entity order. `collect_dropped` skips `entities` or `components` by key instead of cloning to remove them, and grows one path buffer in place, so a path is copied only when a field is reported. The second parse of the file stays (0.25 s), since `read_scene` decodes straight into `SceneDocument`. Probe on 50K: `dropped_field_diagnostics` 1.33 s to about 0.35 s. `rusting validate` on a 50K-cube project (`t7gen.py`, 93 MB), the two binaries run in turn, 3 runs each (`target/perf-loop/t7-ab-*`, load average 2.2): before 1503, 1468, 1502 ms, median 1.50 s; after 520, 501, 522 ms, median 0.52 s (2.9x, under the 0.7 s budget; `lint` is 0.36 s); the JSON output is byte-identical. Test: `dropped_fields_keep_entity_order_and_escape_keys`. Full check passed (2707 tests; one earlier run hit "Text file busy" in `a_scenario_folder_runs_every_scenario_in_the_one_build`, which passes alone and on the rerun).
- [x] **X4 Asset loads on their own pool.** `assets/mod.rs:603` runs
blocking file reads and decodes through `rayon::spawn` on the global pool,
which the physics narrowphase and islands also use
(`cpu_physics.rs:2789`, `:3124`). Use a 2-4 thread
`rayon::ThreadPoolBuilder` pool. Verify: p99 fixed tick while streaming
50 textures.
Done: `spawn_load` (`assets/mod.rs`) runs on `asset_pool()`, a
`rusting-assets-*` pool of half the cores clamped to 2-4 threads, with
`rayon::spawn` as the fallback if the pool cannot start. Bench (temporary
release test, removed): 1200 boxes settling on CPU physics while 50
2048x2048 noise PNGs load through `load_async`. The tick p99 during the
loads went from 121.2, 108.5 and 107.9 ms to 2.5, 4.5 and 3.0 ms (median
108.5 to 3.0 ms). Before, one tick waited for the whole burst. The idle
p99 stayed about 1.4 ms. The 50 loads take 118 to 156 ms (median) on the
smaller pool. Test: `async_loads_run_off_the_global_rayon_pool`.
- [x] **U10 Clip decode off the main thread, and a lock-free meter.** (split; see U10a-U10c)
`audio_output.rs:512-545` decodes the whole clip on first play, which
hitches 10-100 ms for long clips. Streaming plays also stat the file on
every play. The `Meter` effect (`:116-135`) locks a Mutex in the audio
callback that the main thread also locks. Decode at scene load or on a
worker, cache the stream-or-static choice per path, and store peak and RMS
as `AtomicU32` bits.
- Split: U10a moves the first-play decode off the main thread; U10b
caches the stream-or-static choice; U10c makes the meter lock-free.
- [x] **U10a First-play decode off the main thread.** Done: on a window
mixer (`stream_over` set), `play` (`audio_output.rs`) streams a clip
that is not decoded yet and decodes it on the asset pool
(`assets::spawn_on_asset_pool`, the X4 pool) for the next play. A clip
that fails to decode keeps streaming. The offline mixer and reversed
sounds still decode on the spot, so scenario mixes and golden audio are
unchanged. A first play now starts as a streamed sound, which begins
once kira's decoder thread fills a buffer. Bench (temporary release
test, removed): first `run` of a `Play` for a 60 s clip, 3 runs, 2
mixers per clip each: 961 KB MP3 31.1-35.1 ms (median 33.9) to
0.07-8.67 ms (median 0.25; the 8.67 ms was the first symphonia use in
the process); 572 KB Vorbis 30.3-33.7 ms (median 33.2) to 1.14-1.66 ms
(median 1.2). Test:
`a_first_play_streams_while_the_clip_decodes_for_the_next`. Full check
passed (2729 tests).
- [-] **U10b Cache the stream-or-static choice.** Blocked: gain under the
noise. A repeat play, `file_len` stat included, took 0.01 ms in the
U10a bench.
- [-] **U10c Lock-free meter.** Blocked: gain under the noise. Only
`buses()` reads the meter, and only scenarios call it, on the offline
mixer's own thread. A running game never locks the meter outside the
audio callback, so the lock has no contention to remove.
- [x] **U6 Follow targets resolved once.** HUD follow (`game_feel.rs`,
about `:2740`) and `follow_cameras_2d` (`two_d.rs:963-990`) find the
target by a linear name scan every frame. Cache the Entity, and re-resolve
only when it is gone or renamed. Keep the first-match rule for duplicate
names.
- Done: `runtime::find_named` keeps each name's match in a system
`Local` map and scans again only when that object is gone or renamed,
taking the first match as before. `follow_cameras_2d` and the HUD
follow both use it. Bench (temporary ignored release test, removed):
`follow_cameras_2d` with 50K named objects and the target last in
query order, 300 runs, p50 0.0243 / 0.0243 / 0.0240 ms before and
0.0002 / 0.0002 / 0.0002 ms after (median 0.024 to 0.0002 ms). The HUD
paid the same scan once per following element, so 20 name tags at
50K objects drop from about 0.5 ms a frame to nothing. Test:
`cameras_2d_follow_a_renamed_or_despawned_target_to_its_successor`.
Full check passed (2731 tests).
- [x] **U9 Scenario invariants resolve entities once.** `scenario.rs:2333-2366`
runs `find_entity` (an O(N) scan) and `reflected()` twice per invariant
per tick, and `has_gpu_bodies` (`:1962`) scans every body every tick.
Resolve once, reflect once per entity per tick, and keep an index of the
next step. Extends L5a.
- Done: the invariant loop keeps one `find_entity` result and one scene
form per entity per tick (checks change nothing), and `reflected_as`
and `check_state` take them instead of looking the entity up again.
Each invariant used to cost five world scans (two `find_entity` inside
`reflected`, called twice, plus the first lookup), not counting
`scene_entity_lenient`; now an entity costs one, plus L5a's. Bench
(temporary ignored release test, removed): 20K named entities, 60
ticks, 4 invariants on 2 entities, invariant cost per tick (run minus
the same run without invariants) 0.936 / 0.921 / 0.807 ms before and
0.178 / 0.128 / 0.166 ms after (median 0.92 to 0.17 ms). Left out:
`has_gpu_bodies` runs only for captured headless runs and is one pass
over a single component, and the steps loop is a range test per step,
so neither needs an index. Test: two new invariants in
`invariants_are_checked_every_tick_and_report_the_first_bad_tick` (a
second check on a shared entity, and an absent counter).
Full check passed (2731 tests).
### Build and dev loop (lane: tools)
- [ ] **X1 Optimized dependencies in dev and test builds.** Cargo.toml has
no `[profile.dev.package."*"]`, so vulkano, bevy_ecs, nalgebra, egui and
image run at opt-level 0 in debug runs and tests. The engine's own warning
already tells games to set it. Add `[profile.dev.package."*"] opt-level = 3`
and consider `[profile.dev] opt-level = 1`. This is a build-setting
change, so the owner decides. Verify: `cargo test --workspace` wall time
and a debug editor frame.
- [x] **T13 Split the 191 s test.** `regression_scenes_repeat_their_recorded_results`
(`runtime/physics_suite.rs:977`) runs 8 scenes twice each, one after
another, and takes 191 s of the 209 s library suite. Make one test per
scene so libtest runs them in parallel. The recorded values stay the same.
- T13 Done: the loop is now `repeats_its_recorded_result` plus one test per scene (`regression_joint_chain`, `regression_articulated_chain`, `regression_character_stairs`, `regression_vehicle_stress`, `regression_ragdoll_pile`, `regression_buoyancy_sort`, `regression_sheet_catch`, `regression_jelly_raft`), with the same sizes, steps, recorded values, bit-identity check and 1e-4 tolerance. Each test runs its two runs on two threads with `std::thread::scope`. `cargo test --lib` of the test(s), 3 runs: before 177.6, 186.7, 191.9 s, median 186.7 s; after 58.8, 56.8, 57.3 s, median 57.3 s (3.3x). The whole library suite in the full check: 211.0 s (T7 run) to 89.0 s. The guard list in the ground rules now names these tests. Full check passed (2721 tests).
- [x] **T9 Shared engine build across projects.** A new project's first run
compiles the engine in 131 s (902 s CPU), and each project's target is
2-3 GB. Use a shared target dir per engine version (for example
`~/.cache/rusting/target-<version>`, overridable with
`CARGO_TARGET_DIR`). Also find why a scene-only edit rebuilds the game in
7.1 s; it may be `include_bytes!`. Verify: a second new project's first
run under 15 s; a scene edit leaves every crate Fresh.
- T9 Done: new `project::cargo_command()` starts every game cargo call (`cli.rs` `cargo`, the editor's check/build and run) with `CARGO_BUILD_BUILD_DIR` set to `~/.cache/rusting/build-<version>` (`collab::cache_root()`, now shared with the collab mirror; with no user cache directory it sets nothing rather than fall back to a shared temporary folder, where another user could plant build artifacts, as a commit security review flagged), unless `CARGO_TARGET_DIR`, `CARGO_BUILD_TARGET_DIR` or `CARGO_BUILD_BUILD_DIR` is set. It shares Cargo's build dir (stable since Cargo 1.91; 1.95 here), not the target dir, so each game's binary stays in its own `target/` and same-named games cannot overwrite each other; `built_executable` still finds it through `cargo metadata`. Measured with `rusting run <new project> --ticks 1 --json`, projects pinned to a snapshot of the engine at 04ecffd, `XDG_CACHE_HOME` in `target/perf-loop/t9`: before, two new projects' first runs took 132.3 and 118.5 s with 1.9 GB `target/` each; after, the first project took 123.7 s and the next three 6.4, 1.8 and 1.9 s (median 1.9 s) with 46 MB `target/` each (shared folder 2.0 GB). The 7.1 s scene-edit rebuild did not reproduce: the template has no `include_bytes!` (`rusting_game!` reads `build/main.rscene.bin` at run time), and a scene edit run took 172 ms before and 184 ms after with the game binary's mtime unchanged, so nothing was rebuilt. `docs/concepts.md` "Build times" describes the shared folder and the overrides. Unit test `game_builds_share_one_build_dir_unless_the_user_chose_one`. Full check passed (2723 tests).
- [x] **T8 Skip the cook when nothing changed.** (split; see T8a-T8c) A warm `run --ticks 1` at
10K entities takes 852 ms, while the game reaches its first frame in
about 110 ms. The rest is the cook (461 ms), the cargo no-op (120 ms) and
writing the 20 MB `build/final.rscene`. Skip the cook on unchanged scene
and asset hashes, write `final.rscene` only on request, and skip cargo
when the binary is newer than the sources.
- Split: T8a skips the cook; T8b writes `final.rscene` only on request; T8c
skips the cargo no-op.
- [x] **T8a Skip the cook.** Done: `cook_project` (`cli.rs`) writes
`build/cook-inputs.txt`, the path, size and modified time of every project
file outside `build/`, `target/`, `.git/` and `.rusting/` plus the running
`rusting` executable and the cooked scene. When the stamp still matches, it
returns the same success without validating or cooking. A failed
validation or cook writes no stamp. `.rusting/` is left out because every
CLI command appends to its journal; with it in, the stamp never matched.
Limit: asset or instance files outside the project root are not stamped
(`ponytail:` note). Measured on a new project with a 10K-entity main scene
(`target/perf-loop/t8`, engine pinned to a snapshot), new and old binaries
alternating, 3 runs each, twice: `rusting cook` 134-150 ms to 1-2 ms;
warm unchanged `rusting run --ticks 1` 438-487 ms (median 462) to 290-354
ms (median 312) after the first run, which still cooks. Unit test
`an_unchanged_project_skips_the_cook` (unchanged, scene edit, deleted
cooked scene, and a symbolic link loop in `assets/`). The walk reads each real folder once, so a link loop cannot recurse until the stack overflows, which a commit security review flagged in the first version. Full check passed (2725 tests).
- [-] **T8b Write `final.rscene` only on request.** Blocked: owner decision. `run --ticks`
reports `final_scene` and docs point `rusting diff` at it, so making it
opt-in changes the CLI output (owner decision). At 10K entities it is 19 MB.
- [-] **T8c Skip the cargo no-op.** Blocked: not worth the risk. The no-op `cargo build` is 120 ms of the
312 ms warm run. Skipping it means redoing Cargo's freshness check (path
dependencies, `Cargo.lock`, features, profile), so it is left open.
## Phase 2: structural (medium effort)
### Render
- [ ] **R3 Upload only moved instances.** Correction: the 43 MB in the Eco
bench is not instance data (see T1). R3 is the cost measured in T2: with
50K bears, 1, 50 or 5,000 moved instances all upload 12.3 MB a frame
(240 B per instance, `RenderInstanceUpload` at `:489`), and 100K bears
upload 24.3 MB a frame.
- The problem: one moved object re-uploads all of `packed` (`:7537-7550`).
`move_instances` then sets `visibility = None`, which forces a full
recull (`:1201-1232`), and rebuilds the descriptor set (`:3215-3283`).
- Fix part 1: pass the moved render indices from `renderables_changed`.
- Fix part 2: keep one persistent instance buffer per frame context, and
patch dirty ranges across `FRAMES_IN_FLIGHT`.
- Fix part 3: recull only the moved bounds.
- Gate: upload bytes proportional to moved count. `--bears 50000
--bear-updates 50` uploads about 72 KB + 50 × 240 B a frame.
- [ ] **T1 Light clusters without a full upload every frame.**
- The problem: since `5790146`, `build_light_clusters` (`:1489`) rebuilds
the 16×9×24 cluster list on the CPU every frame and uploads it, about
67-71 KB a frame (`CLUSTER_GRID` `:1475`, counter `:3798`). The bench
camera moves every frame, so R10's cache does not help it.
- Fix, cheap part (B6): pack offset and count into one u32 and store light
indices as u16 (the cap is 1,024 lights).
- Fix, full part: build the clusters in a compute pass, or upload only
the rows that changed.
- Verify: the cluster unit test (about `:15025`) and golden images
unchanged; Eco `--no-bodies` upload below 10 KB a frame.
- [ ] **T4 High shadow pass at 50K instances.** Four runs took 56.2, 60.4,
5.5 and 3.2 ms with identical counters (376 draws, 25.8M triangles); Eco
takes about 1 ms. First repeat on an idle GPU. Then cull casters per
cascade, cap small casters, and add a timestamp per cascade. Relates to
R2 and R15. Gate: High `--bears 50000` shadow p95 below 4 ms in three
idle runs.
- [ ] **T3 One device allocator with smaller blocks.** About 9
`StandardMemoryAllocator::new_default` instances map 256 MiB blocks each,
so every windowed run uses about 1.2 GB RSS (524 MiB of
`/dev/nvidiactl` mappings, a 20 MiB heap). The same game headless uses
47 MiB. Sites: `gpu_cloth.rs:96`, `gpu_broad_phase.rs:76`,
`capture.rs:148`, `benchmark.rs:276`, `egui_painter.rs:654`,
`swapchain.rs:274`, `rendering/mod.rs:182` and `:542`,
`scene_renderer.rs:13974`. Share one `Arc` allocator per device with
16-32 MiB blocks (`GenericMemoryAllocatorCreateInfo`), and create the
subsystem allocators lazily. This matters on 2-4 GB iGPUs. Also fixes the
`gpu_memory_bytes` reading in M2.
- [ ] **N11 Find the 251 ms startup allocation.** One `vkAllocateMemory`
call takes 251 ms in Eco (10 calls, 395 ms in all). Log the sizes once,
then split or defer the big block. Check whether a capacity-sized buffer
is cleared when it is allocated. Likely tied to T3.
- [ ] **X8 Pack render instances in parallel.** In
`prepare_render_instances` (`:7655-7800`) each output slot is independent:
a normal-matrix inverse, 3 binary searches and HashMap lookups per
instance. Use `par_iter().map(pack).collect_into_vec`, which keeps index
order, above about 2K instances. Expected 4-8x on that loop at 100K.
- [ ] **X7 Fewer descriptor sets per frame.** A set is built per sprite
batch (`:6283`), per particle emitter (`:6623`) and per skin job
(`:6756`), plus about 9 fixed sets every frame (`:3230`, `:3337`, `:3803`,
`:4153`, `:4224`, `:4281`, `:4321`, `:5722`, `:6083`; `post_effects.rs`,
`sky.rs`, `gpu_broad_phase.rs`, `gpu_cloth.rs`). Put each kind's jobs in
one buffer with dynamic offsets, and cache the fixed sets per frame
context until a bound resource changes.
- [ ] **T12 Static entities cost nothing per frame.** A windowed 50K
static scene with nothing on screen spends 5.47 ms of CPU a frame
(update 4.6, extract 2.2), and the headless tick goes from 2.0 ms at 10K
to 12.0 ms at 50K. Skip unchanged entities in update and extraction, and
cache static extracted instances. Builds on R11, R13, S4 and N5. Gate:
extract under 0.5 ms and update under 1 ms at 50K.
- [ ] **R1b Probe relight batched.** Record all faces into one command
buffer, and fence-poll the readback like the physics readback (`:8299`).
- [ ] **B2 Skip contact rounds that push nothing.** Each round costs about
0.37 ms in Eco, and once a pile sleeps, rounds 2-4 move nothing. Each
round sets one "pushed" word with `atomicOr` when any correction is above
a small epsilon. The next round dispatches through `dispatch_indirect`
with 0 groups when the word is clear, like the oversized pass. Keep 4
rounds as the limit. The flag does not depend on order. Expected: up to
1.1 ms per tick at rest. Verify: the ball pit (3,045 at rest, 0 escaped),
`a_100k_pile_lands_on_the_gpu_without_sinking`, the repeat-exactly tests.
- [ ] **B3 Native math outside CrossPlatform.** Add a `SIM_NATIVE`
specialization constant that switches `sim_recip`, `sim_rsqrt` and
`sim_sqrt` to `1.0/x`, `inversesqrt` and `sqrt`, as the `ponytail:` note
in `sim_math.glsl` proposes. Set it unless the project is CrossPlatform;
one device and driver still repeat bit for bit. The pipeline cache key
must include the mode. Measured: 1.56-1.83 to 1.17-1.19 ms. Verify: the
bit-for-bit reference test only in CrossPlatform, plus a gpu-test that
runs Local twice and gets the same bits.
- [ ] **G4b Shorter physics chain.** Clear the next round's counters at the
end of the contact pass instead of 12 `fill_buffer` calls a tick. Run
`physics_hash` only when hashes are read, or fold it into `physics.comp`.
Keep `CONTACT_ROUNDS` unless M4 says otherwise.
- [ ] **G6 High-quality GPU cost.**
- The cost: High uses 4x MSAA on RGBA16F with D32 (`:12229-12253`,
`:9773`) and a 4096×4 cascade shadow map.
- Options:
- Default High to 2x MSAA.
- Use a B10G11R11 HDR target, after checking that `tonemap.frag:279`
does not need alpha.
- Use 2048 cascades.
- This is a visual trade, so update the golden images deliberately.
- [ ] **G7 Less overdraw.** Sort front to back inside opaque batches, or
draw large objects last. Or turn on the existing depth prepass above an
instance threshold and shade with depth test EQUAL.
- [ ] **G9 Sway loop only for swaying materials.** The vertex shader loops
over every force field per vertex (`:12462-12640`).
- [ ] **G12 Cheaper timestamps, and skip the shadow pass with no casters.**
`PassRecorder` uses `AllCommands`, which drains the pipe at every mark.
- [ ] **R10 Cache light clusters** on (clip bits, lights_revision, depth
range), and reuse the Vecs (`build_light_clusters`, `:1489`).
- [ ] **R11 Change-filtered `renderables_changed`.**
- The problem: it scans every renderable's ticks every frame
(`render_world.rs:1076`).
- Fix: use cached `QueryState`s with `Or<(Changed<..>)>`, and
`RemovedComponents` for despawns.
- [ ] **R12 `PreparedGpuPhysics` shared by `Arc`** across screens
(`:2817`).
- [ ] **R13 Cached extraction QueryStates.** `extract_render_world` creates
about 35 per frame (`render_world.rs:384-977`). Clone strings and decals
only on change.
### CPU simulation
- [ ] **S4 Transform propagation from the moved set.**
- The problem: `hierarchy.rs:247-260` walks every entity's parent chain
when anything moves.
- Fix: start from the moved entities and BFS down `Children`. Keep the
(depth, entity) order.
- This is in rusting-core, so it needs a core release.
- [ ] **S5a Gather bodies once per fixed step.**
- The problem: every substep runs `gather_bodies` (14-component query,
sort, `merge_compounds`, HashSets) and then a full `write_back`
(`cpu_physics.rs:1783`, `:3549`).
- Fix: keep `Vec<Body>` across substeps and write back once.
- The Euler round-trip between substeps goes away, which changes bits
once. Re-record the regression hashes deliberately, in their own commit.
- [ ] **S6 CCD candidates from the broadphase.** The sweep tests every fast
body against every body (`:3410-3426`), and `grown()` clones a Body for
each candidate.
- [ ] **S7 Incremental broadphase.**
- Keep `order` across steps and re-sort it with an insertion sort.
- Skip inert-inert pairs inside the sweep.
- Reuse the buffers (`:2671-2715`).
- [ ] **S9 Animation targets cached.**
- The problem: `find_target` splits a string path twice per track per tick
(`animation.rs:1091`, `:1122`, `:1154`), and three places do O(tracks²)
linear finds.
- Fix part 1: cache the target Entity per track, and index the pose by
track.
- Fix part 2: check `is_sorted` at load. Cache the additive base pose.
- [ ] **S10 Fluid on a flat grid and in parallel.** Use a counting-sort grid
with CSR neighbor lists in reused buffers (`fluid.rs:140`, `:318`, `:734`).
Run the Jacobi passes with `par_iter_mut`, keeping neighbor lists sorted.
- [ ] **S11 Warm start and manifolds without per-pair Vecs**
(`cpu_physics.rs:1243`, `:3005`, `:3209`, `:2843`).
- [ ] **S13 Compound parts by reference, one pass for contact and manifold**
(`cpu_physics.rs:1114`, `:3585`, `:3211`).
- [ ] **S15 Cached QueryStates in the exclusive FixedUpdate systems.**
`manage_debris` should query `&Debris`, not `&mut` (`fracture.rs:538`).
- [ ] **N4 No CPU mirror of GPU bodies that nothing on the CPU touches.**
`gather_bodies` (`cpu_physics.rs:2221-2265`, from `step_cpu_physics`
`:1752`) copies every GPU body into the CPU world: 4.2 ms a frame at 100K
with zero CPU bodies. Skip the CPU step when there is no CPU dynamic or
kinematic body and no controller. Later, mirror only bodies inside the
CPU bodies' bounds. Check the raycast and query users of `PhysicsWorld`
first. Determinism hashes stay the same.
- [ ] **N5 GPU body signature from change ticks.**
`simple_gpu_physics_signature` (`hybrid_physics.rs:1641-1720`, from
`render_world.rs:715`) hashes about 25 fields of every GPU body every
frame: 3-4.7 ms at 100K, 3.3% of Eco. Bump a revision from `Changed` and
`Added` filters on the same components, removals and the body count
(U3). Verify: the signature tests at `runtime/tests.rs:503-507`, plus a
test that editing each formerly hashed field bumps the revision.
- [ ] **X2 Fast hashers on hot maps, with no new dependency.** All 242
HashMap and HashSet uses take SipHash. About 30 are on per-frame or
per-substep paths:
- `cpu_physics.rs:1761`, `:1785`, `:1814`, `:1991`, `:2134`, `:2590`,
`:2600`, `:3105`, `:3126`, and the persistent maps at `:1230`, `:1243`,
`:1245`
- rusting-core `hierarchy.rs:189-205`, `:244`
- `render_world.rs:289`
- `scene_renderer.rs:7493`, `:7643`, `:7660`, `:7755`, `:7816`, `:7823`,
`:7857`, `:1951-1958`, `:1251`, `:8385-8387`, `:3965`
- `hybrid_physics.rs:73`, `fluid_surface.rs:180`
Use `bevy_ecs::entity::{EntityHashMap, EntityHashSet}` for Entity keys.
Pack entity pairs into a u64. Replace dense integer keys (`island_of`,
`slots`, `sways`, `parts`) with `Vec`s. A fixed hasher also removes
RandomState iteration order. Check that nothing relies on iteration order.
- [ ] **X9 Cached QueryStates outside extraction and FixedUpdate.**
224 `world.query` sites rebuild their state on each call; R13 and S15
cover only some. Per-frame sites include rusting-core `hierarchy.rs` (3),
`skinning.rs:100`, `:147`, `:301`, `audio.rs:546`, `:949`,
`soft_body.rs:1927`, `:2139`, `:2250-2256`, `hybrid_physics.rs:842`,
`:1396`, `:1476`, `:1649`, `:1733`, `terrain.rs:278-291`, `actions.rs:95`,
and some of 22 sites in `project_runner.rs`. Copy the cached pattern of
`scene_tree.rs:149` and `particles.rs:973`.
- [ ] **X12 Animation parameters by index.** `animation.rs:423` stores
`BTreeMap<String, f32>`, read per transition per tick (`:347`). Resolve
names to indices into a `Vec<f32>` at load. Serde stays name-based.
- [ ] **X14 Flat heightfield storage.** `Vec<Vec<f32>>` heights and
`Vec<Vec<u8>>` cells (`cpu_physics.rs:107`, `:113`; `terrain.rs:31`,
`:33`, `:84`, `:146`; `reflect/builtin.rs:859`, `:861`) double every
sample lookup. Store one flat `Vec` with a width, and keep the scene
format through `#[serde(with)]`.
- [ ] **T10 Fluid coupling cost.** buoyancy_sort takes 46.8 ms a step and
jelly_raft 40.4 ms, 4x any other suite scene. Profile with nsys. Reuse
the spatial hash across substeps, and couple the fluid only to bodies
whose AABB overlaps the pool. Extends S10. Gate: buoyancy_sort under
10 ms with its recorded error unchanged.
- [ ] **U11 Soft-body skins once per frame.** `sync_soft_skins`
(`soft_body.rs:977-1118`) hashes every particle and re-sorts 4T faces on
every fixed tick, including catch-up ticks. Cache the surface indices per
tear revision, update only positions and normals, and run it in
PostUpdate, because skins are visual.
- [ ] **U15 Soft-body obstacles rejected by AABB.** `solve_holds`
(`soft_body.rs:621-636`) tests every particle against every obstacle,
fluid grains included, in each substep. Reject an obstacle per body with
a conservative AABB test. The solve order stays the same, and the state
hash must match over 300 ticks.
- [ ] **U12 Tile map edits touch only changed tiles.** `two_d.rs:470-570`
despawns and respawns every tile and collider on any change. `kind_at`
uses `chars().nth`, which is O(cols²) per row. The orphan scan runs every
frame, and flood fill calls `set_cell` per cell. Collect rows once, diff
the changed cells, parent tiles to the map, and fill on a char grid.
Keep row-major spawn order, because tile colliders feed physics.
### Networking, scripting and scenarios
- [ ] **U7 One snapshot encode per base sequence.** `net/replicate.rs:206-219`
diffs and encodes once per peer per tick. Memoize the bytes by the acked
base sequence. The bytes on the wire stay the same.
- [ ] **U8 Capture only replicated components.** `net/replicate.rs:281-330`
serializes all about 90 registrations to strings per entity per tick
(`scene_file.rs:1689`), then parses back the wanted ones. Call
`serde_json::to_value` on the listed components only. Verify with a golden
test.
- [ ] **U13 No blocking TCP writes on the game thread.** `net.rs:735-760`
runs `write_all` per peer while it holds the `peers` mutex, so one
stalled client stalls the host. Encode once and hand the frame to a
per-peer writer thread over a bounded channel.
- [ ] **U14 Typed script field access.** `crates/rusting-script/src/lib.rs:430-478`
converts the whole component through `serde_json::Value` on every get and
set, and `run_scripts` (`:212`) clones every script path every frame. Add a
fast path for Transform position, rotation and scale, and borrow the
paths.
### Load and editor
- [ ] **L4 Heightfield serialized by reference, and decoded images cached**
by (path, mtime).
- The problem: serde `into` clones the heights every frame while the
entity is selected (`cpu_physics.rs:102`, `terrain.rs:48`). Each edit
and each undo decodes the PNG again.
- [ ] **L5b Capture walks the archetype's components**, not all of about 90
registrations per entity (`scene_file.rs:2030-2165`). The output must stay
byte-identical.
- [ ] **L7a Typed fast path for component restore.**
- Parse with `serde_json::from_str::<T>` when the type has no references
and no migrations.
- On error, fall back to the current Value, migrate and walk path
(`scene_file.rs:1915-1972`).
- [ ] **L11 Parallel decode of scene assets.** Collect unique mesh and
texture paths and decode them with rayon. Insert them in path order, so
handle numbering stays deterministic (`scene_file.rs:3314-3365`).
- [ ] **L12 Cached hierarchy tree.**
- The problem: `collect_entities` clones, lowercases and sorts every name
every frame (`editor/view.rs:123`, `editor/hierarchy.rs:45-160`).
- Fix: rebuild only on Name or Parent change, or when the entity count
changes.
- [ ] **L13 Pipeline cache, SPIR-V cache and overlapped startup.**
- The problem:
- Every pipeline is created with `None` cache (`scene_renderer.rs:10800`,
about 35 sites).
- Surface shaders run `glslc` inside `render()` (`:4835`,
`:11613-11640`).
- Scene load finishes before Vulkan init starts (`project_runner.rs:5801`,
`bin/editor.rs:420`).
- Fix part 1: persist a `PipelineCache`, and cache `.spv` by GLSL hash.
- Fix part 2: pre-build surface pipelines while loading.
- Fix part 3: load the scene on a thread while the device is created.
- [ ] **L14a Undo snapshots share unchanged entities.**
- The problem: up to 100 full `SceneDocument` copies, plus a full compare
per edit (`editor/mod.rs:992-1010`).
- Fix: store `Vec<Arc<SceneEntity>>` and reuse equal entities. Add a byte
budget.
## Phase 3: big bets (large effort, after phases 1-2 land)
- [ ] **G8 Uber-shader feature specialization.**
- Today `scene.frag` (768 lines) branches at runtime on terrain, water,
decals, probes, lightmap, cookies and fog.
- Add specialization constants per feature, and a pipeline cache keyed on
the flag bits.
- Expected: 10-25% of the Scene pass.
- [ ] **R15 GPU-driven shadow culling.** Run the existing cull compute
against each cascade and draw with `draw_indexed_indirect`.
- [ ] **S5b Persistent CPU bodies.** Keep bodies in `PhysicsWorld` across
steps, and refresh them from `Changed`, `Added` and removed components.
- [ ] **L14b Undo by diff.** Respawn only the changed ids instead of
reloading the whole world.
- [ ] **L15 Asset lifetime and memory.**
- Mark and sweep unreferenced assets after a scene change.
- Drop `TextureAsset.rgba8` after upload.
- Store cooked pixel and vertex data as raw bytes (format bump).
- [ ] **L7b Components as `RawValue` in `SceneEntity`.** This is a format
change.
- [ ] **N8 Less vulkano auto-sync work.** `AutoCommandBufferBuilder::build`
(`scene_renderer.rs:7009`) is 11.4% of Eco main-thread CPU, about
0.12 ms a frame. First cut commands and per-frame descriptor sets (X7).
Then consider prebuilt secondary buffers for the physics chain, or
`_unchecked` calls in the hot draw loop if recording passes 1 ms in the
M3 city scene (X16). Keep the checked path under `debug_assertions`.
- [ ] **X18 Pipeline simulation and render.** Record frame N on a render
thread while frame N+1 simulates. No gain today, because the bench is
GPU-bound; it pays only at 100K bodies, where the CPU reaches 7-16 ms.
## Owner decisions
These need the owner's approval under AGENTS.md (new dependencies or build
settings). Each has a no-code measurement first.
- **X5 Global allocator.** libc malloc, free and memcpy are 18% of Eco
main-thread samples. Measure first with
`LD_PRELOAD=/usr/lib/libjemalloc.so.2` and
`LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so` on physics_suite and
render_bench (both are installed). If either gains more than 5%, decide
on `mimalloc` or `tikv-jemallocator` as `#[global_allocator]` in the
binaries only.
- **X2 follow-up.** `foldhash` is already in Cargo.lock as a transitive
dependency. Making it direct is optional, because `EntityHashMap` and
`Vec`s cover most hot maps.
- **X17 `target-cpu=x86-64-v3`** for shipped builds: 0-10% on vectorizable
loops. It crashes on CPUs without AVX2, and cross-machine replay must be
re-checked.
- **X1 dev-profile optimization** (see Build and dev loop).
## Order and lanes
1. **Phase 0 first.** M2 (with N10, T5, T6, T11) makes the benchmarks
trustworthy. M4 and M5 are done.
2. **Biggest measured wins, take these first:**
- N1: 100K first frame from 13-17 s to under 0.5 s.
- N2, N3, N4, N6, N7: about 27 ms a frame at 100K, mostly early-outs.
- G1 (with N9): -31% GPU frame at 100K, measured.
- B1, B2, B3: Eco Physics pass from about 2 ms toward 0.5 ms.
3. **physics lane:** N3, N2, N7, G1, B1, G4a, B4, G3, G5, then S1, S2, S3,
S8, S12, S14, U4, U5, X13, X11, X6, then B2, B3, N4, N5, X2 and the phase
2 S, U and X items. The GPU physics items touch the physics block of
`scene_renderer.rs`, so coordinate with render before taking them.
4. **render lane:** N6, R1a, R2, R8/G2, X3, X10, X15, then R4-R7, R9, R14,
then R3, T1, T3, T4, X8, X7, T12, then the rest of phase 2.
5. **tools lane:** M1-M3 with the M2 additions, N1, T7, T13, T9, T8, X4,
U10, U6, U9, L1, L2, L5a, L6, L10, then L4, L5b, L7a, L11, L13, N11, and
the networking and scripting items (U7, U8, U13, U14).
6. **editor lane:** L3, L8, L9, then L12, L14a, U12.
7. **Owner decisions:** X1, X5, X17, foldhash.
After each phase: re-run every benchmark, update `benchmarks/README.md`,
and add a CHANGELOG line with the before and after numbers.
Already fine, so do not redo these:
- frames in flight and fence-polled readbacks
- cached pipelines, samplers and descriptor sets
- revision-gated mesh, texture and light uploads
- indirect GPU-cull draws
- the parallel narrowphase and islands
- sort-and-sweep broadphase
- sleeping and warm starting
- the editor sleeps when idle; hierarchy and asset rows use `show_rows`
- GPU mip blits
- release profile (fat LTO, one codegen unit, `panic = "abort"`); Vulkan
validation only in debug or behind its feature; no hot-path logging
- the frame loop: catch-up capped at 8 steps, `WaitUntil` pacing with no
busy spin, present-mode choice, the headless server loop
- editor picking against cached mesh AABBs; overlays; thumbnails decoded a
few per frame
- `world_text` filtered on `Changed`, HUD text through egui's galley cache
- navigation runs only on demand
- memory: RSS is flat over 90 s runs at 10K and 100K bodies (no leak)
## Found by the loop
### Sweep 1 (2026-10-16, after M2a-M2f)
Benchmarks in `target/perf-loop/sweep1/`, no other GPU job. Eco frame p95
4.52 ms, CPU p95 0.55 ms, Physics pass 1.56 ms; High frame p95 6.11 ms; Eco
50K bears frame mean 12.8 ms, CPU mean 3.6 ms; 100K falling 36.9 ms GPU
Physics and 2.25 ms CPU fixed per tick; 10K falling 2.1-2.3 ms GPU Physics
per tick. physics_suite matches the round 2 table.
Nsight on Eco 50K bears with 50 updates (frame-pointer build, render
thread, 5081 samples). The bears scene had no profile before. Every hot
spot maps to an existing item, so these are evidence, not new items:
| % | Function | Item |
|---|---|---|
| 26.5 | `extract_render_world` | R13 |
| 19.8 | `propagate_transforms`, of which `update_moved` 16.0 (its SipHash `HashSet` is 9.4) | S4 |
| 11.9 | `update_skins` (`morph_weights` and `IndexMap::get` per renderer; the bears have no skin) | N7 |
| 8.5 | `prepare_render_instances` | T2 |
| 8.2 | `shadow_runs`, of which `bounds_in_frustum` 6.6 | R2a/b, R15 |
| 4.8 | vulkano `AutoCommandBufferBuilder::build` | N8 |
The main thread spends its other samples in the NVIDIA driver's wait
(`pthread_rwlock_trywrlock`), because this run is GPU-bound.
- [x] **P1 GPU-cull counters drop to 0 when a readback is still locked.**
- The problem: `collect_cull_readbacks` (`scene_renderer.rs:2607-2620`)
skips a draw-command buffer whose `read()` fails, then stores the sum
anyway. vulkano keeps the GPU write lock until the frame's future is
cleaned up, which can be after its fence signals, so `read()` returns
`AccessConflict` and the frame counts 0.
- Evidence: `physics_bench falling 10000` perf lines alternate between
`triangles 120012` and `triangles 0` with the same 10K cubes on screen
(`target/perf-loop/sweep1/phys10k.out`). `visible_instances` has the
same path.
- Fix: leave the entry pending (or keep the old numbers) when any read
fails; take only fully read frames.
- Gate: a unit or GPU test where a locked readback keeps the previous
count; the 10K perf line never shows `triangles 0`.
- Done: `FenceSignalFuture::is_signaled` only polls the fence; the
buffers stay locked until a wait or cleanup. `collect_cull_readbacks`
now calls `wait(Some(Duration::ZERO))` on the signaled fence, which
returns at once and releases them, and if a read still fails it keeps
the last full frame instead of storing a partial sum.
`RUSTING_PERF=1 RUSTING_QUIT_AFTER_MS=20000 physics_bench falling 10000`
(`target/perf-loop/p1-{before,after}.out`, `strata` on the GPU): before,
16 of 18 perf lines show `triangles 0`; after, 20 of 20 show
`triangles 120012`. Frame rate is unchanged within the noise. Test:
`gpu_culling_counts_visible_instances_from_gpu_transforms_into_indirect_draws`
now renders a frame it does not wait on, idles the queue, and expects
(visible, culled) (3, 1); without the fix it reads (0, 4). Full check
passed (2687 tests).
### Sweep 2 (2026-10-16, after M2g-M2j)
Benchmarks in `target/perf-loop/sweep2/`, with another engine's job
(`strata`) on the GPU, so times are rough. Eco frame mean 3.89 ms, p95
4.57 ms, `wait` 3.21 ms; High frame mean 5.52 ms, p95 7.01 ms; Eco 50K
bears frame mean 12.94 ms, CPU mean 3.69 ms (`post_update` 1.11, `extract`
1.41, `prepare` 1.10, all tracked as S4, R13, T2). 100K falling: Physics
238.5 ms per frame, of which `contacts` 197.0, `oversized` 30.4, `grid`
8.2, `step` 1.2, `hash` 1.7 (M2ia parts); 29.7 ms GPU Physics per tick.
physics_suite matches sweep 1. No Nsight run: the worst case is GPU-bound
and the M2ia parts already split it per dispatch. Counters are unchanged
from sweep 1 (Eco 13 draws, 142,768 triangles; High 23 draws).
- [-] **P2 The GPU contact pass costs 6 ms per round at 100K bodies.**
- The problem: `physics_contacts.comp` (contact `main`) visits 27
neighbour hash slots per body, dedupes them through a `uint visited[27]`
array (up to 351 compares, likely in local memory), and the table has
only 2 cells per body (`PhysicsContactGrid::hash_cells`,
`scene_renderer.rs:1368`), so each body also visits bodies of other
cells that share a slot.
- Evidence: physics_bench falling 100000, `contacts` 197.0 of 238.5 ms
Physics per frame (32 rounds, so 6.2 ms per round), with 88,824,640
hash collisions per frame, 28 per body per round. A Python model of a
dense block (one body per cell) predicts 9.4 visits at 2 cells per body,
4.3 at 4, 2.1 at 8; a mixed hash does not beat the current one. Why the
bench sees 3 times the model is open.
- Fix: drop the `visited` dedupe (the contact sums are max, min and or,
so visiting a slot twice gives the same body result; only the
hash-collision counter changes), and raise the table to 8 cells per
body within `contact_hash_budget` (100K bodies: 1M cells, 68 MB).
- Gate: `contacts` at 100K falls at least 30% (M2ia part, median of 3);
the bit-identical physics guards pass unchanged.
- Done (in part): `touch_in_cell` now skips a slot entry filed from a
cell other than the scanned one before `touch` loads its state and
shape, and the `visited` dedupe is gone, so a slot two neighbour cells
share is scanned once for each. No contact is lost: a skipped body in a
neighbour cell is touched when that cell is scanned, and any other is
more than one cell away, while a non-oversized body is at most one cell
wide. Overflowed bodies still go through the fallback list.
`RUSTING_PERF=1 RUSTING_QUIT_AFTER_MS=45000 physics_bench falling 100000`,
median of 3 (`target/perf-loop/p2-*.out`): `contacts` 223.67 to
161.16 ms per frame (-28%, `strata` kernels on the GPU), and in 3
alternating pairs with the GPU 95-100% busy 246.94 to 180.87 ms (-27%).
`Physics` 262.67 to 201.36 ms. Hash collisions still count the skipped
entries (88.8M to 88.9M a frame, as each shared slot is now counted per
cell). No new test: the gain is time only, and the bit-identical
physics guards and contact tests pass unchanged. Full check passed
(2687 tests).
- Tried and dropped: the dedupe removal alone (230 ms, no gain); 8 cells
per body (`contacts` 1014 ms and `grid` 46 ms, while hash collisions
fell only 20%); an integer finalizer in `hash_cell` (collisions up to
101M, 273 ms).
- Blocked: the gate's -30% is not met; three attempts gave at most -28%.
No stash: the -28% change is committed.
- [x] **P3 One workgroup per oversized body serialises its tests.**
- The problem: `OVERSIZED_PASS` (`physics_contacts.comp`) gives each
oversized body one 256-thread workgroup, which tests every body in the
scene, every contact round.
- Evidence: physics_bench falling 100000, `oversized` 30.4 ms of 238.5 ms
Physics per frame (12.7%) for one oversized body (`oversized 32` over
32 rounds), about 0.95 ms per round for 100K tests.
- Fix: split each oversized body over several workgroups, each writing a
partial max/min/or result to its own slot, then reduce the slots in a
fixed order in a second dispatch (per-body slots, no order-dependent
atomics).
- Gate: `oversized` at 100K falls at least 50%; the bit-identical physics
guards pass unchanged.
- Done: `OVERSIZED_PASS` splits each oversized body's tests over up to
`OVERSIZED_PARTS` (64) workgroups, each writing its max/min/or result
to its own slot of a new parts buffer (binding 16); `OVERSIZED_REDUCE`,
a fourth pass on the same indirect count, combines one body's parts in
a shared-memory tree and writes the body. Max, min and or are order-free,
so the result matches one workgroup testing every body. Reading the
parts buffer's length in the contact pass crashed the NVIDIA 615 SPIR-V
compiler at pipeline creation, so `parts_per_body` derives the slot
count from `pc.body_count` instead.
- Evidence: physics_bench falling 100000, 45 s, 3 alternating pairs
(`target/perf-loop/p3-{d,p3}-{1,2,3}.out`): `oversized` 27.57 to
2.85 ms per frame (-90%), Physics 180.14 to 142.27 ms (-21%); `oversized
32`, hash collisions 88939712 and fallback tests 6400032 unchanged. The
oversized-pebble GPU test now counts `4 * CONTACT_ROUNDS + 2` dispatches
and still matches bit for bit on a second run; full check passed (2687
passed, 0 failed), bit-identical guards unchanged. Earlier `oversized`
medians of exactly 32.0 ms were the measuring script also matching the
`oversized 32` counter; fixed.
### Sweep 3 (2026-10-17, after M2k-M2n and M3a)
Benchmarks in `target/perf-loop/sweep3/`, with another engine's job
(`strata`) holding the GPU 98-99% busy, so times are rough. Eco frame mean
7.03 ms, CPU 0.52 ms, upload 72 KB a frame; High frame mean 9.79 ms, CPU
0.67 ms; Eco 50K bears frame mean 17.8 ms, CPU 3.61 ms, instance upload
12.3 MB a frame (R3); Eco `--screens 6 --bears 5000 --bear-updates 500`
frame mean 18.8 ms, `screens` 3.39 ms of CPU, 399 screen draws. Counters
match sweep 2 (Eco 13 draws, High 23). M3a's `--lights 256` run is the
light-scaling evidence. Nsight on the screens run (frame-pointer build,
5024 samples): `render_screens` holds 53% of all samples. The grep audit
of `post_effects.rs` found only `Arc` clones.
- [-] **P4 Camera-screen feeds redo the main view's camera-free work.**
- The problem: `render_screens` (`scene_renderer.rs:2808`) gives each
feed its own `SceneRenderer`, and each feed's `render_frame` packs and
uploads every instance again (`prepare_render_instances`, `:7619`),
builds its own light clusters (`build_light_clusters`, `:1558`) and
shadow runs (`shadow_runs`, `:9112`), and builds a full command buffer.
The instance data does not depend on the camera. Feed uploads are not
counted: `screen_counters` sums only draws and triangles.
- Evidence: within `render_screens` (2660 samples): vulkano command
building 20.8%, `shadow_runs` 12.4% (`bounds_in_frustum` 9.8% self),
`prepare_render_instances` 10.5%, `build_light_clusters` 7.4%,
`collect_cull_readbacks` 3.9%, `PreparedGpuPhysics::clone` 2% of all
samples (R12). The `screens` phase is 3.39 ms (sweep 3) to 5.20 ms
(M2k) for 6 feeds, against 0.35 ms `record` for the main view.
- Fix: count feed uploads in `screen_counters` first, then let feeds
reuse the main renderer's packed instance buffer for the frame
(camera-free, written once) instead of packing and uploading their own.
Sharing shadows or lowering feed quality is a visible change, so it is
an owner decision.
- Gate: `screens` at `--screens 6 --bears 5000 --bear-updates 500` falls
at least 25% (median of 3), feed upload bytes reported and falling;
screen-feed tests and golden images unchanged.
Split into P4a and P4b.
- [x] **P4a** Count feed uploads in `screen_counters` and the report.
- [x] **P4b** Feeds reuse the main renderer's packed instance buffer.
- P4a Done: `render_screens` adds each feed's `upload_bytes` to `screen_counters`, and render_bench reports the total as `screen_upload_bytes` (`upload_bytes` still means the main view, so baselines compare). Before, feed uploads were not counted anywhere. `render_bench eco --screens 6 --bears 5000 --bear-updates 500` (`target/perf-loop/p4a-after.*`, `strata` on the GPU): the main view uploads 1.53 MB a frame, the six feeds 7.75 MB a frame (5x), which is the instance buffer once per feed; `screens` 2.95 ms. Counters are exact, so one run. Test: `benchmark_counts_screen_feeds_and_bears_apart` now also checks `screen_upload_bytes > 0`. Full check passed (2686 tests).
- P4b Done: a moved frame's instance upload is shared through `SceneRenderer::shared_instances` (revisions plus buffer). `render_screens` hands it to each feed and takes it back, and the move path reuses it when the renderables and layout revisions and the length match, instead of uploading its own copy. Packing reads only the render world, the assets and the physics indices, so the bytes are the same. `render_bench eco --screens 6 --bears 5000 --bear-updates 500 --repeat 3` (`target/perf-loop/p4b-{before,after}.*`, `strata` on the GPU at 60%, median of 3): main-view upload 1.53 MB to 0.074 MB a frame (instances part 1.46 MB to 3.3 KB), feed upload 7.75 to 2.13 MB a frame, `prepare` 0.18 to 0.10 ms, `screens` 3.07 to 2.71 ms (-12%), frame 11.62 to 11.15 ms. Draws (72), screen draws (399), triangles and screen triangles unchanged. Test: `screen_feeds_share_the_moved_instance_upload` (the feed and the main view hold the same instance buffer after a moved frame). Full check passed (2687 tests).
- Blocked: the `screens` part of the gate (-25%) is not met; -12% is what the instance upload gives. Packing itself is a matrix scan of microseconds. What is left per feed is camera-dependent: shadow runs, light clusters and command building. The item makes sharing shadows or lowering feed quality an owner decision. No stash: P4a and P4b are committed.
- [x] **P5 The scene light loop shades lights that cannot reach the pixel.**
- The problem: the cluster loop in `scene.frag` (`:655-742`) computes
range and cone attenuation, then runs the cookie lookup, the shadow
test and the whole Cook-Torrance term even when the attenuation is 0.
It skips only `n_dot_l <= 0`. Clusters hold each spot as its full
range sphere (`build_light_clusters`), so most pixels in a spot's
clusters are outside its cone.
- Evidence: M3a, `render_bench eco --lights 256 --repeat 3`: the Scene
pass grows from 1.51 to 5.01 ms (3.3x, about 14 us per clustered
light), more than a third of the frame.
- Fix: `continue` when the attenuation is 0 after the range fade and the
cone (before the cookie and the shadow test). Adding exact 0 changes no
pixel, so golden images stay. Later, test spots by their cone bounds
when building clusters.
- Gate: the Scene pass at `--lights 256` falls at least 25% (median of
3); golden images unchanged.
- P5 Done: the cluster loop in `scene.frag` now `continue`s when the range fade and cone leave the attenuation at 0, before the cookie, the shadow test and Cook-Torrance. The skipped term was finite times 0, so no pixel changes. `render_bench eco --lights 256 --repeat 3`, with another engine (`strata`) on the GPU (88-100% busy) in every set, so runs alternated before and after: before (`target/perf-loop/p5-before{,2,3}.*`) Scene 7.76, 4.99 and 5.17 ms, median 5.17; after (`p5-after{,2}.*`) Scene 3.68 and 3.67 ms: -29%, gate met. Frame median 8.03 to 7.46-7.85 ms within the contention noise. No new test: the change is a shader early-out whose output is unchanged, which the golden-image gpu-tests guard; timing needs real hardware. Spots tested by cone bounds when building clusters stay open as a later step. Full check passed (2686 tests, golden images unchanged).
### Sweep 4 (2026-10-17, after P5, P4a, P4b, P1 and P2)
Benchmarks in `target/perf-loop/sweep4/`, with another engine's job
(`gpu_model`) on the GPU, so times are rough. Counters match sweep 3 (Eco 13
draws, High 23, bears 241). Eco frame mean 3.49 ms, CPU 0.55 ms; High 4.91
ms, CPU 0.70 ms; Eco 50K bears 13.3 ms, CPU 4.45 ms (every CPU phase is
about 20% above sweep 3, including phases no item touched, so the host was
loaded); `--screens 6 --bears 5000 --bear-updates 500`: main upload 919.7 to
44.6 MB over the run and `screens` 3.39 to 2.48 ms (P4b). 100K falling:
`contacts` 154.5 ms, `Physics` 191-194 ms, CPU fixed 2.23-2.26 ms per tick,
extract 4.80 ms a frame. physics_suite matches sweep 3.
Nsight on 100K falling (frame-pointer build, main thread; its CPU side had
no profile before). The first 12.1 s of the 30.8 s run is the spawn, where
`spawn_renderable` holds 80% and `find_named_entity` 18% self (N1). The
4025 samples after it map to existing items, so no new P item:
| % | Function | Item |
|---|---|---|
| 28.6 | `stream_gpu_bodies` collects and sorts every body with no budget | N3 |
| 26.1 | `step_soft_bodies` collects a collider for every body with nothing to step | N2 |
| 18.7 | `extract_render_world` | R13 |
| 11.6 | `gather_bodies` mirrors every GPU body | N4 |
| 5.3 | `render_screens` clones `PreparedGpuPhysics` with no screens | N6 |
| 4.4 | `update_skins` with no skins | N7 |
The grep audit read the E1-E21 features (water, wetness, lens droplets,
clouds, particle blockers, viewmodels, wind), which no round covered: each
is gated on its component or a non-zero setting, so a scene without them
pays nothing.
### Sweep 5 (2026-10-17, after P3, M3b, M3c, M3d and M3e)
Benchmarks in `target/perf-loop/sweep5/`, with other engines' jobs
(`strata`, `gpu_model`) on the GPU. Counters match sweep 4 (Eco 13 draws,
High 23, bears 241). Eco frame mean 3.50 ms, CPU 0.57 ms; High 5.00 ms, CPU
0.71 ms; Eco 50K bears 12.6 ms, CPU 3.76 ms. 100K falling: `contacts` 154.5
to 126.4 ms and `oversized` 2.84 ms (P3), CPU fixed 2.36 ms per tick,
extract 4.80 ms a frame. physics_suite metrics are identical to sweep 4 and
step times are within noise.
Nsight on a frame-pointer build of the new M3 modes, main thread:
| Mode | % | Function | Item |
|---|---|---|---|
| `--rain` | 55.8 | `particles::advance` blocker `retain` (`ground_height` per drop) | S3 |
| `--rain` | 20.8 | `particle_draws`, 19.4 of it the stable depth sort | P7 |
| `--rain` | 7.9 | `particle_batch` builds a fresh instance list per emitter | P7 |
| `--effects` | 48.4 | vulkano `AutoCommandBufferBuilder::build` and `add_resources` | N8 |
| `--effects` | 19.5 | `draw_instances` (vulkano validation and resource tracking) | N8 |
| `--effects` | 13.8 | `collect_cull_readbacks`, 12.3 self | P6 |
The 2.2 MB a frame of the rain mode is the CPU particle instances (48 bytes
each). That cost comes with CPU particles, which are the only kind that
blockers stop, so no item for it. Packing the color to 8 bits would cut it
by a quarter at a precision cost, which is an owner call. The grep audit
covered the code changed since sweep 4 (P3's oversized reduce shader and the
bench-only M3 scenes): no per-frame allocation or scan on an engine path.
- [ ] **P6 The cull counters read the draw commands from write-combined
memory.**
- The problem: `collect_cull_readbacks` (`scene_renderer.rs:2666`) sums
`instance_count` and `index_count` over every indirect draw command of a
finished frame (`:2722-2732`). `gpu_cull_buffers` (`:9664`) allocates
those commands from `instance_allocator` (`:2263`, `PREFER_DEVICE |
HOST_SEQUENTIAL_WRITE`), so on a resizable-BAR GPU the CPU reads each
field across the bus uncached. The cost grows with the draw count.
- Evidence: Nsight on `render_bench eco --effects` (2933 draws): 13.8% of
the main thread's samples, 12.3% self, against 48.4% for all of
vulkano's `build`. At 13 draws (plain Eco) it does not show.
- Fix: after the last draw, copy each command buffer into a
`PREFER_HOST | HOST_RANDOM_ACCESS` buffer (the per-frame `transient`
allocator, `:2315`) with `copy_buffer`, and sum that copy. Or count
`visible` and `triangles` in the cull shader into a small counter
buffer (a counter that feeds no simulation, so atomics are fine).
- Gate: `collect_cull_readbacks` drops below 1% of main-thread samples in
`--effects`, and the `visible`, `culled` and triangle counters keep
their values (the P1 tests pass).
- [ ] **P7 CPU particle draws sort and copy every drop three times a
frame.**
- The problem: `particle_draws` (`scene_renderer.rs:11499`) copies each
batch with `to_vec` (`:11529`), stable-sorts each alpha batch with a
comparator that recomputes both depths per comparison (`:11531`),
builds a second `Vec<ParticleVertex>`, and the caller copies that into
the transient upload (`:5643-5650`). Before that, `particle_batch`
(`particles.rs:1381`) collects a fresh `ParticleInstance` list per
emitter every frame.
- Evidence: Nsight on `render_bench eco --rain` (50K CPU drops in 16
alpha emitters): `particle_draws` 20.8% of the main thread's samples,
19.4% in the sort; `particle_batch` 7.9%. M3e: record 3.01 ms against
0.25 ms plain.
- Fix: compute each drop's depth once, sort `(depth, index)` pairs with
`sort_unstable_by` and the index as tie-break (the same order as
today's stable sort, so the images stay), and write the vertices
straight into the transient slice. Then reuse the batch's instance
`Vec` across frames in extraction.
- Gate: `record` in `render_bench eco --rain --repeat 3` falls at least
40% (median of 3), with a test that the draw order of tied depths is
unchanged.
### Sweep 6 (2026-10-17, after M3f, M3g, M3h, N1 and T7)
Benchmarks in `target/perf-loop/sweep6/`, with `strata` kernels on the GPU.
Counters match sweep 5 (Eco 13 draws, High 23, bears 241, upload bytes
equal). Eco frame mean 3.45 ms, CPU 0.55 ms; High 4.87 ms, CPU 0.65 ms; Eco
50K bears 11.9 ms, CPU 3.54 ms. 100K falling: `contacts` 121.8-122.0 ms,
CPU fixed 2.24 ms per tick, extract 4.8-5.1 ms a frame. physics_suite
step times are within noise of sweep 5. No engine frame code changed since
sweep 5, so the Nsight hot spots of sweep 5 stand; this sweep timed the
tools instead, which no round had covered past `validate` and `lint`.
`rusting` scene commands on the 50K-cube T7 project (93 MB, 3 runs each,
load average 1.2): `scene inspect --summary` 0.64 s, `scene map` 0.11 s,
`scene query --name` 0.21 s, `project summary` 0.21 s, `validate` and
`fix --dry-run` 0.54 s (both 1.49 s before T7), `lint` 0.35 s, `diff` of
the scene with itself 2.49 s (linear: 0.26, 0.53, 1.05 s at 5K, 10K, 20K),
and `scene patch --dry-run` with one `set` 167-187 s. The grep audit of the
code changed since sweep 5 (the N1 observers, T7, the M3f-M3h bench and
test code) found no per-frame allocation or scan.
- [ ] **P8 `scene patch` is quadratic in the entity count.**
- The problem: `apply_tracked` (`scene_patch.rs:844-851`) clones the
whole entity list before each operation, then tests every entity with
`previous.contains(entity)`, a deep `Value` comparison against every
entity of the copy. Then `untouched` (`:856`) formats every entity
twice with `to_string`.
- Evidence: one `set` operation with `--dry-run`: 0.99 s at 5K, 3.2 s at
10K, 17.0 s at 20K, 167-187 s at 50K. An agent or script that edits a
big scene waits minutes per patch.
- Fix: `apply_operation` already knows which entities it changed. Have it
return their ids (or compare `entities[i]` with `previous[i]` by
position, with an id map only when the length changed), and drop the
per-operation clone. Mark untouched entities by id instead of by
string.
- Gate: one-`set` patch on 50K under 2 s; `touched` and the
`InvalidObject` operation numbers unchanged (the existing patch tests
pass), plus a test that a patch which changes one entity of many names
only that entity.
- [ ] **P9 Collab `diff_scene` is quadratic, and the editor runs it every
100 ms.**
- The problem: `diff_scene` (`collab/ops.rs:96`) finds each new entity's
base with a linear `find` over the old list, and each old entity's
deletion with `new.iter().any` (`:124`). `send_local_changes`
(`editor/collab_panel.rs:862`) calls it every `SCENE_POLL` (100 ms) in
a session, and a received op calls it through `pending_keys` (`:841`)
and again at `:848`, all on the editor's main thread.
- Evidence: one changed field: 8.5 ms at 1K entities, 0.89 s at 10K,
3.46 s at 20K (release, a scratch timing test). A 10K-entity collab
session spends most of every second diffing.
- Fix: index the old entities by id in a `HashMap<&str, &Map>` and the
new ids in a `HashSet<&str>` once per call. The op order stays the new
list's order, so the output is the same.
- Gate: `diff_scene` at 10K under 20 ms, and the existing `collab::ops`
tests pass.
- [ ] **P10 `scene inspect` parses the scene twice.**
- The problem: `inspect_scene` (`cli.rs:1638`) reads and parses the
whole file into a `serde_json::Value` again only to read
`format_version`.
- Evidence: 0.64 s at 50K, against 0.21 s for `scene query`, which reads
the scene once.
- Fix: deserialize the bytes into a struct with only `format_version`
(serde skips the rest without building values), or read the version
in `decode_scene` before the migration.
- Gate: `scene inspect --summary` at 50K under 0.4 s, with the same
output (`scene_version` still reports an old scene's version).
### Sweep 7 (2026-10-17, after T13, T9, T8a, X4 and U10a)
Benchmarks in `target/perf-loop/sweep7/`. `strata` was on the GPU for the
render runs. Counters match sweep 6: Eco 13 draws, High 23, bears 241, and
equal upload bytes. Eco frame mean 3.47 ms, CPU 0.55 ms; High 4.84 ms, CPU
0.65 ms; Eco 50K bears 11.8 ms, CPU 3.45 ms. 100K falling: `contacts`
122.9-123.1 ms, CPU fixed 2.29-2.31 ms per tick, extract 4.8 ms a frame.
`physics_bench` does not exit by itself, so it was stopped after 3 reports;
later sweeps should run it under `timeout`. The physics_suite step times are
within noise of sweep 6. `fracture_drop` read 1.32 ms once (0.91 ms in
sweep 6), but three reruns gave 0.91, 0.90 and 1.61 ms, and sweeps 1-6
read 0.90-1.02 ms, so it jumps between runs and has not regressed. No frame
code changed since sweep 6 apart from the asset pool (X4) and the audio
first play (U10a), so the Nsight hot spots of sweep 5 stand. This sweep
audited hot reload and audio, which no round had covered.
- [ ] **P11 The hot reload scan stats every watched file on the main
thread, in shipped games too.**
- The problem: `poll_asset_loads` (`assets/mod.rs`, after
`HOT_RELOAD_SCAN_INTERVAL`) runs in every game, every 500 ms. It calls
`reload_changed`, and through it `Assets::changed` (`:394`) for meshes,
textures and every data asset type. That runs one `std::fs::metadata` per
loaded path-backed asset on the main thread. A packed game has no loose
files to watch, so every stat fails, and it still pays for them.
- Evidence: a stat of a cached file costs about 1.3 µs (5000 stats in
6.5-7.2 ms, measured from Python, which adds a little of its own). A
game with 1000 assets therefore spends about 1.3 ms of one frame in
every 30 (60 fps) on the scan, 8% of a 16.6 ms frame. At 5000 assets
it is about 6.5 ms.
- Fix: skip the scan for assets that came from a pack (or when no loose
asset folder exists). Otherwise take the stats off the main thread:
hand the watched paths to the asset pool, which returns the changed
ones for the next `poll_asset_loads`.
- Gate: the frame that runs the scan at 2000 loaded textures costs under
0.1 ms more than the frames between, and the existing hot reload tests
pass (`changed_files_reload_on_workers_and_failures_keep_last_value`,
`changed_data_files_reload_into_a_running_game`).
## Loop log
- 2026-10-16 18:40 M2a done (baseline compare, JSON stdout, driver drift warnings); full check passed; next M2b.
- 2026-10-16 19:00 M2b done (per-tick CPU fixed and GPU Physics on the perf line), commit follows 416abaa; next M2c.
- 2026-10-16 19:20 M2c done (bench CPU no longer counts extraction twice), commit follows 6d76b63; next M2d.
- 2026-10-16 19:40 M2d done (cpu_phases_ms in RenderBenchmarkReport), commit follows bc5bcd5; next M2e.
- 2026-10-16 20:00 M2e done (gpu_memory_bytes reports in-use bytes, 1 GiB to 103.5 MiB), commit follows 99c53cd; next M2f.
- 2026-10-16 20:30 M2f done (slow-motion warning prints at 100K, where frames are GPU-bound), commit follows 19cf47c; discovery sweep next, then M2g.
- 2026-10-16 21:00 Discovery sweep 1 (benchmarks plus Nsight on 50K bears): hot spots map to R13, S4, N7, T2, R2, N8; added P1 (cull counters read 0); next M2g.
- 2026-10-16 21:15 M2g done (benchmarks/README.md current), commit follows a790e34; next M2h.
- 2026-10-16 21:25 M2h done (physics_bench loads the source scene, no stale-cook warning or NotFound), commit follows 9a5e7d0; next M2i.
- 2026-10-16 21:45 M2ia done (per-dispatch physics parts on the perf line; contacts are 82% of physics at 100K), commit follows 7be9522; M2ib split off; next M2j.
- 2026-10-16 22:05 M2ib done (contact counters on the perf line; 88.8M hash collisions per frame at 100K), commit follows f0903e5; next M2j.
- 2026-10-16 22:30 M2j done (render_bench `wait` and `untimed` phases; the frame gap is in the fence wait), commit follows 8e9c6a3; next M2k.
- 2026-10-16 22:45 Discovery sweep 2: added P2 (contact pass per-round cost) and P3 (oversized pass serialised); corrected the M2ib cause; next M2k.
- 2026-10-16 23:05 M2k done (render_bench `bears` and `screens` phases, screen draws and triangles; six feeds cost 5.2 ms CPU), commit follows 3dc6b4d; next M2l.
- 2026-10-16 23:20 M2l done (render_bench GPU busy state and `--repeat N` minimum, median and spread warning), commit follows 6bf2693; next M2m.
- 2026-10-16 23:40 M2m done (upload bytes split into six parts; light clusters are 94% of eco's uploads), commit follows ba04ec7; next M2n.
- 2026-10-16 23:55 M2n done (release headless instances skip `ext_debug_utils`; record -5%), commit follows 2c46613; M2 done; next M1.
- 2026-10-17 00:10 M2n committed (856e0e8). M1 marked blocked: it writes `benchmarks/` and needs an idle GPU, so the owner records it; next M3.
- 2026-10-17 00:40 M3 split into M3a-M3h; M3a done (render_bench `--lights N`; 256 lights take Scene from 1.5 to 5.0 ms on Eco), commit follows 6cc0442; next discovery sweep 3.
- 2026-10-17 00:55 M3a committed (53eae9b). Discovery sweep 3 (benchmarks, Nsight on 6 camera screens, audit of `post_effects.rs`): added P4 (screen feeds repack instances, shadows and clusters per feed) and P5 (light loop shades zero-attenuation lights); next P5.
- 2026-10-17 01:20 P5 done (light loop skips zero-attenuation lights; Scene at 256 lights 5.17 to 3.67 ms), commit follows a233240; next P4.
- 2026-10-17 01:35 P4 split into P4a-P4b; P4a done (feed uploads counted: 7.75 MB a frame for 6 feeds against 1.53 MB in the main view), commit follows a15ff8b; next P4b.
- 2026-10-17 02:05 P4b done (feeds share the moved-instance upload: feed upload 7.75 to 2.13 MB a frame, `screens` 3.07 to 2.71 ms); P4 blocked on the 25% gate (owner decision on shared shadows); commit follows c984bb5; next P1.
- 2026-10-17 02:35 P1 done (GPU-cull counters: a zero wait on the signaled fence releases the readback; 10K perf lines 16/18 `triangles 0` to 0/20), commit follows dcaf617; next P2.
- 2026-10-17 03:40 P2 in part (contact pass skips other-cell slot entries before loading them: `contacts` at 100K 223.67 to 161.16 ms, -28%); blocked on the 30% gate after 3 attempts; commit follows b962a7a; next sweep 4.
- 2026-10-17 04:20 Discovery sweep 4 (benchmarks, Nsight on 100K falling, audit of E1-E21): every hot spot maps to N1-N4, N6, N7 or R13; no new P item; next P3.
- 2026-10-17 05:05 P3 done (oversized bodies split over up to 64 workgroups plus a reduce pass: `oversized` at 100K 27.57 to 2.85 ms, -90%); full check passed; commit follows 6a93768; next M3b.
- 2026-10-17 05:35 M3b done (render_bench `--city N`: 10K buildings, Eco frame 4.75 to 39.15 ms, CPU record 22.4 ms for 2214 draws); full check passed; commit follows 458a458; next M3c.
- 2026-10-17 05:33 M3c done (render_bench `--effects`: 50K GPU particles, 2K blended spheres, 300 GPU-skinned characters; Eco frame 3.62 to 49.83 ms, CPU record 27 ms for 2933 draws); full check passed; commit follows 197dac9; next M3d.
- 2026-10-17 05:55 M3d done (render_bench `--probes`: 64-probe grid on High, frame 4.97 to 11.67 ms, all of it untimed probe relight; R1a/R1b target); full check passed; commit follows 0fef9bf; next M3e.
- 2026-10-17 06:22 M3e done (render_bench `--rain`: 50K CPU drops with blockers on a 257x257 heightfield; Eco frame 3.74 to 19.04 ms, fixed 8.23 ms, 2.2 MB a frame upload); full check passed; commit follows 7dde9e4; 5 items since sweep 4, so sweep 5 next.
- 2026-10-17 07:20 Discovery sweep 5 (benchmarks plus Nsight on `--rain` and `--effects`): counters match sweep 4, `contacts` at 100K down to 126.4 ms; hot spots map to S3 and N8; added P6 (cull counters read write-combined memory) and P7 (CPU particle draw sort and copies); next M3f.
- 2026-10-17 07:40 M3f done (`physics_suite <path>` writes or checks a JSON `SuiteReport` baseline under render_bench's rules; 29.6 s a run, no regressions on repeat, a tightened copy exits 1); full check passed; commit follows c5c28b6; next M3g.
- 2026-10-17 08:00 M3g done (editor timings on 10K entities: save 44.1, load 34.5, idle 2.93, edit 8.61, Undo frame 18.73 ms; history 100x one snapshot, the L14a target); full check passed; commit follows 1be699c; next M3h.
- 2026-10-17 08:45 M3h done (load test: 50 1024x1024 textures and 200 prefab copies, first playable frame 320 ms cooked, 326 ms source); M3 done, claim dropped; full check passed; commit follows 825cc1a; next N1.
- 2026-10-17 09:15 N1 done (name index kept by `Name` observers, scan dropped; `debris 100000` first frame 10.74 s to 0.26 s); full check passed; commit follows 1579681; 4 items since sweep 5; next T7.
- 2026-10-17 10:05 T7 done (`dropped_fields` compares entities one at a time on rayon, no whole-document clones; `validate` on 50K 1.50 s to 0.52 s, same output); full check passed; commit follows f1af956; 5 items since sweep 5; next discovery sweep 6.
- 2026-10-17 11:40 Discovery sweep 6 (benchmarks plus `rusting` scene commands at 50K): counters match sweep 5; T7 also made `fix` 1.49 to 0.54 s; added P8 (`scene patch` quadratic, 170 s at 50K), P9 (collab `diff_scene` quadratic, 0.89 s at 10K every 100 ms) and P10 (`scene inspect` parses twice); next T13.
- 2026-10-17 12:00 T13 done (regression scenes split into 8 tests, both runs on threads; 186.7 to 57.3 s, library suite 211 to 89 s); full check passed; commit follows 63b8b90; 1 item since sweep 6; next T9.
- 12:19 T9 done (shared Cargo build dir per engine version: a second new project's first run 118-132 s to 1.9 s, `target/` 1.9 GB to 46 MB; the 7.1 s scene-edit rebuild did not reproduce); full check passed (2723 tests); 2 items since sweep 6; next T8.
- 12:58 T8 split into T8a-T8c; T8a done (cook skipped on an unchanged project: cook 140 to 1 ms, warm `run --ticks 1` 462 to 312 ms at 10K entities); T8b blocked (owner decision, CLI output change), T8c blocked (120 ms cargo no-op, would redo Cargo's freshness check); full check passed (2725 tests); 3 items since sweep 6; next X4.
- 13:45 X4 done (asset loads on their own 2-4 thread pool: tick p99 during 50 texture loads 108.5 to 3.0 ms); full check passed (2727 tests); 4 items since sweep 6; next U10.
- 14:10 U10 split into U10a-U10c; U10a done (first play of a 60 s clip streams while the asset pool decodes it: 33.9 to 0.25 ms MP3, 33.2 to 1.2 ms Vorbis); U10b and U10c blocked (gain under the noise); full check passed (2729 tests); 5 items since sweep 6; next discovery sweep 7.
- 15:30 Discovery sweep 7 (benchmarks, plus an audit of hot reload and audio): counters match sweep 6, times within noise (`fracture_drop` jumps 0.90-1.61 ms between runs); added P11 (hot reload stats every watched file on the main thread every 500 ms, about 1.3 ms at 1000 assets); next U6.
- 16:00 U6 done (follow targets cached by name; camera follow at 50K names 0.024 to 0.0002 ms); 1 item since sweep 7; full check passed (2731 tests); next U9.
- 16:55 U9 done (scenario invariants share one lookup and scene form per entity per tick; 4 invariants at 20K entities 0.92 to 0.17 ms a tick); full check passed (2731 tests); 2 items since sweep 7; next L1.
- 17:35 L1 done (folder-scene stale-file sweep uses a name set; unchanged save of 10K entities 7.0 s to 32 ms); full check passed (2731 tests); 3 items since sweep 7; next L2.
- 18:15 L2 done (inline material dedup uses a name and color index; 10K renderers with 10K materials 202 ms to 2.0 ms); full check passed (2733 tests); 4 items since sweep 7; next L5a.