# Performance & hardware findings
## Milestone 2a — GBM scanout + KMS present (2026-10-26)
Host: x86_64 Arch, NVIDIA RTX 5070 Ti (proprietary driver), DP-1 @ 2560x1440@60.
This is a proxy only; the target is Pi 5 / Mesa v3dv, whose GBM modifier and
Vulkan-import behaviour differ (see below).
GBM `gbm_bo_create` flag compatibility on NVIDIA (XRGB8888, 2560x1440):
| SCANOUT\|RENDERING\|WRITE\|LINEAR | FAIL | - | - |
| SCANOUT\|WRITE\|LINEAR | ok | Linear | ok |
| SCANOUT\|LINEAR | ok | Linear | ok |
| SCANOUT\|RENDERING | ok | vendor tiled | ok |
| SCANOUT | ok | vendor tiled | FAIL |
| RENDERING / RENDERING\|LINEAR | FAIL | - | - |
Takeaways:
- NVIDIA rejects `RENDERING|WRITE` together. Keep CPU-filled buffers
(`SCANOUT|WRITE|LINEAR`) separate from GPU-rendered ones (`SCANOUT|RENDERING`).
- `SCANOUT` alone yields a tiled modifier that AddFB2 then rejects — always pair
SCANOUT with an explicit usage.
- AddFB2 with explicit modifiers (FbCmd2Flags::MODIFIERS) works; our BoFb adapter
reports None only for `DrmModifier::Invalid`, else passes the modifier through.
Benchmark (blocking atomic commits = vsync-locked), 2560x1440 XRGB8888:
| flip only | 300 | 54.9 | 18.22 |
| fill+flip | 120 | 59.8 | 16.73 |
Both are display-refresh bound (60 Hz), i.e. present keeps up. A full-screen CPU
fill (~14 MB) plus commit fits inside the 16.7 ms budget with headroom, so the
present path is not the bottleneck. Real UI rendering will be Vulkan (2c); this
only validates the scanout pipeline.
TODO on Pi 5: re-run the flag matrix under v3dv; confirm `SCANOUT|RENDERING`
buffers import into both KMS (AddFB2) and Vulkan (dmabuf) with matching
modifiers.
## Milestone 2b — Vulkan bring-up (2026-10-26)
libvulkan loads via `ash` (runtime loader, not linked). On the NVIDIA host:
- device: NVIDIA GeForce RTX 5070 Ti [DISCRETE_GPU], Vulkan 1.4.351, graphics
queue family 0.
- all four dmabuf-import extensions present: `VK_KHR_external_memory_fd`,
`VK_EXT_external_memory_dma_buf`, `VK_EXT_image_drm_format_modifier`,
`VK_KHR_external_semaphore_fd`.
- logical device + graphics queue created headless (no WSI/swapchain).
So the zero-copy import path (2c) is testable on this host too, not only on the
Pi. v3dv on the Pi remains the real target; re-verify extension availability
there (v3dv is Vulkan 1.2-era; `image_drm_format_modifier` and dmabuf external
memory are supported by Mesa, but confirm on-device).
## Milestone 2c — Vulkan render into scanout dmabuf (2026-10-26)
End-to-end zero-copy on the NVIDIA host:
- gbm `SCANOUT|RENDERING` buffer, modifier `0x300000000606014` (NVIDIA block
linear), stride 10240, 1 plane.
- dmabuf imported as a `VkImage` (B8G8R8A8_UNORM, tiling DRM_FORMAT_MODIFIER_EXT,
`ImageDrmFormatModifierExplicitCreateInfoEXT` + explicit plane layout), memory
imported via `ImportMemoryFdInfoKHR` + dedicated allocation, memory type from
`req.memory_type_bits & VkMemoryFdPropertiesKHR.memory_type_bits`.
- per frame: `vkCmdClearColorImage` (UNDEFINED->GENERAL barrier), submit, wait
fence, KMS atomic flip. No blit: the GPU writes the buffer KMS scans out.
Benchmark (2560x1440 XRGB8888, vsync-locked):
| flip only | 56.0 | 17.84 |
| fill+flip | 56.0 | 17.84 |
| clear+flip | 59.9 | 16.70 |
The Vulkan clear+flip holds a clean 60 Hz lock. ash 0.38 has no safe wrapper for
`vkGetMemoryFdPropertiesKHR`; call it through `Device::fp()`.
Caveats / Pi TODO:
- Sync is a CPU fence wait per frame. For video + UI at 60 fps this should move
to a KMS OUT_FENCE / Vulkan timeline semaphore (VK_KHR_external_semaphore_fd)
so we don't stall the CPU. Fine for bring-up.
- Image left in GENERAL layout; no queue-family transfer to FOREIGN. Works on
same-driver NVIDIA; verify on v3dv, add QFOT if the Pi needs it.
## Milestone 3 — Vulkan UI pipeline (2026-10-26)
Real graphics pipeline into the scanout dmabuf: render pass (loadOp CLEAR,
finalLayout GENERAL for scanout), host-visible vertex buffer (fullscreen quad,
per-vertex color), SPIR-V shaders compiled offline with glslc and embedded via
include_bytes!, push-constant animation, dynamic viewport/scissor.
| draw+flip | 59.9 | 16.70 |
Same 60 Hz lock as the clear path, i.e. the pipeline adds no measurable cost
while display-bound. Deferred to the compositor milestone: IN_FENCE/OUT_FENCE
sync (no measurable benefit while vsync-bound and better designed with the video
plane present), and text/font rendering (UI shell).
## Refresh-rate matching, Pi 5 at 3840x2160 (2026-11-01)
`video-render` (mpv GL render + glFinish, 1920x1080 target), MEDIAS_STATS:
| HEVC Main 10 23.976, hwdec | 60 Hz | 31-35 ms |
| HEVC Main 10 23.976, hwdec | 23.976 Hz| 18 ms |
| H.264 25 fps, software | 60 Hz | 31 ms |
| H.264 25 fps, software | 50 Hz | 23-24 ms |
Cause not measured; candidates: GPU/memory contention with 60 Hz scanout and
UI-plane commits.
## Milestone 4b — mpv video onto a KMS plane (2026-10-26)
Path: EGL 1.5 display on the GBM device (EGL_KHR_platform_gbm) + surfaceless
`mpv_render_context_render` into that FBO -> glFinish -> KMS flip. The same
buffer is the KMS framebuffer, so no copy.
NVIDIA host, lavfi testsrc (30 fps, 3 s), full-screen 2560x1440:
rendered 83 new frames in 3.2 s (~26 fps effective; frame-gated by
`mpv_render_context_update`), stopped at END_FILE. Confirms decode -> GL ->
scanout.
Notes:
- `--vo=libmpv` is required for the render API.
- render-into-imported-dmabuf (via `glEGLImageTargetRenderbufferStorageOES`)
works on NVIDIA EGL; Mesa/v3d supports the same path. Verify on Pi.
- Sync is glFinish per frame (coarse). Compositor milestone: dmabuf/sync_file
fences, and this video buffer on its own plane under the Vulkan UI overlay.
- hwdec is off on the host; Pi uses v4l2/drm-prime (wire in 4b-on-Pi).
## Milestone 5 — two-plane compositor (2026-10-26)
One atomic commit drives both planes: mpv video (XRGB, underlay) + Vulkan UI
(ARGB, overlay, premultiplied alpha), hardware-blended. UI shader clears to
transparent and draws a translucent bottom bar so video shows through.
NVIDIA host, testsrc looping, 2560x1440:
composited 554 frames in 11.0 s = **50.3 fps**.
The ~10 fps shortfall vs. 60 is serial blocking sync per frame: `glFinish`
(video) + `vkWaitForFences` (UI) + blocking atomic commit, with no pipelining.
This is the first measurement that justifies explicit fencing — earlier, single
producer, we stayed at 60 so it bought nothing. Next perf step: dmabuf/sync_file
IN_FENCE per plane + OUT_FENCE + non-blocking commits so GPU/mpv/scanout overlap.
Not visually verified here (no eyes on the panel); the atomic commit of an ARGB
overlay over an XRGB primary succeeds on NVIDIA. Confirm blending on the Pi.
## Pi 5 bring-up (on-device)
First real hardware run (vc4 display + v3d render, Mesa). Findings:
- DRM nodes: card0 = v3d (render, rejects atomic cap -> ENOTSUP), card1 = vc4
(display). Auto-detect scans for the atomic-capable node with a connected
output.
- Needed explicit `acquire_master_lock()` before modeset (EACCES otherwise);
run from the active HDMI console VT, not SSH, or free the console.
- EGL config: Mesa/GBM configs are WINDOW_BIT (no PBUFFER_BIT like NVIDIA);
choose WINDOW_BIT, fall back to PBUFFER.
- v3dv Vulkan + EGL dma_buf import both work; the dmabuf flows vc4<->v3d via
Mesa renderonly transparently.
- Video was upside down with mpv FLIP_Y=1 -> the GL FBO-backed dmabuf is already
top-down for KMS; FLIP_Y=0 is correct.
- Flicker: single-buffering (render into the buffer being scanned out). Fixed
with double-buffering per plane (ping-pong).
- Audio: ALSA dmix "unable to open slave" -> HDMI device not the default PCM.
Set MEDIAS_AUDIO_DEVICE (from `aplay -L`), e.g. alsa/sysdefault:CARD=vc4hdmi0.
TODO: hwdec (currently software; wire v4l2/drm-prime), visual pass on text/UI,
explicit fencing.
## CPU UI rasterizer (2026-10-31)
`render_tests::cpu_raster_matches_vulkan` (ignored; needs Vulkan): a 1920x1080
grid of 10 cards (7 thumbnails, wrapped titles), rendered by Vulkan and by
`ui::raster::Canvas`.
- CPU time (render + copy into a linear buffer): 8.1 ms/frame on a
Ryzen 9 7945HX3D (VM, 12 vCPUs; single-threaded). RK3399 not yet measured.
## Orange Pi Zero3 (2026-11-01)
H618 (4x A53 1.5 GHz), Mali-G31 Panfrost (Mesa 25.0.7, GLES 3.1, no Vulkan),
sun8i mixer, vendor kernel, Debian 13. 1920x1080@60.003, CPU UI rasterizer.
1080p23.976 H.264, software decode, MEDIAS_STATS steady state:
| default | 72-75 ms | 12 fps | 12 |
| `profile=fast` | 29-33 ms | 24 fps | 0 |
- UI render (CPU rasterizer): 15-36 ms per redraw.
- Planes: UI on overlay 33 (zpos 2), video on primary 39 (zpos 0). Without
explicit zpos the opaque video plane covered the UI.
- Audio: ALSA `default` routes to pipewire-pulse and times out (PipeWire has
only `auto_null`). `alsa/plughw:CARD=HDMI,DEV=0` works.
## 4K HEVC Main 10 on the Pi 5 (2026-11-13)
3840x2160 display, 1920x1080 render target, display at the video rate,
MEDIAS_STATS steady state. File A: 3840x1604, 24 fps, bt.2020/pq, Dolby Vision
profile 8, 4.71 Mbit/s. File B: 1920x960, 23.976 fps, bt.709, 3.70 Mbit/s.
| A | hwdec drm| default | 69-71 ms | 12 fps |
| A | hwdec drm| hdr-compute-peak=no | 66-69 ms | 12 fps |
| A | hwdec drm| + tone-mapping=clip, dither=no | 70 ms | 12 fps |
| A | hwdec drm| + profile=fast | 68-70 ms | 12 fps |
| A | software | default | 132-135 ms | 5 fps |
| B | hwdec drm| default | 18-20 ms | 24 fps |
- Shader options do not change the steady-state time of file A. Redraws of an
unchanged frame (paused during the refresh switch) take 77 ms by default,
30 ms with the three options and 7 ms with profile=fast, so the per-new-frame
cost is about 60 ms and lies before shading.
- Time scales with source size: A has 3.3x the pixels of B and takes 3.7x.
- Hypothesis, not verified: per-frame conversion of the decoder's SAND30
(P030) dmabuf in the V3D GL import path.
### Direct scanout (MEDIAS_DIRECT=1, 3f9bfe6)
mpv drmprime-overlay: the decoded dmabuf is the primary plane's framebuffer;
`video-render` covers only mpv's GL pass for the subtitle/OSD plane.
| A | 3.0-3.5 ms | 24 fps | 0/s |
| B | 4.6-12.2 ms | 24 fps | 0/s |
- File A: 70 ms -> 3 ms. File B varies between 4.6 and 12.2 ms within one run
(11-12 ms with the OSD hidden, 6 ms after a seek); cause not measured.
## Tone-mapping pass on the Pi 5 (2026-11-13)
MEDIAS_TONEMAP=1 MEDIAS_HDR=0, HEVC Main 10 3840x1604 PQ (Dolby Vision
profile 8), 24 fps, 3840x2160@24 output, commit d1e1d3e. `video-render` covers
mpv's GL pass plus the Vulkan pass and its synchronous wait.
| off | 9.6-9.8 ms | 24 fps | 1 |
| on | 30.9 ms | 24 fps | 0 |
- Pass cost by difference: 21 ms for 6.16 Mpixel; frame budget 41.7 ms.
- Second run: 30.0-30.2 ms, then 22.1-22.4 ms after one interval with 10
presents (cause of the step not measured; the pass-off time shows the same
kind of step, 4.6-12.2 ms above).
- Colours correct, motion smooth (observed, LG TV).
First shader (10 texture reads per pixel), HEVC Main 10 3840x2160 PQ, 60 fps,
3840x2160@60 output, commit d1e1d3e:
| on | 33.2-33.7 ms | 20-22 fps | 40 /s |
- Frame interval 16.7 ms; each render spans three refresh intervals.
- A 24 fps file of this size would fit (41.7 ms); 60 fps needs a video-render
below about 15 ms.
Second shader (3 texture reads per pixel), same file and output, commit b678f49:
32.4 and 35.4 ms mean video-render over frames 7-30 (two runs), against
33.2-33.7 ms for the first shader: no gain. mpv's pass alone: 2.9-3.8 ms.
The time is therefore not set by the number of table reads; cause not yet
measured (test `bench_pass_variants`).
`bench_pass_variants` on the Pi 5 (commit fb6987a, 3840x2160, offscreen tiled
target, planes in host memory, random plane contents):
| constant colour | 4.9 ms |
| luma read | 6.5 ms |
| luma and chroma reads, matrix | 10.5 ms |
| table lookup, random coordinate | 12.2 ms |
| full, random samples | 153.5 ms |
- The 3D table costs 7.3 ms per 8.3 Mpixel with incoherent coordinates on its
own; with random samples behind the plane reads, 143 ms. The table is
cache-bound: its cost depends on the picture content.
- Live, same 60 fps file: 22.3 ms video-render, of which 18.5 ms submit and
wait of the pass (mean including the first frames). Earlier runs of this
file: 32.4 and 35.4 ms; cause of the spread not measured.
- Floor for one fragment per 4K pixel with both plane reads: 10.5 ms. With
mpv's 3 ms this leaves 1.5 ms of a 15 ms limit for the colour map: 60 fps at
3840x2160 is not reachable with this pass structure.
`bench_pass_variants` with smooth plane contents (commit abafad4, Pi 5):
| full, 3840x2160 | 11.8 ms |
| full at half size, 1920x1080| 4.5 ms |
| constant colour | 4.9 ms |
| luma read | 6.4 ms |
| luma and chroma reads | 10.5 ms |
- With coherent table coordinates the table adds 1.3 ms.
- Live the pass took 18.5 ms on the GPU (mean including the first frames),
7 ms above the benchmark. Not separated: first-frame setup in the mean,
decoder dmabuf memory, linear scanout target, picture content.
MEDIAS_TONEMAP=half live (Pi 5, HEVC Main 10 3840x2160 PQ 60 fps,
3840x2160@60 output):
| 3840x2160 | 22.3 ms | 18.5 ms | 18-22 fps |
| 1920x1080 | 9.2-10.0 ms | 6.6 ms | 59-60 fps |
- Live GPU time is 1.5 times the benchmark at both sizes (18.5 / 11.8,
6.6 / 4.5). Not separated: decoder dmabuf memory, linear scanout target,
memory bandwidth shared with the running decoder.
Dolby Vision profile 5 through the pass (Pi 5, 4K 24 fps profile 5 file,
commit 56cb2b5): 19.0 ms video-render, 13.6 ms pass on the GPU, limit 37.5 ms;
24 fps presented, 17.2-19.0 ms in steady state. Colours correct (observed).
Dolby Vision profile 5 to HDR10 (Pi 5, 4K 24 fps profile 5 file, XR30
targets, commit e25c47c): 27.0-27.6 ms video-render, 15.6 ms pass on the GPU,
limit 37.5 ms; 24 fps presented, 0 drops. Against the SDR path of the same
pass (19.0 / 13.6 ms): 2 ms more on the GPU, 6 ms more outside it; cause of
the latter not measured.