# Rendering
A UI plugin paints *surfaces*: regions the viewer places (a sidebar card, a
pane footer, the stage) and sizes. Each surface is in one of two models,
fixed by the manifest:
| `cells` | a terminal cell | a grapheme, foreground, background, attributes | `Surface<Cells>` |
| `pixels` | a pixel | RGBA8 (straight alpha) | `Surface<Pixels>` |
Examples: [`clock_card.rs`](../examples/clock_card.rs)
(cells) and [`shapes_stage.rs`](../examples/shapes_stage.rs)
(pixels and the shape helpers).
## The shared buffer
A surface's pixels or cells live in the plugin's own linear memory. The
viewer reads them directly; nothing is serialized per frame.
```text
offset 0 header, 8 little-endian u32 words:
seq, model (0 cells, 1 pixels), cols, rows,
px-w, px-h, cell-px-w, cell-px-h
offset 32 slot 0
offset 32+slot-len slot 1
```
- A cell is 16 bytes: `grapheme: u32` (0 empty, else a Unicode scalar),
`fg: u32`, `bg: u32`, `attrs: u16` (bold 1, dim 2, italic 4, underline 8,
reverse 16, strikethrough 32), two bytes of padding.
- A colour word's top byte is its model: `0x00______` default (theme text,
or a transparent background), `0x01RRGGBB` RGB, `0x020000II` terminal
palette index, `0x030000TT` theme slot (0 fg, 1 bg, 2 recede-fg,
3 recede-bg, 4 accent). The SDK's `Colour` encodes these; prefer theme
slots (`Colour::FG`, `Colour::ACCENT`, `Colour::RECEDE_FG`, ...) so the
plugin follows the user's terminal colours.
- A pixel is 4 bytes, `r, g, b, a`, rows top to bottom;
`px-w = cols * cell-px-w`.
The buffer is double-buffered. The plugin paints one slot and commits it
with the rectangles that changed; the viewer samples only committed slots,
only the changed rectangles, and only when a frame is due and the surface
is visible. A half-painted slot is never shown.
The SDK does all of this for you:
- `Surface::new(id)` asks the host for the layout, allocates the region,
writes the header and attaches it.
- Every drawing call records the rectangle it touched. `commit()` sends the
merged rectangles (collapsed to their bounding box past 16), bumps the
header's sequence number, and switches slots. With nothing drawn,
`commit()` returns `Ok(false)` and makes no host call: no frame.
- The next slot starts as a copy of what was just committed, copied lazily
and only where it changed, so drawing is always incremental: change one
cell, commit one cell.
- To repaint everything (an animation), skip that copy: `Surface<Pixels>::
repaint_all()` hands out the whole slot with stale contents and marks it
all dirty, and `clear()` does the same for either model.
## Cells
```text
put(x, y, char, style) -> columns text(x, y, &str, style) -> columns
fill(rect, char, style) clear() / clear_with(style)
cell(x, y) -> Option<Cell> row_text(y)
```
`text` walks the string by grapheme cluster and writes each cluster's first
character: a cell holds one scalar (the contract reserves a grapheme pool
for longer clusters; it is not implemented), so combining marks and
variation selectors are dropped. Widths follow the viewer's rule, the
scalar's East Asian width:
- A wide character takes two cells: the character, then an empty cell.
One that would not fit before the right edge leaves one empty cell
instead of half a glyph.
- Overwriting half of a wide character empties the other half and marks
it dirty, so the viewer never shows a stale half.
- Control, bidi-control and zero-width characters are never written.
`Style` carries `fg`, `bg` and `attrs`: `Style::fg(Colour::ACCENT).bold()`.
Glyphs: write Nerd Font icons at their standard codepoints. The viewer
installs its own symbols font (Standard Code Symbols, drawn from Symbols
Nerd Font) and moves Nerd codepoints in plugin cells to that font's own
copies, so plugin icons match the interface's in every terminal that can
reach the font. Where one cannot (Warp, which draws only its bundled fonts),
the viewer shows Unicode symbols instead. Write Legacy Computing characters
(U+1FB00 to U+1FBFF) at their real codepoints too: in a terminal that
cannot draw one, the native viewer draws its fallback in that cell (the
checkerboard U+1FB95 becomes the closest pattern that terminal can draw).
Ship no font and choose no codepoint by terminal.
Put a space cell after every Nerd icon: write `"\u{f062} 2"`, not
`"\u{f062}2"`, and separate two icons with a space. Ghostty and kitty draw a
private-use glyph at its full size only when a plain space (not a no-break
space) follows it. Before text or another icon, Ghostty shrinks it to one
cell, so the same icon shows at two sizes. The rule holds at a surface's
right edge too: the cell past the last column belongs to the host's frame,
so an icon there has no space after it. Keep the last column free of an
icon. The interface follows the same rule.
Colours follow the viewer. `Colour::Theme(slot)` (`FG`, `BG`, `RECEDE_FG`,
`RECEDE_BG`, `ACCENT`) and `Colour::Indexed(i)` resolve to the viewer's
theme and terminal palette when it paints; `view::theme()` reports them as
RGB, `palette` included (the 16 ANSI colours as this viewer shows them),
for plugins that blend colours themselves. "Grayed out" means receded
toward the viewer's background, never a fixed gray:
`theme.recede(rgb, GRAYED_OUT_PERCENT)` (or `recede(rgb, toward, percent)`)
blends as the viewer does, keeping the colour's hue on light, dark and
tinted themes. `view::surface_tint(id)` is the identity tint of what an
instance surface belongs to (a machine row's machine, a project row's
project, a pane footer's project, else its machine) and `view::identity_tint(id)` any machine's or
project's, so a row can carry the same colour the sidebar gives its owner.
## Pixels
```text
set_pixel(x, y, rgba) blend_pixel(x, y, rgba) pixel(x, y)
clear(rgba) blit(x, y, w, h, &rgba8)
blit_clipped(x, y, Image::new(w, h, &rgba8), clip, blend)
pixels_mut() pixels_untracked() + mark_dirty(rect)
repaint_all()
```
`blit` copies an RGBA8 image, clipped to the surface. `blit_clipped` also
clips to an `Area` (a sprite drawn into a viewport, a scrolling strip) and,
with `blend`, draws each pixel over what is there with its alpha; only the
pixels it writes are marked dirty.
`pixels_mut()` gives the whole slot (holding the last frame) and marks it
all dirty; for small changes use the drawing calls, or write through
`pixels_untracked()` and `mark_dirty` what you changed.
Whether pixels reach the terminal as graphics depends on the viewer:
`capabilities::get().graphics` is true when it paints them as real images
(the kitty graphics protocol, `kitty`); without it the viewer shows a
pixels surface as half-block cells at one by two pixels per cell, so a
pixel plugin still works everywhere, at lower resolution.
`capabilities::get()` also reports the paced `frame_rate` and the cell
pixel size; `Event::CapabilitiesChanged` says when any of them changed.
## Shapes
`Shapes` is implemented by both surface types, with an `Ink` of `Style`
for cells and `Rgba` for pixels:
| `stroke_rect`, `fill_rect` | box-drawing border; full blocks in the style's foreground | one-pixel border; filled (blended when not opaque) |
| `line` | `─ │ ╲ ╱` along a Bresenham line | antialiased (Wu) |
| `bar(area, fraction, fill, track)` | full blocks, one eighth block, `░` track | filled with a coverage edge |
| `rounded_rect`, `fill_rounded_rect` | `╭╮╰╯` corners; quarter blocks when filled | antialiased corners of the radius |
| `circle`, `fill_circle` | `•` outline or full blocks; the radius counts rows and spans twice as many columns | antialiased |
| `fill_rect_at(x, y, w, h, colour)` (pixels only) | | a rectangle at fractional coordinates, edges antialiased |
| `label(x, y, text, ink)` | grapheme text, clipped on both sides | the built-in 5x7 ASCII font; `label_scaled` for integer sizes |
Coordinates are `i32` and may lie outside the surface; sizes may exceed
it. Circles take `f32` centres and radii in continuous coordinates (the
unit at `(x, y)` spans `x..x + 1`), so a moving ball sits between pixels
and antialiases there; cells round to the nearest cell. Everything is clipped, work is bounded by the visible part, and only
what is drawn is marked dirty.
The 5x7 font `label` draws on pixels is public as `standard_plugin::font`:
`font::glyph(c)` is a character's seven rows of five bits (top row first,
the leftmost column in bit 4), `None` outside printable ASCII;
`font::lit(&rows, x, y)` tests one pixel; `GLYPH_WIDTH`, `GLYPH_HEIGHT`
and `GLYPH_ADVANCE` size it. Draw the same letters in cells, as a mask, or
at any scale with them.
## Resize
When the viewer resizes a surface it sends `Event::Resize`. Before the
plugin sees the event the SDK has already asked for the new layout,
allocated a new zeroed region and attached it. The next commit is the
first at the new size and is sent fully dirty; the old region stays
allocated until that commit lands, because the viewer keeps showing it
until then. Repaint everything after a resize (the new buffer is empty):
the examples reset their "what I last painted" state in `event`.
## Cadence
`UiPlugin::frame(&mut self, frame: &Frame)` is called only when the viewer
paints a paced frame *and* one of the plugin's surfaces is visible *and*
the plugin asked for a frame. What the plugin asks for decides its cost:
| `frame.request_frame()` | Called again on the next paced frame: an animation at the viewer's rate |
| `frame.wake_at(ms)` / `wake_after(ms)` | Called at that instant on the viewer's clock: a clock asks for the next second |
| nothing | Not called again until the surface becomes visible again |
A commit from `event` is not a frame request: it is sampled on the next
frame something else causes. A plugin that repaints in `event` (a value
changed, a plugin event, a key) asks for a frame from there with
`cx.request_frame()` (or `cx.wake_at(ms)`); a card that changes only on
events never asks otherwise
([`together_ui.rs`](../examples/together_ui.rs)).
The viewer asks for a frame itself when a surface becomes visible.
A surface that shows nothing is never visible, so it never gets
`frame()`. A `machine.after`, `project.after` or `project.before`
instance at zero rows (`"height": 0`) stays hidden until the plugin asks
for rows. Ask from `activate` and from the events that change what the
row shows, with `Instances::instance(owner).request_size(0, rows)`; asking
only in `frame()` means no row ever shows
([agent guide](agent-guide.md#1-a-collapsed-anchor-never-gets-frame)).
`frame.elapsed()` is the time since the previous frame, and zero on the
first frame after every surface of the plugin was hidden.
`now_ms` is the viewer's monotonic clock, for animation and wakes. For
dates and countdowns, `view::wall_ms()` is the wall clock (milliseconds since
the Unix epoch) where the viewer runs, `view::utc_offset_minutes()` its local
offset right now (read it when formatting; it follows daylight saving),
`view::time_zone()` its IANA name when known, and `view::local_ms()` the two
combined (`local_ms() / 86_400_000` is the local day). A browser reports its
own clock and zone.
Nothing else wakes an idle viewer: a hidden surface costs nothing, and a
clock card costs one wake a second. `frame.power()` is `SavePower` when the
machine is saving power; slow animations down then (the shapes example
drops to one frame a second). `frame()` has half the frame interval of CPU
time (wall time in a browser). A plugin with a visible pixels surface that
keeps overrunning is degraded first: its frame rate is halved at each
overrun down to 15 fps, then its pixel resolution is halved (the viewer
scales the image up; `capabilities::get().pixel_scale` is 2 and the cell
pixel size halves). Every step arrives as `Event::CapabilitiesChanged`, and
the resolution step as a resize. Only then do further overruns restart and
finally pause it (three in total, restarts included, in every viewer).
A cells-only plugin skips straight to that.
## Two models
A surface whose manifest lists both models (`"model": ["pixels", "cells"]`)
is painted in the first one the viewer can show: pixels where it has real
graphics, else cells. The viewer may switch while the plugin runs (the
terminal changes); the plugin sees a resize. `AnySurface::new(id)` follows
it and hands out the surface in the current model:
```rust,ignore
match self.court.get() {
AnySurfaceMut::Pixels(pixels) => { /* draw pixels */ }
AnySurfaceMut::Cells(cells) => { /* draw cells */ }
}
```