exec_viz 0.3.0

Tick-by-tick replay of a trading_data computation DAG: a recording Observer, a scrubbable tape, and an axum server the browser front-end drives
Documentation

exec_viz

Minimum Supported Rust Version Lines Of Code

Tick-by-tick replay for a trading_data computation DAG — one computation structure, several interpretations of it.

Production evaluates the graph through step. Point exec_viz at the same tick chain and it runs under a recording Observer instead, so the browser can walk the run back: layers lighting up as they fire, element values and their local Jacobians drawn on the computation graph itself, next to a candle chart of the same day.

It is a library, not a runner. The app owns the graph, the feed and the runtime; it attaches a Recorder, hands it to its own tick_obs, and serves the Viz on a port of its choosing. Every handler reads whatever has been recorded so far, so the server runs alongside the recording — a live feed is scrubbable while it is still arriving.

cargo add exec_viz

The front-end is a separate wasm bundle, built once and pointed at by an env var:

export EXEC_VIZ_WEB_DIR="$(nix run github:EV-invest/exec_viz)"  # prints where the bundle landed

nix run . builds exec_viz_web with a pinned dioxus-cli/wasm-bindgen pair and prints the bundle path — nothing else. Whoever wants a UI points EXEC_VIZ_WEB_DIR at it and starts their own binary; this repo has no business knowing their package names.

Without the env var the server panics on boot rather than serving a guessed-at (and silently stale) bundle from somewhere inside its own checkout.

Usage

Attach a Recorder to the graph you already have, and serve its Viz. bind is separate from serve_on precisely so the URL answers — and prints — before the work it describes begins:

use exec_viz::{Backpressure, Viz};

// `price_node` names an OHLCV node, whose series is the candle pane; `capacity` bounds the
// retained ticks; `bucket_ms` is the chart's sample period.
let (viz, mut rec) = Viz::new(Some(<Bar1m as Cell>::NAME), 100_000, 60_000, Backpressure::Block);

let listener = Viz::bind(8080).await;
println!("http://{}", listener.local_addr()?);

tokio::join!(viz.clone().serve_on(listener), async {
    for (ts_ns, batches) in feed {
        graph.tick_obs(batches, &mut rec.at(ts_ns));
    }
    rec.seal(); // a finite recording says so; a live feed just drops the recorder
});

rec.at(ts) opens a tick and hands back the observer, so the graph's own sweep is the thing being recorded — there is no second evaluation path to keep in step with the first. Nothing else goes in: the candles are read off price_node's own [open, high, low, close, volume] recording, so a bar the graph already computes is not also held as an output to draw it. seal takes self, so the handle you recorded through is spent and total stops growing.

The recording is not on your thread. A finished tick crosses to a tape thread over a bounded channel, and that thread does the naming, bucketing, thinning and cost accounting; the graph pays a Display, a clipped Debug and one push, into buffers the tape hands back. Backpressure says what a full channel means: Block for a replay, which wants the whole tape and whose feed will wait, and Drop for a live run, where a fill must never queue behind a study aid — dropped ticks are counted into ActivationFrame::dropped rather than passing for a quiet market.

The tape is the storage. A live run cannot be re-run, so ticks are kept rather than replayed-by-rewinding. Past capacity, the buffer thins instead of dropping its front: the newest capacity / 2 ticks stay whole, and everything older is decimated by fire rather than by index — each node keeps one fire in 2^k, and k rises only for whichever node is claiming most of the backbone. So a run many times the capacity stays walkable end to end with its freshest stretch still tick-exact, and the rare event a human scrubs for — the 5-minute bar, the classification — survives the squeeze that the book flood absorbs. See Picking a capacity below for what to size it to. The per-node series the chart draws is downsampled online into bucket_ms buckets and never dropped.

The view

Two dockviewers tiles you can resize, tab and maximize — a lightweight-charts candle pane with a moving replay cursor, and the DAG activations panel. Clicking the chart seeks the tape to that bar; Alt+S caches the arrangement per screen band.

The DAG panel is DOM-inspector-style and present-moment only: one column per topo level, each node card rendering its element grid (shape from the node's dims), cells heat-colored by per-element running min/max. An SVG overlay draws element→element edges weighted by that tick's finite-difference Jacobian. A node that stayed quiet still shows its last value — fired is what says it is live this tick.

key
␣ step one tick
p play / pause
- = slower / faster (events per 50ms poll)
0 seek to the start
b skip to the next bar
c skip to the next Classify
n skip to the next change in the nodes you clicked

Two things the nav says about a thinned stretch. Where a step covers more than one absolute tick it shows a ×n next to the position — the resolution the tape is admitting to, rather than a stepper that looks like it is skipping. And a scan that walks a sealed recording to its end without a hit leaves the cursor where it was and reports that it found nothing, instead of answering a failed search by jumping to the end of the run.

An op can name a tick that has not been recorded yet. The server says so (pending) rather than blocking, and the control that outran the feed shows a ⟳ while it re-issues itself.

HTTP

GET /api/topology · GET /api/day · GET /api/status, and POST /api/{step,seek,seek_ts,step_until,step_until_change}. Shapes live in exec_viz::api_types, which both halves compile — the wasm client depends on exec_viz with default-features = false, leaving just the wire types and none of the axum/tokio server half.

Picking a capacity

capacity is the only number Tape::new asks for that has no right answer — it buys scrollback with RAM, and the exchange rate is your graph's, not this crate's. What it constrains:

capacity / 2 the newest ticks kept whole — the stretch that stays tick-exact
capacity / 4 the ceiling on the thinned backbone, and so how deep the squeeze goes
capacity / 2 the recycle channel's depth: one pass frees between a quarter and a half of the buffer, and the recorder takes all of it back rather than allocating

Thinning is per node, not per tick: a pass raises the exponent of whichever node is claiming most of the backbone until the backbone fits, so a node firing 288 times a day keeps all 288 while the book-driven flood beside it is decimated. What that means for sizing — a node's whole run stays addressable as long as its fires fit inside its max-min fair share of capacity / 4. Rare nodes are never the ones cut, so in practice the share is the whole backbone minus what the handful of greedy nodes settle at. In the block below, at capacity 20 000 the backbone is 5 000: the two nodes that each want ~3 000 land on ~1 790 apiece, and everything rarer than that keeps every fire it ever had. Read the other way: a 5-minute bar over three days is 864 fires and is safe from about 8 000; over a month it is 8 640 and wants ~65 000.

Three prices, and they are not the same shape:

  • Memory is linear, and it is the one you actually pay. Roughly 550 B per retained tick for the mix below; a 42-node graph runs nearer 2.4 KB.
  • Cursor latency is flat. Every replay op is a binary search, so a keypress costs the same against 260 000 retained ticks as against 2 000 — the whole latency column below is the loopback socket, and the tape's own share does not clear the noise.
  • Absorption grows, about 2× from 2 048 to 262 144. Not the graph thread — its leg of a fire is the same two renderings either way — but the tape thread's, which allocates fresh columns where a smaller buffer would have been handing recycled ones back. Under Block that is replay throughput; under Drop it is not latency either, it is dropped.

The flat middle line was not always true. A frame carries every node's standing value, and a node that has been quiet is found by searching back for its last fire — which used to be a linear scan. On a 42-node graph holding a day, one node that fires twice a day was enough to put a scrub at 139ms, growing with wherever the cursor sat. Inner::fired indexes it, and the mix below now carries a twice-a-day node so that number cannot quietly go back to being one nobody waits for.

The block is one synthetic graph on one machine. Viz::bytes is exported so the same reading can be taken against yours.

318420 ticks (2 days at spl's 159210/day), 6 nodes, 107 B/tick of card faces
fires over the run:  Book=318420  Screen=3185  Bar:1m=2895  Bar:5m=579  Bar:1h=49  Cap=4

footprint — Viz::bytes over the retained ticks
     2048 │▬ 1.2 MB  (701 B/tick)
     8192 │▬ 4.0 MB  (635 B/tick)
    20000 │▬▬▬▬ 9.7 MB  (594 B/tick)
    65536 │▬▬▬▬▬▬▬▬▬▬▬▬ 34.1 MB  (558 B/tick)
   262144 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 121.0 MB  (548 B/tick)

reactivity — one /api/seek mid-tape, median of 64; the bare loopback hop under it is 55µs
     2048 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 66µs
     8192 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 66µs
    20000 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 66µs
    65536 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 68µs
   262144 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 66µs

absorption — the whole recording's wall clock per tick, tape-thread bound under `Block`
     2048 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 615ns
     8192 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 714ns
    20000 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 889ns
    65536 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 1297ns
   262144 │▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 1477ns

addressability — fires still reachable by `step_until`, against the run's own
  capacity  retained          Book        Screen        Bar:1m        Bar:5m        Bar:1h           Cap
     2048      1767   1767/318420      248/3185      229/2895       147/579         49/49           4/4
     8192      6316   6316/318420      689/3185      897/2895       579/579         49/49           4/4
    20000     16395  16395/318420     1793/3185     1781/2895       579/579         49/49           4/4
    65536     61059  61059/318420     3185/3185     2895/2895       579/579         49/49           4/4
   262144    220713 220713/318420     3185/3185     2895/2895       579/579         49/49           4/4

Produced by cargo r --release -p exec_viz --example capacity, which splices it in here and writes it to docs/capacity.txt — the numbers are read, not typed.

License