densa 0.2.1

densa is a Rust toolkit for applications that show a lot of live data: trading terminals, monitoring consoles, operational dashboards, developer tools. It keeps live data in order as it changes, and draws it on the GPU at the display's refresh rate, from the data stream to the pixel.

Status: 0.x, experimental. On crates.io; APIs will change between releases (what changed). Builds and tests on Linux, macOS and Windows.

Architecture

• Data. Arrow record batches from files, streams or query engines go into live tables; views sort and filter them, groups aggregate them, and both follow every change incrementally.
• Interface. A tree of widgets, laid out with flexbox, takes pointer, keyboard and input method events; widgets (buttons, text inputs, scroll views, a data grid, charts) draw into a display list that is independent of any GPU API, and densa-window presents a whole frame in one draw call with wgpu.
• Runtime. Hand data between threads without locks, place threads on the right cores, format numbers without allocating, and prove that frames never allocate.

Each crate is usable on its own, and only densa-render and densa-window depend on a GPU or windowing crate.

Highlights

• Live tables. Upsert Arrow batches by key; sorted, filtered views and grouped aggregates are kept up to date as rows change instead of being recomputed.
• Charts over live data. Lines, candlesticks and volume in stacked panes that follow the newest data, zoom around the pointer and pan; a line is drawn one pixel column at a time, so a million points cost about as much as the chart is wide and every high and low shows.
• Widgets with flexbox layout. A retained widget tree laid out with flexbox, with pointer capture, hover, focus and Tab order, and widgets for toolbars, forms, lists, split panes and docking workspaces; it runs headless, so interfaces are tested without a window.
• Accessible. Widgets describe themselves to screen readers (through AccessKit on Windows, macOS and Linux) and do what they are asked: press, type, scroll, choose a tab. Docking tabs are reachable and usable from the keyboard.
• Arrow from end to end. Batches are shared, never copied, from the source to the screen; a grid over millions of rows costs only its visible rows.
• No allocation per frame. Drawing, scrolling, hit-testing, delivering events and moving a caret allocate nothing once fonts and glyphs are loaded; tests prove it with a counting allocator.
• No locks between threads. A wait-free triple buffer hands the newest data to the UI thread; its memory orderings are model-checked with loom.
• Real text. Shaping, wrapping, bidirectional text, selection, editing and input methods (IME), built on parley. Characters the bundled font lacks, such as Korean, Japanese and Chinese, come from the system's fonts.
• One draw call per frame, with a glyph atlas whose memory is bounded.

Quick start

You need Rust 1.90 or later and a GPU with Vulkan, Metal, DirectX 12 or OpenGL.

git clone https://github.com/densa-rs/densa
cd densa
cargo run --release -p densa-widgets --example monitor      # a live fleet monitor: grid and chart
cargo run --release -p densa-widgets --example gallery      # widgets in a window
cargo run --release -p densa-widgets --example dock         # a docking workspace, live grid (--layout FILE saves it)
cargo run --release -p densa-widgets --example live_table   # a live table in the grid widget
cargo run --release -p densa-widgets --example chart        # live candles, volume, pan and zoom
cargo run --release -p densa-widgets --example linked       # a grid and a chart over live tables
cargo run --release -p densa-arrow --example live            # a live table, sorted as it moves
cargo run --release -p densa-arrow --example table           # a million-row Arrow table
cargo run --release -p densa-paragraph --example editor      # editable text
cargo run --release -p trading-demo                          # a streaming market grid

To use densa, depend on the densa crate, which gathers the libraries as modules (densa::data, densa::widgets, densa::window, ...):

[dependencies]
densa = "0.2"

Or take only the crates you use, each on its own (densa-data needs no GPU or window):

[dependencies]
densa-data = "0.2"
densa-widgets = "0.2"
densa-window = "0.2"

Crates

Connectors that bring data in, with the large dependency trees their formats and protocols need, live in densa-connectors: today densa-files, which reads CSV and Parquet files as Arrow batches on a background thread, densa-ws, which reads WebSocket streams of JSON rows or Arrow IPC as Arrow batches and reconnects on its own, densa-flight, which reads Arrow Flight and Flight SQL results, and densa-datafusion, which runs SQL with Apache DataFusion over files, batches and snapshots of live tables (LiveTable::snapshot).

Usage

A live table, a sorted view, aggregates and a time window

use std::time::Duration;
use densa_data::{
    Aggregate, CompareOp, Filter, GroupSpec, LiveTable, Order, Scalar, ViewSpec, Window,
};

let mut table = LiveTable::new(schema, "symbol")?; // keyed by the `symbol` column
let movers = table.add_view(
    ViewSpec::new()
        .filter(Filter::compare("volume", CompareOp::Gt, Scalar::Int(1_000)))
        .sort_by("change_pct", Order::Descending),
)?;

table.upsert(ticks)?; // a RecordBatch: new symbols insert, known ones update
table.delete(&delisted)?; // an array of keys

let view = table.view(movers).unwrap(); // in order, after every change
let top = view.row(0); // where the row's values are in table.batches()

// Grouped aggregates follow every change the same way.
let by_sector = table.add_groups(
    GroupSpec::new()
        .group_by("sector")
        .aggregate(Aggregate::count())
        .aggregate(Aggregate::sum("volume").named("volume")),
)?;
let summary = table.groups(by_sector).unwrap().to_batch()?; // one row per sector

// Keep the last five minutes by the `time` column: older rows go as newer ones come.
table.set_window(Some(Window::duration("time", Duration::from_secs(300))))?;

A window ends at the newest time in the table, not at the clock, so replayed data is windowed by its own times. Window::span does the same for a column of plain numbers, in their units.

To show it in the grid widget, densa_arrow::LiveModel wraps the table: a click on a column title sorts the view, the app sets a filter, and the selection follows its row as values change (every row has a RecordId that lasts as long as its key). The live_table example does this with 100,000 instruments, a thousand of them updated every frame; densa_arrow::LiveGrid paints a view without widgets.

Widgets in a window

use densa_ui::{Action, ActionKind, Align, Style, Ui, WidgetId};
use densa_widgets::{Button, Label, Panel, TextInput};
use densa_window::{App, Host, WindowOptions};

#[derive(Default)]
struct Search {
    query: Option<WidgetId>,
    run: Option<WidgetId>,
}

impl App for Search {
    fn build(&mut self, ui: &mut Ui, _: &mut Host<'_>) -> Result<(), Box<dyn std::error::Error>> {
        let bar = Style::row().gap(6.0).padding(8.0).align_items(Align::Center);
        let bar = ui.add(ui.root(), Panel::new(), bar)?;
        ui.add(bar, Label::new("Symbol"), Style::row())?;
        let input = TextInput::new("").with_placeholder("AAPL");
        self.query = Some(ui.add(bar, input, Style::row())?);
        self.run = Some(ui.add(bar, Button::new("Run").primary(), Style::row())?);
        Ok(())
    }

    // Called with what widgets report: a click, an edit.
    fn action(&mut self, ui: &mut Ui, _: &mut Host<'_>, action: Action) {
        if action.kind == ActionKind::Clicked && Some(action.source) == self.run {
            let query = self.query.and_then(|id| ui.widget::<TextInput>(id));
            println!("search {}", query.map_or("", |input| input.text()));
        }
    }
}

densa_window::run(WindowOptions::new("Search"), Search::default())?;

A Ui also runs without a window: build it, feed it events and read its actions in tests.

A grid over your own data

use densa_fmt::write_fixed;
use densa_grid::{CellStyle, CellText, Color, GridSource};
use std::fmt::Write as _;

/// A monitoring table: one row per sensor.
struct Sensors<'a> {
    names: &'a [&'a str],
    celsius: &'a [f64],
}

impl GridSource for Sensors<'_> {
    fn rows(&self) -> usize {
        self.celsius.len()
    }

    // Called for every visible cell, every frame.
    fn cell(&self, row: usize, column: usize, text: &mut CellText) -> CellStyle {
        let value = self.celsius[row];
        if column == 0 {
            let _ = text.write_str(self.names[row]);
        } else {
            let _ = write_fixed(text, value, 1);
        }
        let hex = if value > 80.0 { 0xF87171 } else { 0xE2E8F0 };
        CellStyle { color: Color::srgb(hex), background: None }
    }
}

GridPainter::paint draws a GridSource onto a Canvas; GpuRenderer draws the result.

A chart over your own data

use densa_chart::{paint_line, points_within, y_extent, Axis, Color, LineStyle, Plot, Rect};

// A day of readings, one a second: (seconds since the epoch, value).
let readings: Vec<(f64, f64)> = (0..86_400)
    .map(|s| (1_792_368_000.0 + f64::from(s), (f64::from(s) / 3_600.0).sin()))
    .collect();

// The last hour, scaled to what it holds.
let x = Axis::time(1_792_450_800.0, 1_792_454_400.0);
let (low, high) = y_extent(&readings[..], points_within(&readings[..], &x)).unwrap();
let plot = Plot::new(Rect::new(0.0, 0.0, 800.0, 300.0), x, Axis::number(low, high).padded(0.05));

// One span per pixel column, however many readings fall in it.
paint_line(&mut canvas, &plot, &readings[..], LineStyle::new(Color::WHITE));

ChartPainter adds gridlines, tick labels and a crosshair; paint_candles and paint_bars draw candlesticks and volume. In a window, implement ChartModel (stacked panes on one time axis, each named above it if you give it a title) and give it to the Chart widget, which follows new data, zooms with the wheel and pans by dragging: see the chart example. For live data, densa_arrow::LiveSeries reads two numeric columns of a live table's view (numbers, timestamps, dates) as a series, so a chart keeps up with the same table a grid shows: see the linked example.

Text

use densa_paragraph::{Motion, ParagraphStyle, TextAlign, TextEditor, TextSystem};

let mut text = TextSystem::new()?; // fonts and layout scratch space, once per application

// A paragraph: shaped once, re-wrapped to any width without shaping again.
let style = ParagraphStyle::new(15.0).with_align(TextAlign::Justify).with_max_width(Some(480.0));
let mut paragraph = text.paragraph("Shaped once, wrapped to any width.", &style, &[])?;
paragraph.reflow(Some(320.0))?;

// Editable text: typing, deletion by character or word, input-method composition, undo.
let mut editor = TextEditor::new(&mut text, "Hello", ParagraphStyle::new(15.0))?;
editor.insert(&mut text, ", brave world")?;
editor.delete(&mut text, Motion::PreviousWord)?; // ctrl+Backspace
editor.undo(&mut text)?; // ctrl+Z: "world" is back

TextSelection turns clicks into caret positions and moves by character, word and line, in screen order for arrow keys and in text order for deletion, so right-to-left text behaves. See the paragraphs and editor examples.

Handing data to the UI thread

use densa_sync::triple_buffer;

let (mut writer, mut reader) = triple_buffer([0.0_f64; 64]);

// Producer thread: fill the back slot, then publish it. Never blocks.
std::thread::spawn(move || {
    writer.write(|rows| rows[0] = 101.25);
    writer.publish();
});

// UI thread, once per frame: switch to the newest snapshot, if any.
if reader.update() {
    reader.read(|rows| draw(rows));
}

Examples

Guarantees and how they are checked

Platform support

A window draws with the best API the system offers unless the application picks one (WindowOptions::with_backend); the WGPU_BACKEND environment variable (vulkan, dx12, metal, gl) overrides both. Graphics drivers differ in what each API costs, memory most of all, so measure on the machines you target.

Limitations

• Widgets: labels, buttons, text inputs (with undo, redo and the clipboard), scroll views, splitters, panels, docking, menus and a data grid, with flexbox layout, events, focus and accessibility. Not there yet: keyboard shortcuts handled for menus (a menu shows them; the application handles the keys), and text inputs' caret and selection for screen readers. The grid selects a row or a range of rows (kept on their records when the model gives rows keys) and copies them as tab-separated text; it does not yet select cells or scattered rows, paste or edit. On Linux the clipboard goes through X11 (XWayland under Wayland). Docking saves and restores layouts by panel key, but does not yet float panels in their own windows.
• Charts: lines, candlesticks and bars on number and time axes (UTC: no time zones yet), in stacked panes, fed from your data or a live table's view. Not there yet: logarithmic axes, area fills, annotations and drawing tools, several y axes in one pane, and candles built from a live table's trades (the app aggregates them). A line is one pixel column wide where it is steep, however thick it is set.
• Live data: views sort and filter, groups count, sum, average and find extremes; time windows and joins are not there yet. A table is used from one thread at a time.
• Text: grid cells are single-line, unshaped text in the bundled font. Word breaks in Chinese and Japanese and line breaks in Thai, Lao, Khmer and Burmese need densa-paragraph's complex-scripts feature (several MB of dictionaries). Not supported yet: choosing a language per paragraph (it picks the regional forms of Han characters), inline boxes, variable-font axes, synthetic bold and italic, color emoji.
• Threads: placement and priority are implemented on Windows only.

The roadmap lists what comes next.

Development

cargo test --workspace                       # includes GPU readback and a short window run
cargo clippy --workspace --all-targets -- -D warnings
RUSTFLAGS="--cfg loom" cargo test -p densa-sync --release --test loom_triple_buffer
cargo +nightly miri test --target x86_64-unknown-linux-gnu -p densa-sync -p densa-alloc -p densa-platform
cargo +1.90 check --workspace --all-targets  # minimum supported Rust version
cargo bench -p densa-bench

Set DENSA_SKIP_GPU_TESTS=1 on machines without a GPU or desktop session. Miri runs against the Linux target: with a custom global allocator, Miri reports a false positive in the Windows System allocator.

Bug reports and pull requests are welcome. CONTRIBUTING.md explains how the workspace is laid out, what the code must guarantee and every check a pull request has to pass.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

The bundled font crates/densa-text/fonts/JetBrainsMono-Regular.ttf (JetBrains Mono 2.304) is licensed separately under the SIL Open Font License 1.1; see crates/densa-text/fonts/OFL.txt.