zc2 0.0.27

P2P compute broker with credit-based billing, WAL, and broker mesh support
# `zc` Interactive Terminal — Refactoring Architecture & Plan

> Goal: bring the `zc` interactive terminal up to the level of a Claude Code–style
> interface, focused on **rich REPL UX**, **slash commands & modes**, and
> **theming & rendering**. This document is the architecture proposal and a
> phased plan; it is deliberately delivered *before* code so the direction can be
> agreed first.

---

## 1. Where we are today

`zc` is a Rust binary (`name = "zc"`, crate `zc2` v0.0.16, edition 2021, MSRV
pinned to **Rust 1.75** via `ratatui = "0.26"` / `crossterm = "0.27"`). The
"interactive terminal" today is really **two things that are not a REPL**:

1. **A CLI dispatcher** in `src/main.rs` (~670 lines). Argument handling is a
   hand-rolled `match args.len() { 2 => …, 3 => …, 4 => … }` ladder with nested
   string matches. Help text is a wall of `println!` in `help()`. There is no
   single registry of commands — each command is wired inline, and the same
   command surface (`broker`, `up`/`down`, `workers`, `bench`, `me`, `attach`,
   `info`, container ops…) is repeated across arity arms.

2. **A read-only dashboard TUI** in `src/broker/tui.rs` (~1465 lines), built on
   `ratatui`. It renders a fixed dashboard (header, task pile, transactions,
   workers, metrics, sparkline, footer, help overlay). Input is limited to
   navigation keys (`q`, `Tab`, `j/k`, `?`, `r`). There is no text input, no
   command entry, no scrollback you can type into.

### Key problems to fix

- **No REPL.** You cannot type a command inside the terminal; you exit and
  re-run `zc …`. There is no input line, history, multiline, or streaming output
  area.
- **Massive duplication.** `TuiApp` (local) and `RemoteTuiApp` (remote/attach)
  are near-identical: `render_transactions`, `render_workers`, `render_metrics`,
  `render_help_overlay`, `render_footer`, the `Panel` enum, and both event loops
  are copy-pasted. ~600 lines are duplicated.
- **Hardcoded theming.** Colors live in a private `mod colors` with literal
  `Color::Rgb(...)` scattered through render code. No theme abstraction, no
  user config, no light/dark.
- **No command model.** Commands are strings matched in two places (CLI ladder +
  none in the TUI). Help, autocomplete, and dispatch cannot share a definition.
- **No rich rendering.** Output is `println!` + `colored` for CLI and ad-hoc
  `Span`s for TUI. No markdown, no code/JSON syntax highlighting, no spinners for
  long operations (`bench`, broker boot), no reusable panel primitives.

### What is actually good and worth keeping

- The background-fetch architecture in `run_remote_tui` (UI never blocks; a
  detached thread polls `/stats`, diffs snapshots into a stream log) is a solid
  pattern and should be generalized, not thrown away.
- `ratatui` + `crossterm` is the right stack; we build *on* it.
- The domain layer (`broker::stats::{StatsResponse, BrokerMetrics, WorkerStats,
  TransactionRecord, TaskOfferRecord}`) already gives us a clean data shape to
  render from.

---

## 2. Target experience (what "Claude Code level" means here)

Concretely, the interactive `zc` terminal should offer:

- **A persistent prompt** at the bottom: a real editable input line with a
  cursor, in-line hints, multiline support (`Shift+Enter`), and a blinking
  caret. Output streams into a scrollback region above it.
- **Slash commands** (`/workers`, `/attach node`, `/bench`, `/theme dark`, …)
  with a **`/`-triggered command palette**: fuzzy search, descriptions,
  argument hints, and `Tab`/`Enter` to complete.
- **Modes** surfaced in a status line: an *input* mode (typing a command), a
  *monitor* mode (the live dashboard, today's TUI, now reachable as a view), a
  *palette* overlay, a *search* overlay, and a *help* overlay. Mode is always
  visible, transitions are explicit, and `Esc` always backs out one level.
- **Rich rendering**: markdown for help and command output, syntax-highlighted
  code/JSON blocks, boxed panels, spinners and progress for long tasks, and a
  context/status bar (broker URL, auth state, mode, latency).
- **Theming**: a `Theme` type with named palettes loadable from
  `~/.config/zc/theme.toml`, hot-swappable via `/theme`.

Non-goals (per current scope): no agentic tool-use/diff-apply loop. The focus is
the *interface*: REPL UX, slash commands/modes, theming/rendering.

---

## 3. Proposed architecture

### 3.1 Module layout

Promote the terminal out of `broker::tui` into a dedicated top-level `tui`
module, and introduce a shared **command layer** used by both the non-interactive
CLI and the interactive REPL.

```
src/
  main.rs                 # thin: parse argv → cli::run() OR tui::run() (no args / `zc shell`)
  cli/
    mod.rs                # non-interactive entry; routes argv through command registry
    args.rs               # argv tokenizing + global flags (-d/-t/-v/-h)
  command/
    mod.rs                # Command trait, CommandRegistry, CommandResult, dispatch
    spec.rs               # CommandSpec: name, aliases, args, summary, help (markdown)
    builtins/             # one file per command, reused by CLI and REPL
      broker.rs up.rs workers.rs bench.rs me.rs attach.rs info.rs
      theme.rs help.rs clear.rs quit.rs ...
  tui/
    mod.rs                # pub run(opts) -> entry point; App wiring
    app.rs                # App state machine: Mode enum, event loop, key routing
    event.rs              # input → Action mapping; keymap table
    input.rs              # editable prompt (wraps tui-textarea), history, multiline
    palette.rs            # slash command palette: fuzzy match, completion, hints
    scrollback.rs         # output buffer: Blocks of rendered Lines, wrapping, scroll
    statusline.rs         # mode + context bar
    theme.rs              # Theme struct, named palettes, load/save TOML
    render/
      mod.rs
      markdown.rs         # markdown → ratatui Text (pulldown-cmark or tui-markdown)
      highlight.rs        # code/JSON syntax highlight (syntect, optional feature)
      panels.rs           # reusable boxed panels, tables, key/value grids
      spinner.rs          # spinner + progress widgets
    views/
      mod.rs
      monitor.rs          # the live dashboard (today's TUI), now a View
      snapshot.rs         # normalized data model the views render from
    data/
      mod.rs              # DataSource trait
      local.rs            # LocalSource: reads SharedBrokerState in-process
      remote.rs           # RemoteSource: background HTTP fetch thread (from run_remote_tui)
```

This keeps `broker::stats` etc. as the domain layer; `tui::views` only depends on
the normalized `Snapshot`.

### 3.2 Unify the two TUIs behind a `DataSource`

The single biggest structural win. Both `TuiApp` and `RemoteTuiApp` render the
same things from differently-sourced data. Introduce a normalized snapshot and a
trait; the rendering code is then written **once**.

```rust
// tui/views/snapshot.rs
pub struct Snapshot {
    pub header: HeaderInfo,                 // mode (Local/P2P/Remote), host:port, uptime, wireguard
    pub metrics: BrokerMetrics,             // reuse broker::stats::BrokerMetrics
    pub workers: Vec<WorkerRow>,            // normalized from WorkerStats / Worker
    pub transactions: Vec<TransactionRecord>,
    pub task_offers: Vec<TaskOfferRecord>,
    pub rps_history: Vec<f64>,
    pub events: Vec<String>,                // stream log (diffed offers/executes)
    pub connection: Connection,             // Connected | Connecting | Error(String)
}

// tui/data/mod.rs
pub trait DataSource: Send {
    /// Non-blocking: latest snapshot for rendering.
    fn snapshot(&self) -> Snapshot;
    /// Ask the source to refresh now (e.g. user pressed `r`).
    fn request_refresh(&self) {}
    /// Run a command that mutates/queries this source (used by slash commands).
    fn execute(&self, cmd: &ResolvedCommand) -> CommandResult;
}
```

`LocalSource` wraps `SharedBrokerState + StatsCollector` (today's `TuiApp::get_metrics`
logic). `RemoteSource` owns the detached fetch thread + `Arc<Mutex<…>>` shared
state and the offer/transaction diffing already in `run_remote_tui`. The
~600 duplicated lines collapse into one render path parameterized by
`Box<dyn DataSource>`.

### 3.3 One command model for CLI **and** REPL

```rust
// command/spec.rs
pub struct CommandSpec {
    pub name: &'static str,
    pub aliases: &'static [&'static str],   // e.g. me => ["whoami","credits"]
    pub args: &'static [ArgSpec],
    pub summary: &'static str,              // one line, for palette + `zc -h`
    pub help_md: &'static str,              // full markdown help, rendered in REPL
    pub category: Category,                 // Broker | Cluster | Benchmark | Identity | ...
}

// command/mod.rs
pub trait Command: Send + Sync {
    fn spec(&self) -> &CommandSpec;
    fn run(&self, ctx: &mut Ctx, args: &Args) -> CommandResult;
}

pub enum CommandResult {
    Output(RenderBlock),   // markdown/table/text streamed into scrollback (or printed by CLI)
    EnterView(ViewKind),   // e.g. switch REPL into monitor/attach view
    Quit,
    Error(String),
}
```

`Ctx` abstracts the sink: in CLI mode it writes to stdout via `colored`/markdown;
in REPL mode it pushes `RenderBlock`s into the scrollback. Each builtin
(`command/builtins/*.rs`) is written once and reused by both. `help()` and `zc -h`
become generated from the registry — no more hand-maintained `println!` wall, and
the help text can never drift from the real command set.

The giant `match args.len()` ladder in `main.rs` is replaced by:
`cli::run(argv)` → tokenize → `registry.resolve(tokens)` → `cmd.run(ctx, args)`.

### 3.4 The REPL app: modes & event loop

```rust
// tui/app.rs
pub enum Mode {
    Input,            // typing at the prompt (default)
    Palette,          // `/` command palette overlay open
    Search,           // `/`-in-scrollback or Ctrl+R history search
    Monitor,          // live dashboard view focused (keys drive the dashboard)
    Help,             // help overlay
}

pub struct App {
    mode: Mode,
    input: InputLine,          // tui-textarea-backed prompt + history
    scrollback: Scrollback,    // rendered output blocks
    palette: Palette,          // fuzzy command search
    registry: CommandRegistry,
    source: Box<dyn DataSource>,
    theme: Theme,
    status: StatusLine,
}
```

Event loop mirrors the proven pattern already in `run_tui`/`run_remote_tui`
(draw on a tick, `event::poll` with timeout, `KeyEventKind::Press`), but key
events are first mapped to an `Action` via a keymap (`tui/event.rs`) and then
dispatched per `Mode`. `Esc` pops one mode level; `Ctrl+C` quits. Long-running
commands run on a worker thread and stream `RenderBlock`s back through a channel
so the UI never blocks (generalizing the remote fetch thread).

### 3.5 Slash commands & palette

- Typing `/` at column 0 opens the **palette** overlay: a filtered list of
  `CommandSpec`s with summaries and arg hints. Fuzzy ranking via `nucleo` or
  `fuzzy-matcher`. `Tab` completes the name; `Enter` runs (or descends into arg
  entry); `Esc` closes.
- Inline autocomplete: as you type `/att`, show the best match ghosted after the
  cursor and a hint line (`/attach <zc://node>`) drawn from `ArgSpec`.
- Because the palette is backed by the same registry as the CLI, every command
  is automatically discoverable, documented, and runnable from inside the REPL.

### 3.6 Theming

```rust
// tui/theme.rs
pub struct Theme {
    pub bg: Color, pub fg: Color, pub accent: Color,
    pub success: Color, pub warning: Color, pub error: Color,
    pub muted: Color, pub border: Color,
    pub header_bg: Color, pub highlight: Color,
    pub row_alt: Color,
}
impl Theme {
    pub fn dark() -> Self { /* current mod colors values */ }
    pub fn light() -> Self { /* … */ }
    pub fn load(path: &Path) -> io::Result<Theme>;   // TOML; `toml` is already a dep
}
```

Every `Style::default().fg(colors::X)` site is replaced by `self.theme.x`. Themes
load from `~/.config/zc/theme.toml`, default to `dark()`, and hot-swap via
`/theme <name>`. This is a mechanical but pervasive change — best done early
(Phase 0) so all later rendering uses the theme from day one.

### 3.7 Rich rendering

- **Markdown**: render `help_md` and command output through `render/markdown.rs`
  (pulldown-cmark → `ratatui::text::Text`, or the `tui-markdown` crate) so help,
  errors, and results look consistent.
- **Syntax highlighting** (optional `--features highlight`): `syntect` for code
  and pretty-printed JSON (e.g. `/me`, `/info`, raw `/stats`). Gated behind a
  feature flag to protect MSRV and binary size.
- **Spinners/progress**: `render/spinner.rs` for broker boot, `bench`, and remote
  connect, replacing silent waits and `eprintln!` polling.
- **Panels**: `render/panels.rs` centralizes boxed blocks, tables, and key/value
  grids so `monitor.rs` and command output share primitives.

---

## 4. Backwards compatibility

- **All existing invocations keep working.** `zc broker`, `zc up --workers 4`,
  `zc workers zc://node`, `zc -t broker`, `zc attach …`, etc. still run
  non-interactively through the new registry. The arity ladder is replaced, not
  the behavior.
- **New default:** `zc` with no args currently prints help. Proposal: keep
  printing help by default, and add an explicit `zc shell` (alias `zc repl`) to
  launch the interactive terminal — opt-in, no surprise for scripts/CI. (Decision
  point — see §7.)
- **`zc -t broker`** continues to open the dashboard; internally it now opens the
  REPL in `Monitor` view with a `LocalSource`. `zc attach` opens it with a
  `RemoteSource`.
- **MSRV stays 1.75.** Every new crate must be checked against Rust 1.75 and
  ratatui 0.26 in Phase 0 before adoption.

---

## 5. Candidate crates (verify MSRV against Rust 1.75 in Phase 0)

| Need | Candidate | Notes |
|------|-----------|-------|
| Editable prompt | `tui-textarea` | Integrates as a ratatui widget; multiline, history-friendly. Confirm a 0.26-compatible version. |
| Fuzzy palette | `nucleo` *or* `fuzzy-matcher` | `fuzzy-matcher` (SkimMatcherV2) is lighter and very MSRV-friendly; `nucleo` is faster but newer. |
| Markdown | `tui-markdown` *or* `pulldown-cmark` | `pulldown-cmark` + a small renderer keeps deps minimal and MSRV-safe. |
| Syntax highlight | `syntect` | Heavy; gate behind `--features highlight`. |

Keep the dependency footprint conservative; prefer `pulldown-cmark` +
`fuzzy-matcher` for the MSRV-safe baseline, treat `nucleo`/`syntect`/`tui-markdown`
as optional upgrades validated individually.

---

## 6. Phased plan

Each phase is independently shippable and leaves `zc` working.

**Phase 0 — Scaffolding & theming (no behavior change).**
Create `tui/` and `command/` skeletons. Extract `theme.rs` from `mod colors` and
route all existing render sites through `Theme`. Verify candidate crates compile
on Rust 1.75 / ratatui 0.26. *Exit:* identical behavior, themed internals,
green build + clippy.

**Phase 1 — Unify the data layer.**
Add `Snapshot` + `DataSource`; implement `LocalSource` and `RemoteSource` from the
existing `TuiApp`/`RemoteTuiApp` logic. Collapse the two TUIs into one render path
in `tui/views/monitor.rs`. *Exit:* `-t broker` and `attach` look/behave the same;
~600 duplicated lines removed.

**Phase 2 — Command registry.**
Define `CommandSpec`/`Command`/registry; migrate each command into
`command/builtins/*`. Replace the `main.rs` arity ladder with `cli::run`, and
generate `help`/`-h` from the registry. *Exit:* every existing CLI command runs
through the registry; help is generated.

**Phase 3 — Interactive REPL shell.**
Build `App`, `InputLine`, `Scrollback`, status line, and the mode-aware event
loop. Wire `zc shell`. Commands typed by name run through the registry and stream
output into scrollback. *Exit:* a usable REPL with history and streaming output.

**Phase 4 — Slash commands & palette.**
Add `/`-triggered palette with fuzzy match, autocomplete, arg hints, and mode
indicators in the status line. `Esc` back-out semantics. *Exit:* discoverable,
completable slash commands backed by the registry.

**Phase 5 — Rich rendering & theming polish.**
Markdown rendering for help/output, optional syntax highlight for code/JSON,
spinners/progress for long ops, themes loadable from config + `/theme`. *Exit:*
the "Claude Code level" look and feel.

**Phase 6 — Verification & docs.**
Snapshot tests for views via `ratatui::backend::TestBackend` (+ `insta`); unit
tests for the registry, palette matching, and theme TOML round-trip; update
`README.md` and `docs/USAGE.md`. *Exit:* tested, documented, CI green.

---

## 7. Decisions I need from you before Phase 0

1. **Default launch behavior** — keep `zc` (no args) printing help and add
   `zc shell`/`zc repl` to launch the interactive terminal (safest for CI), or
   make bare `zc` launch the REPL? (Recommendation: explicit `zc shell`.)
2. **Dependency appetite** — MSRV-safe baseline (`pulldown-cmark` +
   `fuzzy-matcher`, no syntect) vs. richer stack (`nucleo` + `tui-markdown` +
   `syntect` behind a feature). (Recommendation: baseline now, feature-gate the
   rest.)
3. **Scope confirmation** — confirm tool-use/agentic loop stays out for now, so I
   keep the focus on REPL UX, slash commands/modes, and theming/rendering.

Answer these three and I'll start at Phase 0 (scaffolding + theme extraction),
which is pure internal refactor with no behavior change.