sandbox-quant 1.0.4

Exchange-truth trading core for Binance Spot and Futures
Documentation

sandbox-quant

docs.rs crates.io

Exchange-truth trading core for Binance Spot and Futures, with separate operator, recorder, collector, backtest, and optional GUI entrypoints.

sandbox-quant shell startup

The current codebase is a reset v1 architecture focused on:

  • authoritative account, position, and open-order sync
  • typed execution commands
  • safe close primitives
  • Binance adapter with signed HTTP transport
  • foreground market-data recorder
  • dataset-backed backtest terminal
  • UI-independent core testing

The old strategy-heavy terminal dashboard described in earlier revisions is no longer the active implementation.

Current Scope

Implemented today:

  • refresh of authoritative exchange state
  • close-all
  • close-symbol <instrument>
  • set-target-exposure <instrument> <target>
  • strategy watch start/list/show/stop in the operator terminal
  • separate sandbox-quant-recorder terminal for market data collection
  • separate sandbox-quant-collector binary for historical Binance public-data imports
  • separate sandbox-quant-backtest terminal for dataset inspection/backtest runs
  • optional sandbox-quant-gui desktop app for charting and backtest exploration
  • Binance signed REST transport
  • runtime event logging
  • CLI summaries for refresh and execution results
  • automatic dataset schema bootstrap/version surfacing for recorder/collector flows

Not implemented as first-class runtime features yet:

  • automated strategy execution engine
  • liquidation trigger evaluator for live trading
  • full historical replay engine beyond dataset summary
  • detached recorder supervision model

Legacy strategy/UI documents are archived under docs/archive/legacy.

Architecture

Top-level modules:

  • src/app
  • src/backtest_app
  • src/charting
  • src/command
  • src/dataset
  • src/domain
  • src/error
  • src/exchange
  • src/execution
  • src/gui
  • src/market_data
  • src/portfolio
  • src/record
  • src/recorder_app
  • src/storage
  • src/terminal
  • src/ui
  • src/visualization

Core rules:

  • exchange state is the source of truth
  • local storage is not authoritative trading state
  • canonical position representation is signed_qty
  • execution is command-driven
  • recorder terminal owns live market-data ingestion in-process
  • strategy logic may be shared between operator and backtest, but it must not depend directly on DuckDB
  • tests live in tests/

The design rationale is documented in 0056-v1-reset-exchange-truth-architecture.md. Recorder ownership is documented in 0058-recorder-foreground-terminal-semantics.md and 0059-recorder-single-owner-runtime.md.

GUI/charting implementation notes and current limitations are tracked in docs/gui-charting-status.md.

Storage / Dependency Notes

This project currently uses both embedded and external storage dependencies:

  • duckdb / rusqlite for local dataset and recorder-facing storage workflows
  • postgres for collector storage targets and PostgreSQL-backed market-data import / summary flows

In other words, PostgreSQL is not just an optional environment detail; it is a real crate dependency in Cargo.toml and is used by the collector/storage path when --storage postgres is selected.

Recent Hardening Notes

Recent follow-up work focused on making the current GUI/backtest path safer and clearer to operate:

  • GUI market charts now avoid the high-resolution timestamp panic that previously appeared on some BTCUSDT ranges.
  • GUI chart time labels are rendered through an overflow-safe footer-label path, with adaptive width-based label density.
  • GUI hover/tooltip behavior was polished with safer placement near chart edges, plus better hover snapping to visible points.
  • GUI controls now expose clearer empty/error states, plus reset-zoom affordances.
  • Backtest CLI now rejects reversed date ranges before any DB initialization work begins.
  • Backtest output now distinguishes state=ok, state=no_trades, state=empty_dataset, and state=missing.
  • Collector/recorder summary surfaces now expose schema_version metadata so schema bootstrap state is visible to operators.

Known current caveats:

  • GUI footer time labels are adaptive and panic-safe, but they are still not full plotters-native mesh ticks.
  • Tooltip sizing/placement is still heuristic.
  • Backtest UX is being refined further around symbol_not_found vs generic empty_dataset messaging.
  • Full strict clippy across the repo still reports pre-existing too_many_arguments warnings outside the focused GUI/backtest hardening scope.

Environment

Required:

BINANCE_DEMO_API_KEY=your_demo_key
BINANCE_DEMO_SECRET_KEY=your_demo_secret
BINANCE_REAL_API_KEY=your_real_key
BINANCE_REAL_SECRET_KEY=your_real_secret

Optional:

BINANCE_API_KEY=legacy_shared_key
BINANCE_SECRET_KEY=legacy_shared_secret
BINANCE_SPOT_BASE_URL=https://api.binance.com
BINANCE_FUTURES_BASE_URL=https://fapi.binance.com
BINANCE_OPTIONS_BASE_URL=https://eapi.binance.com
SANDBOX_QUANT_RECORDER_STORAGE=duckdb
SANDBOX_QUANT_POSTGRES_URL=postgres://localhost/sandbox_quant

The runtime reads demo and real credentials separately based on BINANCE_MODE and when using /mode real|demo. The legacy shared key names are still accepted as a fallback. The default runtime mode is demo. Optional base URLs are useful for explicit testnet or custom routing.

Storage-specific env vars:

  • SANDBOX_QUANT_RECORDER_STORAGE=duckdb|postgres selects the live recorder sink
  • SANDBOX_QUANT_POSTGRES_URL (or DATABASE_URL) is used by PostgreSQL-backed recorder/collector flows
  • SANDBOX_QUANT_BACKTEST_AUTO_SNAPSHOT=postgres makes backtest run pull the requested symbol/date range from PostgreSQL into DuckDB before executing
  • SANDBOX_QUANT_BACKTEST_SNAPSHOT_PRODUCT / SANDBOX_QUANT_BACKTEST_SNAPSHOT_INTERVAL can narrow the imported snapshot

Binaries

  • sandbox-quant
    • operator terminal
    • manual execution and strategy watch management
  • sandbox-quant-recorder
    • foreground market-data recorder terminal
    • /start, /status, /stop, /mode
  • sandbox-quant-backtest
    • dataset consumer terminal
    • interactive shell plus run, list, report latest|show <run_id>
  • sandbox-quant-collector
    • one-shot historical Binance public data backfill
    • binance-public import, summary, snapshot postgres-to-duckdb
  • sandbox-quant-gui
    • optional desktop GUI for charting + backtest exploration
    • requires Cargo feature gui

The terminal binaries share the same line-oriented UX style, but lifecycle ownership is separate. The GUI is a separate optional launch path built on the same dataset/charting core.

Running

Operator shell:

cargo run --bin sandbox-quant

Refresh authoritative state:

cargo run --bin sandbox-quant -- refresh

Close all currently open positions:

cargo run --bin sandbox-quant -- close-all

Close one symbol:

cargo run --bin sandbox-quant -- close-symbol BTCUSDT

Set target exposure:

cargo run --bin sandbox-quant -- set-target-exposure BTCUSDT 0.25

target exposure must be in -1.0..=1.0.

Submit an options limit order:

cargo run --bin sandbox-quant -- option-order BTC-260327-200000-C buy 0.01 5

Options orders are handled as a separate workflow. They appear in portfolio positions and open orders, but they are not integrated into set-target-exposure.

Recorder terminal:

cargo run --bin sandbox-quant-recorder -- --mode demo

Recorder terminal with PostgreSQL sink:

export SANDBOX_QUANT_RECORDER_STORAGE=postgres
export SANDBOX_QUANT_POSTGRES_URL=postgres://localhost/sandbox_quant
cargo run --bin sandbox-quant-recorder -- --mode demo

Then inside the recorder terminal:

/start BTCUSDT
/status
/stop

Backtest terminal:

cargo run --bin sandbox-quant-backtest -- --mode demo

One-shot dataset run:

target/debug/sandbox-quant-backtest run liquidation-breakdown-short BTCUSDT --from 2026-03-13 --to 2026-03-13 --mode demo --base-dir var

If --from is after --to, the command now fails fast with an invalid date-range error before touching the dataset DB.

List recent runs:

target/debug/sandbox-quant-backtest list --mode demo --base-dir var

Show the latest persisted report:

target/debug/sandbox-quant-backtest report latest --mode demo --base-dir var

Backtest report/list output now uses explicit state markers so operators can tell the difference between:

  • a normal run with trades: state=ok
  • a valid dataset window with no executed trades: state=no_trades
  • no available dataset rows for the requested symbol/date range: state=empty_dataset
  • a missing persisted run id: state=missing

Historical public-data backfill:

cargo run --bin sandbox-quant-collector -- \
  binance-public import \
  --products um \
  --symbols BTCUSDT,ETHUSDT \
  --from 2026-03-12 \
  --to 2026-03-13 \
  --kline-interval 1m \
  --mode demo \
  --base-dir var

Dataset summary after import:

cargo run --bin sandbox-quant-collector -- summary --mode demo --base-dir var

PostgreSQL historical ingest (first migration slice):

export SANDBOX_QUANT_POSTGRES_URL=postgres://localhost/sandbox_quant
cargo run --bin sandbox-quant-collector -- \
  binance-public import \
  --products um \
  --symbols BTCUSDT,ETHUSDT \
  --from 2026-03-12 \
  --to 2026-03-13 \
  --kline-interval 15m \
  --storage postgres

PostgreSQL summary:

cargo run --bin sandbox-quant-collector -- summary --storage postgres

Backtest-ready DuckDB snapshot exported from PostgreSQL:

cargo run --bin sandbox-quant-collector -- \
  snapshot postgres-to-duckdb \
  --symbols BTCUSDT,ETHUSDT \
  --from 2026-03-12 \
  --to 2026-03-13 \
  --interval 15m

By default the PostgreSQL snapshot now exports any matching raw_klines, raw_liquidation_events, raw_book_ticker, and raw_agg_trades rows into DuckDB so existing backtest/GUI flows can keep reading DuckDB snapshots. Use --skip-book-tickers / --skip-agg-trades / --skip-liquidations to narrow the export.

Collector/recorder summary output now includes schema metadata such as schema_version so DB bootstrap state is visible without manual inspection.

GUI launch (optional feature):

cargo run --features gui --bin sandbox-quant-gui -- \
  --base-dir var \
  --mode demo \
  --symbol BTCUSDT \
  --from 2026-03-12 \
  --to 2026-03-13

src/bin/sandbox-quant-gui.rs currently accepts launch args directly and does not expose a dedicated --help screen; unsupported args fail fast.

Current limitation: the GUI market chart currently reads raw_book_ticker, raw_liquidation_events, and derived_kline_1s (from raw_agg_trades). Historical collector backfills stored only in raw_klines may appear in collector summary but still not render in the GUI until raw_klines support is added there.

The GUI uses the same DuckDB-backed dataset/backtest pipeline as the terminal tools:

  • load recorded symbols and summary metrics from var/
  • render book-ticker / 1s kline market charts with liquidation + trade markers
  • run a strategy backtest and inspect equity + trade tables in the same app session

Recent GUI usability improvements include:

  • clearer empty/error/status guidance when no data is loaded
  • reset zoom control and double-click viewport reset
  • safer hover snapping and tooltip placement
  • overflow-safe adaptive footer time labels on charts

Recorder data is stored by default under:

var/market-v2-demo.duckdb
var/market-v2-real.duckdb

demo and real here refer to account mode metadata. Public market-data streams currently use Binance public futures streams for both modes.

When recorder / collector / backtest tooling opens a dataset DB, the shared market-data schema is applied automatically. Summary/status surfaces now expose schema_version so older recorder-created DBs can be bootstrapped forward without manual table creation.

Recommended storage split for concurrency-sensitive workflows:

  • PostgreSQL for concurrent ingest / accumulation of raw market data
  • DuckDB for read-heavy snapshots used by backtesting and research
  • Collector currently supports PostgreSQL historical imports plus snapshot postgres-to-duckdb
  • Recorder still writes directly to DuckDB today; migrating live recorder ingest to PostgreSQL is the next architectural step if lock contention remains a problem

Output

refresh prints a summary like:

refresh completed
staleness=Fresh
balances=1
positions=2
open_order_groups=1
last_event=app.portfolio.refreshed

Execution commands print a summary like:

execution completed
command=close-all
batch_id=1
submitted=2
skipped=0
rejected=0
outcome=batch_completed

or:

execution completed
command=set-target-exposure
instrument=BTCUSDT
target=0.25
outcome=submitted

Testing

Library:

cargo test -q --lib

Current integration suite:

cargo test -q \
  --test core_types_tests \
  --test reconciliation_tests \
  --test binance_adapter_tests \
  --test app_runtime_tests \
  --test binance_http_transport_tests \
  --test bootstrap_tests \
  --test cli_command_tests \
  --test cli_output_tests

Release

Release automation is driven by GitHub Actions on main.

  • default bump: patch
  • merge commit with #minor: minor
  • merge commit with #major or BREAKING CHANGE: major

For the 1.0.0 release, the final merge into main should include #major.

Automation outputs:

  • bump Cargo.toml and Cargo.lock
  • create git tag vX.Y.Z
  • create GitHub release
  • publish to crates.io

Notes

  • set-target-exposure refreshes authoritative portfolio state before planning and can open from flat if the exchange symbol resolves.
  • execution and refresh flows are tested without any UI dependency.
  • README examples reflect the current runtime surface, not the removed legacy system.
  • recorder terminal live status is in-process worker truth, not a stale external status file.