Expand description
§tradefloor
The Rust core of tradefloor: a deterministic market simulator with a real limit order book.
A seed and a starting state run forward into prices, an order book with depth, fills, volume, macro state and a full order log. The same seed produces bit-identical output on Linux, macOS and Windows, because the crate ships its own transcendental maths rather than calling the platform’s libm.
§What it gives you
- Determinism across platforms. A cross-platform gate runs on every release and compares digests of a fixed simulation on five targets. A seed, a roster and a model preset name are a complete specification of a market.
- Emergent market impact. Orders match against a real book with price-time priority, so a large order pays worse prices because it consumed levels, not because a slippage coefficient said so.
- Ground truth. The simulator knows the fair value it computed and the macro regime it is in, so both are readable. No real dataset has labels.
- Named, frozen model presets. Coefficients ship as twenty-one presets,
pt-v1throughpt-v22with nopt-v17, all selectable and all bit-reproducing.pt-v22is the default (params::DEFAULT_PRESET_NAME). A modified coefficient set fingerprints ascustom-XXXXXXXXand can never present as a shipped one. - A published realism envelope. A one-year realism table of 19 statistics
graded against real-market bands (19 in band for
pt-v22), and a long-run check: 190 gated rows forpt-v22, all met, and the 40 registered rows ofpt-v21andpt-v20, all met by both, with the misses named as gaps rather than omitted. See https://docs.tradefloor.dev/how-its-measured.html.
§Using it
A trading day is three calls: open_market, run_session for the ticks,
then close_day, which settles the day and steps the economy. This runs
five days of a 20-name market, with 390 one-minute ticks a day as the
Python package does:
use tradefloor::economy::{create_initial_central_bank_state, create_initial_economy_state};
use tradefloor::engine::{Engine, SessionBuffer, SessionRequest};
use tradefloor::market::GameTime;
use tradefloor::universe::random_universe;
let seed = 42;
let companies = random_universe(20, seed)
.iter()
.enumerate()
.map(|(i, g)| g.to_init().to_tick_company(i))
.collect();
let mut engine = Engine::new(
seed,
companies,
create_initial_economy_state(&Default::default()),
create_initial_central_bank_state(0),
tradefloor::sectors::keys().iter().map(|s| s.to_string()).collect(),
);
let opening = engine.prices();
let mut buffer = SessionBuffer::new();
for day in 1..=5 {
engine.open_market();
let bell = GameTime::new(9, 30, 3);
engine.run_session(&SessionRequest::new(bell, 390), &mut buffer);
engine.close_day(day);
}
assert_ne!(engine.prices(), opening);close_day on its own does not trade, so a loop that skips run_session
leaves every price where it started. SessionRequest::new is a session with
no news and no orders; set its news, order_volumes or fills fields to
add them. SessionBuffer holds the last session’s prices, volumes and
attribution, one row per tick.
To pause a run and continue it later, possibly in another process,
engine.snapshot() captures the whole state and engine.restore(&snapshot)
puts it back onto an engine built the same way, and the restored run
continues bit for bit. The snapshot module documents the fields, the checks
a restore makes, and EngineSnapshot::to_bytes and from_bytes, which give a
snapshot one exact binary form to store.
Every preset is fitted with the engine making its own news and jumps, one
economy step a session and no economic shocks. A host that adds its own
news, shocks or earnings revisions can keep a tradefloor::flow::ExternalFlow
beside the engine, feed it what it passes in, and call assess with the
engine’s params() to learn which channels are outside the fitted flow.
tradefloor::flow::CalibratedFlow::of states that flow for any preset.
The Rust API is the engine itself and is low level: it takes tick requests and day advances and hands back state. Most users want the Python package, which wraps this crate and adds universes, scenarios, checkpoints, an Arrow bar reader, strategy evaluation and the realism panel:
pip install tradefloorDocumentation, including the realism envelope and what the simulator is not suitable for, is at https://docs.tradefloor.dev/.
§Driving the engine from a host
A host that owns the economy (a game, a trading desk simulator, anything that feeds the engine its own macro state and news) needs these things from this crate that the example above does not show. docs/EMBEDDING.md has the measurements behind each one, and how much news and how many shocks of your own the model can take.
Choose your opening. Engine::new and Engine::with_params run the
preset’s own opening over the economy you pass: on pt-v21 that is 755
days of the macro step before day 1, a draw of the business cycle’s phase,
and 504 sessions of market prehistory that set the volatility state. That
is right for the library’s default economy, which would otherwise open
every run in expansion at age zero, and wrong for yours: on pt-v20 an
economy passed in at a VIX of 45 in a contraction opens at 22.28 in an
expansion. Build
with Engine::with_params_keeping_opening to keep exactly what you pass,
and check engine.opening_settled() once after construction if you want a
refactor that swaps the constructor to fail loudly. A kept opening runs no
prehistory, so its volatility state starts at the constructor’s baseline
and its first quarter runs hot. A host whose new market starts on the
default economy should take the library’s opening and read the economy
back. The burn-in arrived with pt-v18 in 0.7.0; presets before it have
none.
Number your days and tick the session. Call
engine.set_current_day(day) with the trading day, counted from zero,
before each open_market. Nothing else advances the day, and pt-v21’s
earnings and dividend calendars and its buyback yield read it. Tick only
the 390 minutes from 09:30 and step the economy once a session, which
close_day does; ticks outside the session add noise the preset was not
fitted with, and weekend macro steps run the macro calendar fast. A host
that passes economic shocks closes with close_day_with_shocks, which
keeps the rest of close_day.
Keep splits out of the engine. The engine has no stock split.
set_fundamentals moves fair value one for one with the earnings it
writes, so dividing a name’s earnings by a split ratio without dividing
its price cuts its fair value by that ratio. Keep a split factor on your
side and send the engine earnings in its own units.
Size buffers from the width constants. A host that saves state into
flat arrays takes every width from the crate root: COMPONENT_COUNT (12
numbers in an attribution row), TICK_COMPONENT_COUNT (9 in a tick row),
RNG_STREAM_WIDTH (5 per random stream), ENGINE_RNG_STREAMS (10 streams)
and the others listed in the widths module. EngineRngState::to_words
and from_words pack and read the streams at those widths. A width derived from
something else, such as S_COMPONENT_KEYS.len() + 1, keeps compiling when
the engine gains a slot and under-sizes the buffer. A test pins every
value, and a change to one is a breaking change with its own CHANGELOG
line.
Measure a day change from prior_closes. On a preset without an
overnight move (pt-v20 and earlier) each stock’s previous_close is
reset at open_market to the day’s opening price, after the close
re-marks prices to newly published macro data, so it measures open to now.
On pt-v21 it is reset to the price the night started from, after that
re-mark and before the overnight move. Either way it is the anchor the
engine reads the day’s return from, and not the previous session’s last
print. For a
close-to-close change use engine.prior_closes(), the previous session’s
last print, and engine.last_closes() for the session just closed. On
pt-v20 the two anchors differ on every name every day, by a median of
0.12 per cent. Neither is part of the trajectory; a host that saves an
engine between days carries them with restore_closes.
Read the crisis line from the engine. The VIX level above which the
crisis behaviour runs is a preset coefficient, 30.88325108 from pt-v13 on
and 25.5 before. Read it with engine.crisis_vix_threshold() rather than
copying it, and engine.vix_above_crisis_threshold() applies the same
strict test the engine’s gates use. It is settable like any coefficient:
ModelParams::preset("pt-v20")?.with_override("crisis_vix_threshold", x).
The dollar’s safe-haven bid has its own threshold,
usd_crisis_vix_threshold, 25.5 on every preset.
§Versions and API stability
The crate takes the Python package’s version number and, from 0.10.0,
follows Cargo’s semver rules
for its public API. While the version is 0.x, a minor release (0.10 to
0.11) may break the API and a patch release (0.10.0 to 0.10.1) does not.
So tradefloor = "0.10" takes every patch release safely. The release
workflow runs cargo semver-checks against the newest published crate and
refuses to publish a patch release that breaks the API. Every break in a
minor release is listed in the CHANGELOG, and so is every change to a
state width or to what a constructor does with its arguments, which the
compiler cannot catch for you.
Releases before 0.10.0 did not follow this. 0.8.5 broke code written for
0.8.1 in a patch release; the CHANGELOG lists every change under “The Rust
crate since 0.8.1”, and tradefloor = "=0.8.1" stays on the old API.
From 0.10.0 every public struct with public fields is
#[non_exhaustive], so adding a field to one breaks no build, and so is
every public enum except Side, so adding a variant breaks none either. Outside this
crate you build one with its constructor or Default, then set or read
fields on the value. The next section lists the constructors.
§Upgrading to 0.10.0
The breaking changes in 0.10.0 are in how a host builds the crate’s structs
and matches on its enums. No existing function, field or variant changed,
and a struct built the new way holds what the old literal held. The structs
a host fills in, such as TickRequest, TickCompany,
EconomyState and RngState, are now #[non_exhaustive]. Outside the
crate a struct literal no longer compiles, ..Default::default() included,
and a pattern that destructures one needs a trailing ... Code that reads
or assigns fields is unchanged.
A tick request written as a literal on 0.9.1:
let request = TickRequest {
time: GameTime { hour: 9, minute: 30, day_of_week: 3 },
volatility_multiplier: 0.9,
news: &[],
news_impact_queue: &[],
order_volumes: &[],
};is built from its constructor on 0.10.0, which fills in a quiet tick, and the fields you need are set on the value:
let mut news = NewsEvent::default();
news.company_id = Some("ACME-0".to_string());
news.price_impact = Some(0.02);
let news = [news];
let mut request = TickRequest::new(GameTime::new(9, 30, 3));
request.volatility_multiplier = 0.9;
request.news = &news;A host-built company follows the same shape. TickStock::new(price, shares_outstanding) opens the stock at price with everything else empty,
and TickCompany::new(id, ticker, sector, stock) lists it:
let base = tradefloor::sectors::by_key("technology").unwrap().base_daily_variance();
let mut stock = TickStock::new(120.0, 5e7);
stock.avg_volume = 2e6;
stock.garch_variance = base;
stock.beta = Some(1.3);
let mut company = TickCompany::new("ACME-0", "ACME", "technology", stock);
company.eps = Some(6.0);The constructors for the structs a host builds:
| Struct | Build it with |
|---|---|
market::GameTime | GameTime::new(hour, minute, day_of_week) |
market::TickStock | TickStock::new(price, shares_outstanding) |
market::TickCompany | TickCompany::new(id, ticker, sector, stock) |
market::OrderVolume | OrderVolume::new(buy, sell) or Default |
market::NewsEvent, market::NewsImpactEntry | Default |
engine::TickRequest | TickRequest::new(time) |
engine::DayCloseRequest | DayCloseRequest::new(daily_innovations, sector_base_variances) |
engine::DayAdvanceRequest | DayAdvanceRequest::new(game_day, timestamp) |
economy::EconomyState | create_initial_economy_state(&options) or Default |
economy::InitialEconomyOptions | Default |
economy::CentralBankState | CentralBankState::new(start_timestamp) or Default |
economy::EconomicShock | EconomicShock::new(kind, severity, gdp_impact) |
rng::RngState, engine::EngineRngState | from_words, from what to_words wrote |
universe::InstrumentInit | InstrumentInit::new(ticker, sector, initial_price, shares_outstanding) |
agent_book::AgentOrder | AgentOrder::new(id, agent, ticker, side, limit, quantity, mode) |
agent_book::BookState, engine::GdpPublication | Default |
The inputs to the model’s component functions, such as
market::TickInputs, economy::DailyInputs, market::CloseInputs and
microstructure::CompanyMicrostructure, have a new or Default too, and
each one’s documentation says what it fills in. Structs the engine only
hands back, such as engine::DayAdvanceOutcome and agent_book::AgentFill,
have no constructor because a host never builds one.
Every public enum except order_book::Side is #[non_exhaustive] as well,
so a release can add a variant. Outside the crate a match on one needs a
wildcard arm. Side has two variants and keeps them, so a match on it stays
as it is. The enums are economy::CyclePhase, ForwardGuidance,
ShockKind and central_bank::Decision, market::MarketStatus,
AvgVolumePolicy and SettleDrawPolicy, engine::PriceField and
StopCondition, agent_book::RestMode and Liquidity, rates::CurvePoint,
rng::DrawKind and Site, and types::Difficulty.
This match on ShockKind, written for 0.9.1 with no wildcard arm, no longer
compiles:
fn name(kind: ShockKind) -> &'static str {
match kind {
ShockKind::OilShock => "oilShock",
ShockKind::Pandemic => "pandemic",
ShockKind::War => "war",
ShockKind::Other => "other",
}
}takes a wildcard arm on 0.10.0, which decides what a variant from a later release means to the host:
fn name(kind: ShockKind) -> Option<&'static str> {
match kind {
ShockKind::OilShock => Some("oilShock"),
ShockKind::Pandemic => Some("pandemic"),
ShockKind::War => Some("war"),
ShockKind::Other => Some("other"),
_ => None,
}
}The random streams no longer need their fields named. to_words writes a
stream as RNG_STREAM_WIDTH numbers and the engine’s ten as
ENGINE_RNG_STATE_WIDTH, and from_words reads them back:
let saved: Vec<f64> = engine.rng_state().to_words().to_vec();
assert_eq!(saved.len(), tradefloor::ENGINE_RNG_STATE_WIDTH);
engine.set_rng_state(EngineRngState::from_words(&saved)?);The words run in stream-id order, five per stream:
| words | stream | words | stream |
|---|---|---|---|
| 0 to 4 | market | 25 to 29 | news |
| 5 to 9 | economy | 30 to 34 | volume_idio |
| 10 to 14 | external | 35 to 39 | overnight |
| 15 to 19 | jumps | 40 to 44 | market_vol_level |
| 20 to 24 | volume | 45 to 49 | crisis_epicentre |
EngineRngState::STREAM_NAMES lists the same order. The struct declares
its volume_idio field before news, the reverse of the words, so a host
that packs the fields as declared puts those two streams in each other’s
slots, and from_words restores them swapped without an
error, because both are valid streams. A host that already saved states
that way swaps the two blocks once when it loads one. To stay clear of the
order, save the streams under their names with to_named_words and read
them back with from_named_words, which matches by name and refuses an
unknown, repeated or missing stream.
Each stream is the LCG state and increment as the raw bits of an f64,
the Box-Muller spare or NaN for none, and the two draw counts. The first
two words are bit patterns and some are NaNs, so move them as bytes:
JavaScript may rewrite a NaN’s bits and JSON has no NaN. A rewritten NaN
has an even increment word, and from_words refuses one.
§Scope of this crate
The published crate carries the engine, the unit tests in its source
modules and thirteen integration tests that run standalone:
circuit_breaker, depth_counterfactual, fed_put, host_constructors,
host_day_restore, maker_ladder_allocations, platform_maths, postcut,
roster_mutation, shares_outstanding, snapshot_restore,
stream_alignment and vix_target_input. The parity corpus that pins the
engine’s output is 140 MB of fixtures and stays in the repository, so the
tests that read it are left out of the package rather than shipped in a
state where they cannot pass.
In the repository, the suite.yml workflow runs cargo test --release
with the parity corpus, cargo clippy with warnings as errors, a build on
the declared minimum Rust (1.83) and cargo doc with warnings as errors.
Before it publishes, release.yml packages the crate and runs
cargo test --offline on the package.
§Licence
MIT or Apache-2.0, at your option. See LICENSE-MIT and LICENSE-APACHE.
§Where things are
engine::Engine owns a market and runs the day loop. It takes its
coefficients from a params::ModelParams, which is one of the named
presets (params::ModelParams::preset) or a preset with overrides, and
its roster from universe. The rest are the parts the engine is built
from, and each can be called on its own:
market: the tick, the factor model, GARCH and the trading clock.economy: the macro chain (GDP, inflation, unemployment, the VIX, the business cycle) and the central bank.order_book,agent_book,market_makerandmicrostructure: the limit order book and the depth around it.fair_valueandmispricing: what a stock is worth and how far its price has strayed from it.rates: bond indices priced off the engine’s yield curve.derivatives: what futures and options read: the engine’s forecast of its own state. The price index ismarket::IndexLevel.flow: the shock flow each preset was fitted at, and a tally a host keeps of the news, shocks and revisions it adds.rngandmathx: the random streams and the transcendental maths, both written here so that a seed gives the same bits on every platform.widths: how many numbers each state row and record holds, for a host that saves state into buffers. Re-exported at the root.
The python feature builds the Python extension module and the wasm
feature builds the WebAssembly binding. Both drive
engine::Engine::close_day, so a day in Python and a day in a browser
run the same code.
A program that runs its own day loop, such as a game, should read Embedding the engine in a host first: which constructor to open with, the day loop the presets were certified on, and how much news and how many shocks of its own a preset takes before its measured statistics stop describing the market.
Many modules began as line-for-line ports of an earlier reference implementation, and their comments still cite it. The crate has since changed the model on purpose and is now its only definition.
What it promises is determinism: the same seed, roster and preset give the same market, bit for bit, on every supported platform and in every patch release of an LTS line.
Re-exports§
pub use rng::stream;pub use rng::to_uint32;pub use rng::GameRng;pub use rng::Pcg32;pub use widths::COMPONENT_COUNT;pub use widths::ENGINE_RNG_STATE_WIDTH;pub use widths::ENGINE_RNG_STREAMS;pub use widths::MARKET_VARIANCE_STATE_WIDTH;pub use widths::NOISE_PART_COUNT;pub use widths::RNG_STREAM_WIDTH;pub use widths::SECTOR_COUNT;pub use widths::TAKEN_WIDTH;pub use widths::TICK_COMPONENT_COUNT;
Modules§
- agent_
book - The book an agent trades against: depth past the maker’s ladder, depth that agents consume and that refills, and agents’ own resting orders.
- derivatives
- Derivatives on the simulated market, and what they read.
- earnings
- The earnings calendar (
earnings_surprise_sigma): when each public company reports, and the draws that price its reports. - economy
- Economy — the price-relevant chain, ported from the reference implementation.
- engine
- The engine — WP5. Where the port becomes something you can run.
- fair_
value - Fair value — Layer 2 of the stationary price decomposition.
- flow
- The shock flow a preset was calibrated at, and a check on the flow a caller adds to it.
- market
- Market — the stationary price path, ported from the reference implementation’s market module plus its daily-lifecycle transitions.
- market_
maker - Market-maker quoting — ported from the reference implementation’s market-maker module.
- mathx
- The deterministic maths layer — Phase 1.
- microstructure
- Microstructure — ported from the reference implementation.
- mispricing
- Mispricing — the daily
s-process, ported from the reference implementation. - order_
book - Central limit order book — price-time priority matching.
- params
- The runtime parameter seam:
ModelParamsand the preset table. The runtime parameter seam —ModelParams, the settable half of the model preset. - population
- A population: background traders that share the book with the agents.
- rates
- Simulated constant-maturity bond indices priced off the engine’s curve. Rate instruments: simulated constant-maturity bond indices priced off the engine’s own yield curve.
- rng
- PCG32, ported from the reference implementation.
- sectors
- The twelve sectors and their model parameters. The sector table.
- snapshot
- An engine’s whole state, saved and put back.
- types
- Shared types, ported from
src/types/. - units
- The single place a factor of 100 exists - see the module docs. The one place a factor of 100 lives.
- universe
- Universe generation - plausible rosters, deterministically. Universe generation.
- wasm
- The browser surface: the same engine, compiled to WebAssembly.
- widths
- The widths a host sizes state buffers by, all re-exported below. The widths of the engine’s state, for a host that sizes buffers by them.