Expand description
§High-Performance Lock-Free Order Book Engine
A high-performance, thread-safe limit order book implementation written in Rust. This project provides a comprehensive order matching engine designed for low-latency trading systems, with a focus on concurrent access patterns and lock-free data structures.
§Key Features
-
Lock-Free Architecture: Built using atomics and lock-free data structures to minimize contention and maximize throughput in high-frequency trading scenarios.
-
Multiple Order Types: Support for various order types including standard limit orders, iceberg orders, post-only, fill-or-kill, immediate-or-cancel, good-till-date, trailing stop, pegged, market-to-limit, and reserve orders with custom replenishment logic.
-
Thread-Safe Price Levels: Each price level can be independently and concurrently modified by multiple threads without blocking.
-
Advanced Order Matching: Efficient matching algorithm for both market and limit orders, correctly handling complex order types and partial fills.
-
Performance Metrics: Built-in statistics tracking for benchmarking and monitoring system performance.
-
Memory Efficient: Designed to scale to millions of orders with minimal memory overhead.
§Design Goals
This order book engine is built with the following design principles:
- Correctness: Ensure that all operations maintain the integrity of the order book, even under high concurrency.
- Performance: Optimize for low latency and high throughput in both write-heavy and read-heavy workloads.
- Scalability: Support for millions of orders and thousands of price levels without degradation.
- Flexibility: Easily extendable to support additional order types and matching algorithms.
§Use Cases
- Trading Systems: Core component for building trading systems and exchanges
- Market Simulation: Tool for back-testing trading strategies with realistic market dynamics
- Research: Platform for studying market microstructure and order flow
- Educational: Reference implementation for understanding modern exchange architecture
§What’s New in Version 0.13.0
§v0.13.0 — the public API hands out no level handles (#228); exclusive submit gate under STP (#225); replay re-executes coded submit rejections (#224)
- Breaking (semver-minor under 0.x):
OrderBook::get_bidsandOrderBook::get_asksare removed (#228). Both cloned the book’s liveArc<PriceLevel>handles into aDashMap, andPriceLevelexposesadd_order,update_orderandmatch_orderpublicly, so a caller holding one could mutate a price level behind the submit gate, theorder_locations/ user-order indices, the risk state, self-trade prevention, the kill switch, the order-state tracker and the trade / book-change listeners. Deprecating them would have left the bypass reachable, so they are gone and 0.13.0 is the release boundary for breaking changes. Migrate to the read-only APIs, which return values rather than handles:create_snapshot(depth)for a full snapshot of every level and order;levels_with_cumulative_depth,levels_until_depth,levels_in_rangeandfind_levelforLevelInfoviews;order_count_at_price,get_orders_at_price,get_all_ordersandtotal_depth_at_levelsfor per-price and per-book order data;best_bid/best_askfor the top of book. Every level mutation now goes throughOrderBook. - Breaking (semver-minor under 0.x): the level iterators’
newconstructors are crate-private (#228).LevelsWithCumulativeDepth::new,LevelsUntilDepth::newandLevelsInRange::neweach take a reference to the book’s live price-level map, and withget_bids/get_asksgone no public API yields one. The iterator types stay public; obtain them fromOrderBook::levels_with_cumulative_depth,levels_until_depthandlevels_in_range. - Self-trade prevention holds under concurrent same-user admission
(#225). An STP-relevant submit decided a price level’s
STPActionfrom a queue snapshot and then filled that level in a second operation, both under the shared side of the submit gate — so a concurrent same-user admission could land between the two and be filled by the very sweep the scan was protecting. STP-relevant submits and the cancel-then-add modify variants whose re-add can match (UpdatePrice,UpdatePriceAndQuantity,Replace) now take the exclusive side, so the scan and the fill it authorises observe the same queue. Cost: on an STP book every identified submit except post-only, and every matching-capable re-price, is serialized.STPMode::Nonebooks, post-only submits,UpdateQuantityandCancelkeep the shared, fully concurrent path. With no level handles left to bypass it (#228), the gate now covers every mutation. SequencerResult::RejectedWithCode { reason, code, may_have_mutated, stp_mode }.add_orderemits real fills and then returnsErrfor an IOC’s unfillable remainder and for a taker STP cancels after non-self fills;ReplayEngineskipped every rejected event, so replay rebuilt liquidity the live book had consumed. Producers now opt in by recording the typed outcome —SequencerResult::from(&error)fills all four fields — and replay decides by the recorded code: a submit rejected under a code replay can reproduce from the book state andReplayBookConfigis re-executed and must fail the same way again, while codes whose trigger lives outside the config (kill switch, risk limits,Other) are skipped rather than re-executed, because a rejection that never touched the book is reproduced by doing nothing.last_applied_seq/ the applied count / the progress callback follow what was dispatched, so a re-executed rejection advances them.- The two facts the reject code cannot carry.
may_have_mutatedflags the errors the engine can return after changing the book, including the residual-admissionPriceLevelErrorthat maps toRejectReason::Other(0); a flagged submit is re-executed whatever its code says, so that rejection no longer replays as a no-op that resurrects consumed liquidity.stp_moderecords the mode that decided a self-trade-prevention rejection, and replay refuses a mismatchedReplayBookConfig. ReplayError::OutcomeMismatch { sequence_num, recorded, actual }aborts replay when a re-executed rejection succeeds or fails under a different code than the journal recorded;ReplayError::StpModeMismatch { sequence_num, recorded, actual }aborts it when a journaled STP rejection was decided under a differentSTPModethan the replay book uses.- Migration. The string-only
SequencerResult::Rejectedkeeps its historical skip, so a journal written with it keeps the pre-existing gap for traded-then-rejected submits; switch producers toRejectedWithCode. Journals carrying the new variant cannot be decoded by older readers (existing journals decode unchanged, as forMarketOrderByAmount). Limitations: only the reject code is reconciled, never the error’s details or the fills behind it, so a discrepancy confined to them can go undetected —snapshots_match(directly, or viaReplayEngine::verify) is the check that catches a diverged book, andreplay_fromperforms none. Thestp_modeguard only fires on journals that recorded an STP rejection. Breaking (semver-minor under 0.x):ReplayErrorgained two variants, so exhaustive matches need new arms; 0.13.0 is the release boundary for them together with the #228 removal. No snapshot format change. - Reserve orders are lot-size validated per tranche and on their
replenishment transfer (#226). A
ReserveOrderused to be checked on its total only, so a 15 visible / 5 hidden reserve was admitted to a lot-10 book while the identical iceberg was rejected. It now takes the iceberg’s per-tranche rule and, additionally, validates the capped quantity replenishment transfers from hidden into the visible tranche —min(replenish_amount.unwrap_or(DEFAULT_RESERVE_REPLENISH_AMOUNT), hidden), checked whilehidden > 0andauto_replenishis on. Admission is strictly tighter: a shape previously admitted on its total is now rejected withInvalidLotSize, carrying the offending tranche or transfer. - A reserve residual follows
auto_replenish(#230). The residual-resting helper behindOrderQuantity::set_total_remainingrefreshed an emptied visible tranche fromreplenish_amountalone, ignoringauto_replenish, falling back to a refresh of zero (which could rest a zero-visible order) and never consultingreplenish_threshold. It now appliespricelevel’s rule: with automatic replenishment on and hidden left, a visible tranche belowmax(replenish_threshold, 1)grows by the explicit amount orDEFAULT_RESERVE_REPLENISH_AMOUNT, capped by hidden; with it off the residual does not rest at all and its hidden remainder is discarded, mirroring the removal of a depleted non-auto maker. A 10 visible / 20 hidden reserve withreplenish_amount = Some(10)and no automatic replenishment, filled for 10, used to rest 10 / 10 and now ends asFilled { filled_quantity: 10 }; the same order with automatic replenishment and a threshold of 5, filled for 8, used to rest 2 / 20 and now rests 12 / 10. The discard needs the visible tranche to be exhausted: with automatic replenishment off, a 10 / 20 reserve filled for 5 still rests 5 / 20. An explicitreplenish_amountis the transfer, added to whatever visible quantity survived, not a target display size. The accounting rule issubmitted = executed + resting (visible + hidden) + discarded, and discarded quantity is never counted as executed. A discard emits anINFOtrace and, under themetricsfeature, the neworderbook_reserve_discards_total/orderbook_reserve_hidden_discarded_totalcounters, carrying apathfield so the aggressive taker and the removed maker report the same discard the same way; the returned order handle carries both tranches at zero. Because the three cancel-then-add modify variants re-add the order as a taker, a validate-first pre-check now rejects a re-price that would exhaust such a reserve’s visible tranche with the newOrderBookError::ReserveResidualWouldBeDiscardedbefore the original is cancelled, so a re-price of such a reserve cannot destroy the order it modifies: the exclusive guard covers the lookup, the validation, the cancel and the re-add, so the dry run is exact. That is the scope of the guarantee; it is not a claim about every possible modification failure. Crossing into depth smaller than the visible tranche, a non-crossing re-price and a projected full fill are all allowed through. The error carries both the projectedhidden_quantityand thediscarded_quantitythat would actually be destroyed.RejectReasongains the matching wire code 14; both enums are#[non_exhaustive]. In a book that holds such a reserve, every sweep now takes the exclusive submit gate in everySTPMode— matching-capable submits, cancel-then-add re-prices and the match-only entry points alike, plus the admission of the first one — so nothing can cancel, admit or replace an order inside a sweep’s capture window: the sweep cannot consume a maker it never captured, nor report a captured maker after a cancel freed its id. Cancels and mass cancels keep the shared side. Those books serialize their sweeps; books holding none are unchanged. - A non-replenishing reserve must display a positive visible tranche
(#230). A
ReserveOrderwithauto_replenish == false,visible_quantity == 0andhidden_quantity > 0used to rest as a ghost: no visible depth, andpricelevelremoves it without a trade, stranding the whole hidden tranche, on the first taker to reach the level. Since #221 a zero quantity onUpdatePriceAndQuantity/Replacecould drive a healthy resting reserve into that shape too;UpdateQuantitywith a zero quantity cannot, because it is a removal taken before the validator runs (#223, below).validate_order_shapenow rejects it with the newOrderBookError::ZeroVisibleTranche, coveringadd_order, every modify projection and snapshot restore; a rejected modify leaves the original resting and a rejected restore leaves the book untouched. The rule is that shape only: a zero-visible iceberg draws its whole hidden tranche into visible on match, and a zero-visible auto-replenishing reserve refreshes and re-queues, so both execute and stay admissible. Single-tranche kinds are unaffected. Maps to the existingRejectReason::InvalidQuantity. OrderUpdate::UpdateQuantitywith a zero quantity cancels the order (#223). A zeronew_quantitywas accepted and applied as a resize:pricelevelkeeps a non-growing total in place, so the maker rested at zero depth, heldbest_bid/best_askon a level with nothing behind it and was later dropped by a sweep with no trade and no cancel event, leaking itsorder_locationsentry —cancel_orderthen returnedOk(None)while re-adding the id reportedDuplicateOrderId. The arm now routes to the sameUserRequestedcancelOrderBook::cancel_orderperforms, so the level-change event, theCancelled { UserRequested }transition, the per-account risk release, the location / user-index untrack and the empty-level removal happen in lockstep. A zero requested quantity is a removal, not a resize: it cancels the entire order, hidden depth of an iceberg or reserve included (a nonzeronew_quantitystill resizes only the visible tranche), and it runs neither the projected-order validator nor the modify-aware risk check, so neither a configuredmin_order_sizenor a risk limit vetoes it. The kill switch still refuses it, as it refuses every modify. The removal semantic isUpdateQuantity’s alone: a zero quantity onReplace/UpdatePriceAndQuantityre-adds through validate-first (#230 above). Compatibility: a journal recorded before this change that contains a zeroUpdateQuantityreplays to the new outcome, so the replayed book legitimately differs from the one the original run produced.- Reserve
UpdatePriceAndQuantityhonours the requested visible quantity (#221).OrderQuantity::set_quantityread a reserve’s argument as a total target and only ever reduced, so a requested increase was silently dropped (a 30 / 70 reserve asked to move to 80 ended at 10 / 70) and a decrease was drawn across both tranches. It now sets the visible tranche and leaves hidden untouched for both two-tranche kinds, matchingUpdateQuantity,Replaceand the upstreampricelevelcontract. CancelTakerandCancelBothfire only on a same-user maker the taker can reach (#222). Both arms used to cancel unconditionally once a same-user maker rested at a crossed level, even when the non-self depth queued ahead of it already satisfied the taker. A client sawSelfTradePreventedon an order that had in fact filled, andCancelBothdestroyed a maker the sweep never touched — silently on the market paths, which drop the taker-cancelled flag and returnOk. The arms now execute against the non-self depth first and cancel only if the taker could still execute at that price afterwards.STPMode::CancelMakeris deliberately unchanged: it still cancels every same-user order at a level the sweep touches, since it never destroys the taker. The modify pre-checkcheck_modify_stp_self_crossfollows the same per-level rule and sizes the pre-match withpricelevel’s authoritative dry run rather than the counted visible depth, so it cannot admit a re-price the sweep would then kill after the original was cancelled.- A quote-notional sell walks past a bid it cannot afford (#222). A zero per-level quantity cap ended the whole sweep. That is right for a base-quantity budget, whose cap ignores the level price, and for a notional buy, which walks asks ascending so the cap only shrinks. It was wrong for a notional sell, which walks bids descending: a budget too small at one bid can fund a whole lot at a cheaper one. Selling 150 into bids of 100, 75 and 50 executed one unit instead of two. The sell walk now skips the unaffordable level and stops only on a spent budget, an exhausted side, or a remainder below one lot — the point at which no price could fund a lot. It may therefore visit every level on the bid side; each skipped level costs one division and mutates nothing.
§What’s New in Version 0.12.0
§v0.12.0 — pricelevel 0.9 hardening bump; upsize demotion survives snapshot restore (#205)
pricelevel0.8.4 → 0.9.1. Major upstream hardening release: level admission validates before mutating (duplicate id, counter capacity, price/side topology), PostOnly / fill-or-kill decisions are atomic with the sweep, execution statistics are torn-read-safe, and level snapshots materialize orders in queue-consumption order. 0.9.1 fixes theMatchResultbincode round-trip (PriceLevel#135), keeping thebincodefeature’s trade-event round-trip intact.- The upsize queue-priority demotion now survives a snapshot
round-trip (#205). Restoring a snapshot rebuilds each level’s queue
exactly as matching would consume it, so an order demoted by a quantity
increase keeps its back-of-queue position after
restore_from_snapshot_package. Locked in by a proptest regression (tests/unit/props_quantity_update_priority.rs). Snapshots captured with pricelevel < 0.9 restore demoted orders at their old(timestamp, seq)position — re-snapshot to pin the corrected order. - Breaking (semver-minor under 0.x):
get_bt_bids/get_bt_asksnow returnResult<BTreeMap<u128, PriceLevel>, OrderBookError>(snapshot-to-level conversion is validating and fallible upstream), and the re-exported pricelevel surface changed —PriceLevel::add_orderreturnsResult,matchable_quantitytakes the taker id,PriceLevelErrorgainedDuplicateOrderId. - Atomic PostOnly / multi-level FOK (#209). PostOnly submits thread
TakerKind::PostOnlyinto every per-level match, making it structurally impossible for a post-only order to take liquidity under any interleaving; fill-or-kill submits hold a new book-level submit gate exclusively across feasibility + sweep, so multi-level all-or-nothing can no longer partially execute against concurrent cancels. Other mutating entry points take the gate’s uncontended read side; the matching core stays lock-free. Full 0.11.0 → 0.12.0 HDR tail-latency comparison inBENCH.md: every scenario’s median is unchanged by this release’s book-level work; the one median shift (stp_sweep, from the pricelevel 0.9 hardening) is documented there with its bisection. - Atomic, observable mutation failures (#211).
UpdateQuantityis validate-first (projected tick / lot / min-max / representability / risk before touching the level), propagates upstreamPriceLevelErrors instead of returningOk(None), and updates risk counters on success; a taker whose residual cannot rest is rejected before the sweep trades; a failed racy admission cleans up any empty level it created. - Two-tranche quantity conservation (#210). An aggressive iceberg’s
residual rests with exactly the unmatched total distributed across
tranches (
visible = min(display, remainder), rest hidden) instead of inflating the book, and avisible + hiddenoverflow is rejected at admission with the new typedOrderBookError::QuantityOverflowbefore any trade or mutation. Conservation (executed + resting == submitted) is property-tested. snapshots_matchcompares full maker state and FIFO (#208). The replay oracle now checks every level’s order vector in queue-consumption order (ids, variants, users, quantities, timestamps, TIF, type-specific fields) and the deterministic statistics counters includingstats_degraded; only the wall-time statistics aggregates (first_arrival_time,last_execution_time,sum_waiting_time— see thesnapshots_matchdocs for why each is inherently divergent) and the capture timestamp stay excluded. Contract tightening: aggregate-equal books with reversed FIFO or different maker identity no longer certify as replay-equal.- Failure-atomic snapshot restore (#207).
restore_from_snapshotandrestore_from_snapshot_packagevalidate every level (and reject cross-level duplicate order ids withDuplicateOrderId) against off-book structures before clearing the live book, so a failed restore leaves the pre-restore state — orders, indices, config, risk, kill-switch, engine sequence — completely untouched. - Snapshot package format v3 (#206). Pricelevel 0.9 statistics can
serialize a
stats_degradedfield that 0.8 readers reject, so newly written packages are stampedORDERBOOK_SNAPSHOT_FORMAT_VERSION = 3. Reads acceptORDERBOOK_SNAPSHOT_MIN_READ_VERSION (2)..=3— legacy v2 packages still restore — while1and future versions stay rejected with the existing typed error.
§What’s New in Version 0.11.0
§v0.11.0 — replay reproduces the trade-ID stream: namespace in ReplayBookConfig (#200)
ReplayBookConfig.trade_id_namespace: Option<Uuid>. v0.10.5 (#199) made the trade-ID namespace injectable onOrderBook, but everyReplayEngine::replay_from*entry point still built its book with a random namespace, so trade IDs produced through the shipped replay API were not reproducible. The config now carries the live book’s namespace and applies it viaOrderBook::set_trade_id_namespacebefore any journal events are replayed; a*_with_configreplay under an injectedClockthen reproduces the live trade-ID stream byte-identically.ReplayBookConfig::newkeeps its six structural parameters (namespace defaults toNone) — chain the newwith_trade_id_namespace(namespace)builder to set it. Without a namespace the fresh book keeps a random one, as before.- Suffix replays with a namespace are rejected. Applying a
namespace restarts the trade-ID counter at 0, so a namespace-carrying
config with
from_sequence != 0would mint wrong or duplicate IDs; the*_with_configentry points return the new typedReplayError::NamespaceRequiresFullReplayinstead. Namespace-free suffix replay keeps working. - Breaking (semver-minor under 0.x):
ReplayBookConfiggained a public field, so exhaustive struct literals no longer compile — addtrade_id_namespace: Noneor use..Default::default(); andReplayErrorgained theNamespaceRequiresFullReplayvariant, so exhaustive matches need a new arm.ReplayBookConfig::new(...)callers are unaffected. No journal or snapshot format change, noORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump.
§What’s New in Version 0.10.5
§v0.10.5 — injectable trade-ID namespace (#199)
OrderBook::set_trade_id_namespace(&mut self, namespace: Uuid). Every constructor used to mint the trade-ID namespace internally withUuid::new_v4(), so trade IDs differed between a live run and its replay even with an injectedClockand an identical command stream — the namespace was the only entropy left in the trade-ID stream (pricelevel::UuidGeneratoris UUID v5 over namespace + counter). The new setter, symmetric withset_clock, replaces the generator (counter restarts at 0) and composes with every existing constructor. Call it before any orders are submitted.OrderBook::with_clock_and_namespace(symbol, clock, namespace). Convenience constructor for the fully deterministic setup (injected clock + injected namespace): the same command stream then produces byte-identical trade IDs across live/replay. A deterministic namespace choice such as UUID v5 of the symbol under a venue root gives every book a stable, distinct stream.- Default constructors are unchanged: without injection each book still
gets a fresh random namespace. No wire-format or snapshot change, no
ORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump. Note the guarantee currently applies to books you construct yourself; the sequencer’sReplayEngineentry points still build their books with a random namespace — wiring the seam intoReplayBookConfigis tracked in issue #200.
§What’s New in Version 0.10.4
§v0.10.4 — exact-fee API: try_calculate_fee + published guaranteed-exact bound (#197)
FeeSchedule::try_calculate_fee(notional, is_maker) -> Result<i128, FeeOverflow>. Fallible variant ofcalculate_feewith identical rounding (truncation toward zero, sign applied after the unsigned-domain magnitude) that returns the newFeeOverflowerror instead of clamping whennotional × |bps|overflowsu128. AnOkvalue is always the mathematically exact fee and equalscalculate_fee’s output, so journaled / replayable venues can reject an order rather than record a clamped fee.- Published guaranteed-exact input bound.
FeeSchedule::max_guaranteed_exact_notional_for_bps(bps)(const fn) returns the multiplication-safety boundu128::MAX / |bps|(u128::MAXfor a zero rate) at or below which the fee is guaranteed exact, andFeeSchedule::max_guaranteed_exact_notional()takes the minimum over the maker and taker legs — a single venue-level admission bound that makes the saturating branch ofcalculate_feeprovably unreachable. The guarantee is sufficient, not tight: above the boundtry_calculate_feerejects conservatively even though the clampedcalculate_feevalue can coincide with the exact fee at isolated notionals. calculate_feebehavior is unchanged (bit-identical, including the saturated clamp of magnitudeu128::MAX / 10_000); its docs now state the exactness guarantee.FeeOverflowis re-exported at the crate root. No wire-format change, noORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump.
§What’s New in Version 0.10.3
§v0.10.3 — special-order tracker survives snapshot restore (#194)
- Restored pegged / trailing-stop orders re-price again.
restore_from_snapshotrebuilt the resting book but left thespecial_order_tracker(thespecial_ordersfeature) freshly-initialized, so a restored pegged or trailing-stop order was never re-registered and never re-priced after a snapshot restore. The shared rebuild pass now re-registers every restored resting special order in the same deterministic price-then-insertion-sequence walk that repopulatesorder_locations/user_orders. The tracker holds only order ids — the trailing-stop watermark (last_reference_price) and the pegged / stop price live in the order data and survive the round-trip, so no re-pricing state is lost. - No wire-format or public-API change: no new fields, no event-shape change,
and no
ORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump.
§What’s New in Version 0.10.2
§v0.10.2 — deterministic user_orders rebuild on snapshot restore (#192)
cancel_orders_by_useris now byte-identical across restores.restore_from_snapshot_packagerebuilt theuser_ordersindex from each level’s order-unstableiter_orders()view, so the per-userVec<Id>came back in a different order on every fresh book (theDashMaphasher is seeded per instance) and a post-restorecancel_orders_by_userdiverged across restores of the same package. The rebuild now walks price levels in the same fixed price-then-insertion-sequence order the mass-cancel sweeps use (PriceLevel::snapshot_by_seq_into), so the restored index — and any subsequent by-user cancel — is deterministic across every restore. The order reflects the resting book at snapshot time, not the original admission history (a snapshot cannot recover that). Pure journal replay was unaffected.- No wire-format or public-API change: no new fields, no event-shape change,
and no
ORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump.
§What’s New in Version 0.10.1
§v0.10.1 — replay-stable mass-cancel result ordering (#190)
- Deterministic
cancelled_order_idsordering.cancel_all_orders,cancel_orders_by_side, andcancel_orders_by_price_rangenow enumerate cancelled orders through the same fixed traversal the eviction sweep uses: bids first then asks; within a side, price levels in ascending price (theSkipMap’s natural key order, no sort); within a level, ascending insertion sequence (PriceLevel::snapshot_by_seq_into, the exact order the matching engine consumes resting orders). Previously the ids were read from order-unstable structures (order_locations/ per-leveliter_orders) whoseDashMaphasher is seeded per instance, so two processes replaying the same command stream could journal divergentSequencerResult::MassCancelledpayloads. The cancelled set and count are unchanged — only the order ofcancelled_order_idsis now byte-identical across processes and replay. cancel_orders_by_useris unchanged and was already replay-stable: it drains theuser_ordersindex in admission-history order. That determinism contract is now documented alongside the others.- No wire-format or public-API change: no new fields, no event-shape change,
and no
ORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump.
§What’s New in Version 0.10.0
§v0.10.0 — host-driven GTD / DAY expiry sweep (#189)
- New
OrderBook::evict_expired_orders(now_ms)— a host-driven sweep that removes every resting order whose time-in-force has expired as of the caller-supplied timestamp.now_msisTimestampMs(Unix milliseconds, the same unitclock().now_millis()compares against) and is passed in by the caller — the sweep never reads the book’s own clock — so a scheduler drives cadence and the sequencer can journal the exact cutoff. The matching hot path is untouched: there is no lazy per-match expiry check, so expiry is an explicit maintenance pass, not an implicit cost on every submit. The honest consequence: an expired-but-unsweptGtd/Dayorder remains resting and matchable — it can still trade until the host calls the sweep; the no-post-expiry-trade guarantee holds only after the sweep runs. Expiry uses the single boundary predicate that admission uses (now >= deadlineforGtd,now >= market_closeforDay), so an order admitted at a given instant is never simultaneously evictable at that instant. Returns the evicted orders asVec<Arc<OrderType<T>>>; a second sweep at the samenow_msis idempotent and returns empty. - Deterministic eviction order. Evicted orders — and the
Cancelled { reason: TimeInForceExpired }state transitions andPriceLevelChangedEvents emitted as a side effect — follow one fixed, replay-stable order: bids first then asks; within a side, price levels in ascending price (theSkipMap’s natural key order, no sort); within a level, ascending insertion sequence (the exact order the matching engine consumes resting orders — not the non-deterministiciter_ordersview). Each order is removed through the same single-order cancel path ascancel_order, so the price-level cache, depth statistics,order_locations/user_ordersindices, risk state, special-order tracker, and order-state tracker all stay consistent. - Manager parity.
BookManagerStdandBookManagerTokiogainevict_expired_orders(symbol, now_ms)(per-symbol pass-through,Nonefor an unknown symbol) andevict_expired_across_books(now_ms)(all books, mirroring thecancel_*_across_booksidiom). - Journaled as a sequencer command. New
SequencerCommand::EvictExpiredOrders { now_ms }variant (appended, so existing journals’ bincode variant indices are unchanged). Replay applies the journaled cutoff — never the replay clock — so the sweep reproduces byte-identically;snapshots_matchholds between a live book and its replay. Old journals replay unchanged; new journals carrying the variant fail on older binaries, consistent with theMarketOrderByAmountprecedent. NoORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump required (the version gates the snapshot package, not the journal command enum). - Breaking (the reason this is 0.10.0):
SequencerCommandandSequencerResultare now#[non_exhaustive]. Downstream code that matches them exhaustively must add a wildcard arm (_ => …) once and recompile; in exchange, future command/result additions are source-compatible instead of repeating this break. Adding theEvictExpiredOrdersvariant itself is what surfaced the hazard: on the previously-exhaustive enum it would have silently broken downstream matches inside the 0.9.x range. TimestampMsre-exported at the crate root and viaprelude, so the newnow_msparameter can be constructed without reaching intopriceleveldirectly.- Runnable example:
cargo run -p examples --bin gtd_expiry_sweep.
§What’s New in Version 0.9.2
§v0.9.2 — constant-work per-price aggregates + attribution & unit docs (#185, #186, #187)
- Constant-work per-price aggregate accessors (#186) — an O(log N)
point lookup + O(1) counter read, with no per-order materialization. New
read-only methods on
OrderBook<T>:visible_quantity_at_price,hidden_quantity_at_price,total_quantity_at_price, andorder_count_at_price. Each does an O(log N)SkipMappoint lookup then reads the level’s maintained atomic counter (one relaxed load; two fortotal_quantity_at_price, which sums visible + hidden) — no per-orderArcis materialized and noT: Defaultconversion runs, so they are the cheap way to poll one level’s depth or count.order_count_at_priceis the counterpart toqueue_ahead_at_pricethat drops the per-order term: O(log N) here vs O(log N + K) for the queue-walking version. All four returnNonefor an absent level and read advisory, eventually-consistent counters — takecreate_snapshotfor a mutually-consistent view. - Per-call fill attribution, documented and proven (#185). The
add_order_with_resultguarantee is now explicit: concurrent submits on the same book each receive exactly their own fills, because theTradeResultis built from that call’s privateMatchResultand the engine holds no shared trade accumulator. On the error-after-fills paths (an unfillable IOC remainder, or a self-trade-prevention cancellation after earlier non-self fills) the caller instead gets the typedErrand the executed fills reach only the trade listener. A multi-thread concurrency test pins it. New convenience wrappersadd_limit_order_with_resultandadd_limit_order_with_user_and_resultmirror the plainadd_limit_order*builders while returning theTradeResultdirectly. - GTD / market-close millisecond unit documented (#187).
has_expired,set_market_close_timestamp, and thetime_in_forceparameter docs now state that GTD deadlines and the market-close timestamp are milliseconds since the Unix epoch (the same unitclock().now_millis()compares against). A pinning test proves a seconds-form deadline reads as instantly expired.
§What’s New in Version 0.9.1
§v0.9.1 — add_order_with_result (#184)
- New public API on
OrderBook<T>:add_order_with_resultsubmits an order and returns theTradeResultproduced by the match directly —Ok((Arc<OrderType<T>>, Option<TradeResult>))— instead of relying on theTradeListenercallback.Nonewhen the order produced no fills; an installed listener still fires with the exact sameTradeResult.add_orderis unchanged in behavior and stays free of the extraMatchResultclone when no listener is installed.
§What’s New in Version 0.9.0
§v0.9.0 — Upgrade to pricelevel 0.8.0 (#130)
- Price-time priority preserved across partial fills. Picks up the
upstream
pricelevelfix (PriceLevel#39) where a partially-filled resting maker keeps its place at the front of the level queue, resolving #88: a partial fill no longer demotes the maker behind later same-price arrivals. Locked in bytest_partial_fill_preserves_price_time_priority_issue_88. - Deterministic match timestamps.
PriceLevel::match_orderno longer reads the wall clock; the engine passes the book’sClocktime as the taker timestamp, so trade timestamps follow the installed clock and replay stays deterministic. - Domain newtypes on the public surface (breaking). Through the
pricelevelre-exports andMatchResult/OrderTypeaccessors, several values now carryQuantity/Price/TimestampMsinstead of rawu64/u128(e.g.MatchResult::remaining_quantity()now returnsQuantity). OrderBook-rs’s own snapshot / statistics queries are unchanged and still return raw integers; downstream code readingpriceleveltypes through the re-exports may need.as_u64()/.as_u128(). Minor bump under0.xsemver. - Dependency refresh:
pricelevel0.7→0.8,async-nats0.47→0.49,dashmap6.1→6.2,bitflags2.11→2.13,either1.15→1.16,crc32fast1→1.5,proptest1.7→1.11.
§What’s New in Version 0.8.0
§v0.8.0 — Quote-notional market orders (#85)
- New public API on
OrderBook<T>:match_market_order_by_amountand the STP-awarematch_market_order_by_amount_with_user, plus the conveniencesubmit_market_order_by_amountandsubmit_market_order_by_amount_with_userwrappers that run the kill-switch and pre-trade risk gates. - Binance
quoteOrderQtysemantics. Callers say “buy ~$1,000 of BTC” without converting to base quantity. The matching loop walks the opposite side until the requested quote-notionalamountis consumed, the book is exhausted, or — whenlot_sizeis configured on the book — the residual notional cannot fund another whole lot. Fees are exclusive: caller paysamount + taker_fee. - Lot enforcement preserved. Per-level base quantity is rounded
down to a multiple of
lot_size, so notional walks never emitqty=0trades when the budget falls below one full lot at the current level price.lot_size = Noneis equivalent tolot = 1. - New error variant
OrderBookError::InsufficientLiquidityNotional— distinct fromInsufficientLiquidityso callers can pattern-match on quote-vs-base semantics. TradeResult.quote_notional: u128— populated for both the base-quantity and quote-notional market-order paths so consumers readΣ price × quantitydirectly without recomputing per-trade.#[serde(default)]keeps existing JSON / Bincode payloads parseable.- Additive
SequencerCommand::MarketOrderByAmount { id, amount, side }variant. Old journals replay byte-identical; new journals carrying this variant fail on older binaries — consistent with the precedent for priorSequencerCommandrollouts. NoORDERBOOK_SNAPSHOT_FORMAT_VERSIONbump required. StopConditionrefactor of the matching loop — single inner implementation drives both base-qty and notional walks. Base-qty path stays allocation- and branch-light: the new helpers fold to the same arithmetic the previous loop emitted whenlot <= 1.- Runnable example:
cargo run -p examples --bin market_order_by_amount. - HDR bench:
notional_walk_hdrmirrorsaggressive_walk_hdrwith the notional path so p50/p99/p99.9/p99.99 can be compared.
§What’s New in Version 0.7.0
§v0.7.0 — Feature-gated allocation counter
- New feature
alloc-counters(default off). Exposes [CountingAllocator] and [AllocSnapshot] at the crate root. Wraps any innerGlobalAllocand tracks fourAtomicU64counters:allocs,deallocs,bytes_allocated,bytes_deallocated. - Bench / test binaries opt in via
#[global_allocator] static A: CountingAllocator<System> = .... The libraryrlibdoes not install a global allocator. bench_countbench +alloc_budget_testsintegration test run the mixed 70/20/10 workload; the bench reportsallocs_per_op, the test asserts a conservative ceiling for regression detection.BENCH.mdgains an “Allocation profile” section.
§v0.7.0 — Metrics and Observability (#60)
- New optional
metricsfeature wires Prometheus-style counters and gauges into the matching engine. Defaultoff; when enabled, every increment goes through the globalmetricsfacade so any compatible recorder (Prometheus exporter, OpenTelemetry bridge, etc.) can collect them. - Surface (stable across
0.7.x):orderbook_rejects_total{reason="..."}— counter, one increment per rejected order. Label value is theRejectReasonDisplaystring.orderbook_depth_levels_bid/orderbook_depth_levels_ask— gauges, current count of distinct price levels on each side. Updated on every structural mutation (add, cancel, modify, fill).orderbook_trades_total— counter, monotonic count of every emitted trade transaction (one increment perMatchResulttransaction).
- Determinism preserved. Metrics emission is out-of-band:
no allocation on the happy path, no influence on matching
outcomes, and
restore_from_snapshot_packagedeliberately does not rehydrate counters — they are operational only and live for the process lifetime. The integration testtests/metrics/proves byte-identical snapshots between two books with metrics enabled. - Compile-time no-op. When the feature is off every helper
in
orderbook::metricscompiles to an empty function so call-sites in the matching hot path stay unconditional. - Example:
examples/src/bin/prometheus_export.rs(run withcargo run --features metrics --bin prometheus_export) demonstrates installing themetrics-exporter-prometheusrecorder and dumping the exposition payload.
§v0.7.0 — Feature-gated binary wire protocol
- New
wirefeature flag behind which a small, length-prefixed binary protocol lives — every frame is[len:u32 LE | kind:u8 | payload],lencoverskind + payload, and all multi-byte integers are little-endian. Disabled by default; the existing JSON and bincode paths are unchanged. The protocol is additive. MessageKind—#[repr(u8)]enum with stable explicit discriminants. Inbound:NewOrder = 0x01,CancelOrder = 0x02,CancelReplace = 0x03,MassCancel = 0x04. Outbound:ExecReport = 0x81,TradePrint = 0x82,BookUpdate = 0x83.- Zero-copy inbound —
NewOrderWire,CancelOrderWire,CancelReplaceWire,MassCancelWireare#[repr(C, packed)]withzerocopy::{FromBytes, IntoBytes, Unaligned, Immutable, KnownLayout}derives. Each ships aconst _: () = assert!(size_of::<…>() == N)guard. Decoding is safe —zerocopyperforms the layout validation, nounsafeis required at any wire call site. - Byte-cursor outbound —
ExecReport,TradePrintWire,BookUpdateWireare encoded via explicitextend_from_slicecalls. Outbound is I/O-dominated; this keeps the layout free to evolve. TryFrom<&NewOrderWire> for OrderType<()>— boundary mapping that copies each packed field into a stack local first (taking a reference to a packed field is UB), validates the side / TIF / order_type discriminants, and rejects negative prices viaWireError::InvalidPayload.doc/wire-protocol.mdwith per-message layout tables, discriminant table, framing rule, and endianness statement.- Round-trip
proptestcoverage in everysrc/wire/{inbound,outbound}/*.rsmodule. - Example:
examples/src/bin/wire_roundtrip.rs(required-features = ["wire"]).
§v0.7.0 — HDR-histogram tail-latency bench suite
- Six new
*_hdrbench binaries underbenches/order_book/:add_only,cancel_only,aggressive_walk,mixed_70_20_10,thin_book_sweep,mass_cancel_burst. Each records per-sample nanosecond latencies into anhdrhistogram::Histogramand emitsp50/p99/p99.9/p99.99+min/max. Coexists with the existing Criterion benches. make bench-hdrconvenience target.- Headline numbers + methodology in
BENCH.mdat the repo root, with a closed-loop disclosure block (the suite measures service time, not load-induced tail).
§v0.7.0 — Closed RejectReason enum
- New
RejectReason— closed#[non_exhaustive] #[repr(u16)]enum with stable explicit discriminants (1..13 +Other(u16)). The canonical wire-side reject taxonomy; consumers can route on the numeric code without parsing strings. OrderStatus::Rejected.reason: Stringis nowRejectReason— typed, machine-routable, and stable across0.7.x. Breaking change on a public variant shape; allowed under the0.6.x → 0.7.xminor delta in0.xsemver.impl From<&OrderBookError> for RejectReason— operational ergonomics. Exhaustive match — a futureOrderBookErrorvariant addition is caught at compile time.- Tracker emission on every reject path that already transitioned
the tracker. Kill switch, risk gates, and the three internal
sites in
modifications.rs(validation / post-only / missing user id) now record typed reasons. STP cancel-taker and IOC/FOKInsufficientLiquiditypaths still return typed errors without transitioning the tracker — deferred to a follow-up.
§v0.7.0 — Pre-trade risk layer
- New
RiskConfigwith three opt-in guard-rails:max_open_orders_per_account,max_notional_per_account, andprice_band_bpsagainst a configurableReferencePriceSource(LastTrade/Mid/FixedPrice). Builder pattern —RiskConfig::new().with_*(...)chained. - Three new typed reject variants on the existing
#[non_exhaustive]OrderBookError:RiskMaxOpenOrders,RiskMaxNotional,RiskPriceBand. Each carries enough context (account, current, limit, deviation) for downstream consumers to act without parsing a string. OrderBook::set_risk_config(...)/risk_config()/disable_risk()— operator-driven gating. Check ordering on submit/add:kill_switch → risk → STP → fees → match. Market orders bypass the risk layer (no submitted price, no rest); kill switch still gates them.- Allocation-free on the happy path. Per-account counters are
(AtomicU64, AtomicCell<u128>)pairs; per-order risk state is aDashMap<Id, RiskEntry>. OrderBookSnapshotPackage.risk_config: Option<RiskConfig>— config persists across snapshot/restore. On restore, per-account counters and the per-order map are rebuilt by walking the snapshot’s resting orders. Snapshot format version stays at2; the field is additive via#[serde(default)].- Example:
examples/src/bin/risk_limits.rs.
§v0.7.0 — Operational kill switch
- New
OrderBook::engage_kill_switch(),OrderBook::release_kill_switch(), andOrderBook::is_kill_switch_engaged()— atomic operational halt for new flow. While engaged, everysubmit_market_order*,add_order, and non-Cancelupdate_ordercall returns the newOrderBookError::KillSwitchActivevariant before any matching, fee, or STP work happens. Cancel and mass-cancel paths are explicitly not gated so operators can drain the resting book. The flag persists across snapshot/restore. OrderBookError::KillSwitchActive— new typed reject variant on the existing#[non_exhaustive]enum.OrderBookSnapshotPackage.kill_switch_engaged: bool— operational state persists across snapshot/restore. Snapshot format version stays at2; the field is additive via#[serde(default)].- When an
OrderStateTrackeris configured, kill-switched rejections are recorded asOrderStatus::Rejected { reason: RejectReason::KillSwitchActive }. - Example:
examples/src/bin/kill_switch_drain.rs.
§v0.7.0 — Global engine_seq across outbound streams
- New
OrderBook::next_engine_seq()andOrderBook::engine_seq()accessors backed by anAtomicU64counter. Every outbound emission (trade event, price-level change event) mints exactly one seq, in emission order, so external consumers can perform cross-stream gap detection and merge events fromTradeListenerandPriceLevelChangedListenerinto a single ordered view. engine_seq: u64field added to every outbound event type:TradeResult,TradeEvent,PriceLevelChangedEvent, and the NATSBookChangeEntry. JSON payloads are forward-compatible (#[serde(default)]falls back to0for v0.6.x payloads).- Snapshot format version bumped to
2.OrderBookSnapshotPackagecarriesengine_seqso thatrestore_from_snapshot_packageresumes monotonicity exactly from the snapshotted point.version: 1packages are rejected byvalidate(). BookChangeBatch.sequenceretains its existing per-batch publisher-counter semantics; cross-stream gap detection moves to the per-eventBookChangeEntry.engine_seq. Both fields ship in the same payload for incremental adoption.
§v0.7.0 — Clock trait for deterministic replay
- New
Clocktrait with two implementations,MonotonicClock(production, wrapsSystemTime::now) andStubClock(replay / tests, monotonicAtomicU64counter with configurable start and step). Re-exported at the crate root and viaprelude. OrderBook::with_clockconstructor plusset_clockandclock()accessors. The defaultOrderBook::newkeeps wrappingMonotonicClockinternally — existing callers observe no behavioural change.ReplayEngine::replay_from_with_clockfor byte-identical replay tests and disaster-recovery pipelines that must reproduce engine timestamps deterministically.- Wall-clock reads are no longer present inside the matching core —
every stamp flows through
self.clock().now_millis(). - Behavioural change (same type signature):
OrderStateTracker::get_historyandOrderBook::get_order_historynow returnVec<(u64 /* milliseconds */, OrderStatus)>instead of nanoseconds; theClock::now_millisunit is the only one the trait exposes.
§v0.6.2 — Dependency Bumps & Bincode 2.x Migration
- Dependency refresh:
uuid1.23,tokio1.52,sha20.11,async-nats0.47,bincode2.0 (crates.iobincode 3.0.0is acompile_error!stub, so2.0is the current usable major). - Bincode API migration (feature
bincode): theBincodeEventSerializernow usesbincode::serde::encode_to_vec/decode_from_slicewithbincode::config::standard(). The public trait and type surface are unchanged. - Wire-format note: bincode 1.x and 2.x produce different byte
layouts on the NATS transport path. The on-disk journal uses
serde_jsonand is unaffected (ORDERBOOK_SNAPSHOT_FORMAT_VERSIONstays at1).
§v0.6.1 — NATS Integration, Sequencer & Order State
- NATS JetStream Publishers: Trade event and book change publishers with retry, batching, and throttling (
natsfeature) - Zero-Copy Serialization: Pluggable
EventSerializertrait with JSON and Bincode implementations (bincodefeature) - Sequencer Subsystem:
SequencerCommand,SequencerEvent,SequencerResulttypes for LMAX Disruptor-style total ordering - Append-Only Journal:
FileJournalwith memory-mapped segments, CRC32 checksums, and segment rotation (journalfeature) - In-Memory Journal:
InMemoryJournalfor testing and benchmarking - Deterministic Replay:
ReplayEnginefor disaster recovery and state verification from journal - Order State Machine:
OrderStatus,CancelReason,OrderStateTrackerfor explicit lifecycle tracking (Open → PartiallyFilled → Filled / Cancelled / Rejected) - Order Lifecycle Query API:
get_order_history(),active_order_count(),terminal_order_count(),purge_terminal_states() - Upgrade to pricelevel v0.7:
Id,Price,Quantity,TimestampMsnewtypes for stronger type safety
§v0.5.x — Validation, STP, Fees & Mass Cancel
- Order Validation: Tick size, lot size, and min/max order size validation with configurable limits
- Self-Trade Prevention (STP):
CancelTaker,CancelMaker,CancelBothmodes with per-orderuser_idenforcement - Fee Model: Configurable
FeeSchedulewith maker/taker fees, fee fields inTradeResult - Mass Cancel Operations: Cancel all, by side, by user, by price range — with
MassCancelResulttracking - Cross-Book Mass Cancel:
cancel_all_across_books(),cancel_by_user_across_books(),cancel_by_side_across_books()onBookManager - Snapshot Config Preservation:
restore_from_snapshot_package()preserves fee schedule, STP mode, tick/lot size, and order size limits
§v0.4.8 — Performance & Architecture
- Performance Boost:
PriceLevelCachefor faster best bid/ask lookups,MatchingPoolto reduce matching engine allocations - Cleaner Architecture: Refactored modification and matching logic for better separation of concerns
- Enhanced Concurrency: Improved thread-safe operations under heavy load
§Status
This project is in active development. The core matching engine, order validation, STP, fees, mass cancel, NATS integration, sequencer journal, and order state tracking are production-ready. The Sequencer runtime (async event loop) is under development.
§Advanced Features
§Market Metrics & Analysis
The order book provides comprehensive market analysis capabilities:
- VWAP Calculation: Volume-Weighted Average Price for analyzing true market price
- Spread Analysis: Absolute and basis point spread calculations
- Micro Price: Fair price estimation incorporating depth
- Order Book Imbalance: Buy/sell pressure indicators
- Market Impact Simulation: Pre-trade analysis for estimating slippage and execution costs
- Depth Analysis: Cumulative depth and liquidity distribution
§Intelligent Order Placement
Advanced utilities for market makers and algorithmic traders:
- Queue Analysis:
queue_ahead_at_price()- Check depth at specific price levels - Tick-Based Pricing:
price_n_ticks_inside()- Calculate prices N ticks from best bid/ask - Position Targeting:
price_for_queue_position()- Find prices for target queue positions - Depth-Based Strategy:
price_at_depth_adjusted()- Optimal prices based on cumulative depth
§Functional Iterators
Memory-efficient, composable iterators for order book analysis:
- Cumulative Depth Iteration:
levels_with_cumulative_depth()- Lazy iteration with running depth totals - Depth-Limited Iteration:
levels_until_depth()- Auto-stop when target depth is reached - Range-Based Iteration:
levels_in_range()- Filter levels by price range - Predicate Search:
find_level()- Find first level matching custom conditions
Benefits:
- Zero allocation - O(1) memory vs O(N) for vectors
- Lazy evaluation - compute only what’s needed
- Composable - works with standard iterator combinators (
.map(),.filter(),.take()) - Short-circuit - stops early when conditions are met
§Multi-Book Management
Centralized trade event routing and multi-book orchestration:
- BookManager: Manage multiple order books with unified trade listener
- Standard & Tokio Support: Synchronous and async variants
- Event Routing: Centralized trade notifications across all books
§Aggregate Statistics
Comprehensive statistical analysis for market condition detection:
- Depth Statistics:
depth_statistics()- Volume, average sizes, weighted prices, std dev - Market Pressure:
buy_sell_pressure()- Total volume on each side - Liquidity Health:
is_thin_book()- Detect insufficient liquidity - Distribution Analysis:
depth_distribution()- Histogram of liquidity concentration - Imbalance Detection:
order_book_imbalance()- Buy/sell pressure ratio (-1.0 to 1.0)
Use cases:
- Market condition detection and trend identification
- Risk management and liquidity monitoring
- Strategy adaptation based on real-time conditions
- Trading decision support and analytics
§Enriched Snapshots
Pre-calculated metrics in snapshots for high-frequency trading:
- Enriched Snapshots:
enriched_snapshot()- Single-pass snapshot with all metrics - Custom Metrics:
enriched_snapshot_with_metrics()- Select specific metrics for optimization - Metric Flags: Bitflags for precise control over calculated metrics
Metrics included:
- Mid price and spread (in basis points)
- Total depth on each side
- VWAP for top N levels
- Order book imbalance
Benefits:
- Single pass through data vs multiple passes
- Better cache locality and performance
- Reduced computational overhead
- Flexibility with optional metric selection
§Performance Analysis of the OrderBook System
This analyzes the performance of the OrderBook system based on tests conducted on an Apple M4 Max processor. The data comes from a High-Frequency Trading (HFT) simulation and price level distribution performance tests. The figures below are representative single-run numbers measured on orderbook-rs 0.9.0 with the bundled examples orderbook_hft_simulation and orderbook_contention_test (cargo run --release -p examples --bin <name>); absolute throughput is workload-, machine-, and run-dependent.
§1. High-Frequency Trading (HFT) Simulation
§Test Configuration
- Symbol: BTC/USD
- Duration: 5000 ms (5 seconds)
- Threads: 30 threads total
- 10 maker threads (order creators)
- 10 taker threads (order executors)
- 10 canceller threads (order cancellers)
- Initial orders: 1020 pre-loaded orders
§Performance Results
| Metric | Total Operations | Operations/Second |
|---|---|---|
| Orders Added | 465,314 | 93,040.99 |
| Orders Matched | 191,555 | 38,302.02 |
| Orders Cancelled | 183,700 | 36,731.39 |
| Total Operations | 840,569 | 168,074.41 |
§Initial vs. Final OrderBook State
| Metric | Initial State | Final State |
|---|---|---|
| Best Bid | 9,900 | 9,840 |
| Best Ask | 10,000 | 10,010 |
| Spread | 100 | 170 |
| Mid Price | 9,950.00 | 9,925.00 |
| Total Orders | 1,020 | 44,987 |
| Bid Price Levels | 21 | 11 |
| Ask Price Levels | 21 | 12 |
| Total Bid Quantity | 7,750 | 346,031 |
| Total Ask Quantity | 7,750 | 473,092 |
§2. Contention Pattern Performance Tests
§Configuration
- Threads: 12
- Test Duration: 3000 ms per sub-test
- Concurrent Operations: Multi-threaded lock-free architecture
§Read / Write Operation Ratio
Mixed read/write workload over 500 resting orders across 40 price levels;
the Read % is the fraction of operations that are read-only (snapshot /
best-price / depth queries) versus mutating (add / cancel / match).
| Read % | Operations/Second |
|---|---|
| 0% | 305,435.88 |
| 25% | 63,103.90 |
| 50% | 51,933.72 |
| 75% | 54,960.34 |
| 95% | 100,379.54 |
§Price Level Distribution
Throughput as the resting depth is spread across a varying number of price levels (100 orders per level, except the 5- and 1-level cases which pack the same orders into fewer levels).
| Price Levels | Operations/Second |
|---|---|
| 100 | 184,986.35 |
| 50 | 188,085.24 |
| 10 | 70,338.52 |
| 5 | 68,610.25 |
| 1 | 61,302.26 |
§Hot Spot Contention Test
All threads hammer a single shared price level (20 hot-spot orders + 480
regular); higher hot-spot percentages concentrate more operations on that one
lock-free level, where the crossbeam-skiplist + dashmap + atomics design
shines.
| % Operations on Hot Spot | Operations/Second |
|---|---|
| 0% | 14,978,484.36 |
| 25% | 19,191,927.99 |
| 50% | 25,890,620.87 |
| 75% | 31,529,898.64 |
| 100% | 31,607,744.24 |
§Performance Improvements and Deadlock Resolution
The significant performance gains, especially in the “Hot Spot Contention Test,” and the resolution of the previous deadlocks are a direct result of refactoring the internal concurrency model of the PriceLevel.
-
Previous Bottleneck: The original implementation relied on a
crossbeam::queue::SegQueuefor storing orders. While the queue itself is lock-free, operations like finding or removing a specific order required draining the entire queue into a temporary list, performing the action, and then pushing all elements back. This process was inefficient and created a major point of contention, leading to deadlocks under heavy multi-threaded load. -
New Implementation: The
OrderQueuewas re-designed to use a combination of:- A
dashmap::DashMapfor storing orders, allowing for highly concurrent, O(1) average-case time complexity for insertions, lookups, and removals byId. - A sequence-keyed index (a
crossbeam_skiplist::SkipMap<sequence, Id>) that maintains the crucial First-In-First-Out (FIFO) order for matching while still allowing O(log n) ordered iteration and deterministic snapshots.
- A
This hybrid approach eliminates the previous bottleneck, allowing threads to operate on the order collection with minimal contention, which is reflected in the massive throughput increase in the hot spot tests.
§3. Analysis and Conclusions
§Overall Performance
The system demonstrates excellent capability to handle over 165,000 operations per second in the high-frequency trading simulation, distributed across order creations, matches, and cancellations.
§Price Level Distribution Behavior
- Optimal Performance Range: The system performs best with 50-100 price levels, achieving roughly 185,000-188,000 operations per second.
- Performance Degradation: Performance decreases with fewer price levels (more per-level contention), dropping to around 61,000-70,000 operations per second with 1-10 levels.
- Scalability: The lock-free architecture demonstrates excellent scalability characteristics across different price level distributions.
§Hot Spot Contention
- Surprisingly, performance increases as more operations concentrate on a hot spot, reaching its maximum with 100% concentration (31,607,744 ops/s).
- This counter-intuitive behavior might indicate:
- Very efficient cache effects when operations are concentrated in one memory area
- Internal optimizations to handle high-contention cases
- Benefits of the system’s lock-free architecture
§OrderBook State Behavior
- During the HFT simulation, the order book handled a significant increase in order volume (from 1,020 to 44,987).
- The spread increased from 100 to 170, reflecting realistic market behavior under pressure.
- The final state shows substantial liquidity with over 346,000 bid quantity and 473,000 ask quantity.
§4. Practical Implications
- The system is suitable for high-frequency trading environments with the capacity to process over 165,000 mixed operations per second (and tens of millions of operations per second on a single hot price level).
- The lock-free architecture proves to be extremely effective at handling contention, especially at hot spots.
- Optimal performance is achieved with moderate price level distribution (50-100 levels).
- For real-world use cases, the system demonstrates excellent scalability and maintains performance under concurrent load.
This analysis confirms that the system design is highly scalable and appropriate for demanding financial applications requiring high-speed processing with data consistency.
Re-exports§
pub use orderbook::book_change_event::PriceLevelChangedEvent;pub use orderbook::book_change_event::PriceLevelChangedListener;pub use orderbook::clock::Clock;pub use orderbook::clock::MonotonicClock;pub use orderbook::clock::StubClock;pub use orderbook::implied_volatility::BlackScholes;pub use orderbook::implied_volatility::IVConfig;pub use orderbook::implied_volatility::IVError;pub use orderbook::implied_volatility::IVParams;pub use orderbook::implied_volatility::IVQuality;pub use orderbook::implied_volatility::IVResult;pub use orderbook::implied_volatility::OptionType;pub use orderbook::implied_volatility::PriceSource;pub use orderbook::implied_volatility::SolverConfig;pub use orderbook::iterators::LevelInfo;pub use orderbook::manager::BookManager;pub use orderbook::manager::BookManagerStd;pub use orderbook::manager::BookManagerTokio;pub use orderbook::market_impact::MarketImpact;pub use orderbook::market_impact::OrderSimulation;pub use orderbook::order_state::CancelReason;pub use orderbook::order_state::OrderStateListener;pub use orderbook::order_state::OrderStateTracker;pub use orderbook::order_state::OrderStatus;pub use orderbook::reject_reason::RejectReason;pub use orderbook::risk::ReferencePriceSource;pub use orderbook::risk::RiskConfig;pub use orderbook::risk::RiskState;pub use orderbook::sequencer::InMemoryJournal;pub use orderbook::sequencer::Journal;pub use orderbook::sequencer::JournalEntry;pub use orderbook::sequencer::JournalError;pub use orderbook::sequencer::JournalReadIter;pub use orderbook::sequencer::ReplayBookConfig;pub use orderbook::sequencer::ReplayEngine;pub use orderbook::sequencer::ReplayError;pub use orderbook::sequencer::SequencerCommand;pub use orderbook::sequencer::SequencerEvent;pub use orderbook::sequencer::SequencerResult;pub use orderbook::sequencer::snapshots_match;pub use orderbook::serialization::EventSerializer;pub use orderbook::serialization::JsonEventSerializer;pub use orderbook::serialization::SerializationError;pub use orderbook::snapshot::EnrichedSnapshot;pub use orderbook::snapshot::MetricFlags;pub use orderbook::statistics::DepthStats;pub use orderbook::statistics::DistributionBin;pub use orderbook::stp::STPMode;pub use orderbook::trade::TradeEvent;pub use orderbook::trade::TradeInfo;pub use orderbook::trade::TradeListener;pub use orderbook::trade::TradeResult;pub use orderbook::trade::TransactionInfo;pub use orderbook::FeeOverflow;pub use orderbook::FeeSchedule;pub use orderbook::ManagerError;pub use orderbook::MassCancelResult;pub use orderbook::OrderBook;pub use orderbook::OrderBookError;pub use orderbook::OrderBookSnapshot;pub use utils::current_time_millis;
Modules§
- orderbook
- OrderBook implementation for managing multiple price levels and order matching.
- prelude
- Prelude module that re-exports commonly used types and traits.
- utils
- Shared internal helpers exposed at the crate root.
Structs§
- Timestamp
Ms - Domain value type representing a timestamp in milliseconds.
Enums§
- Id
- Represents a unique identifier in the trading system.
- Order
Type - Represents different types of limit orders
- Side
- Represents the side of an order
- Time
InForce - Specifies how long an order remains active before it is executed or expires.
Type Aliases§
- Default
Order Book - Default type alias for
OrderBook<()>representing the most common use case. - Default
Order Type - Default type alias for
OrderType<()>representing the most common use case. - Legacy
Order Book - Legacy type alias for
OrderBook<()>to maintain backward compatibility. - Legacy
Order Type - Legacy type alias for
OrderType<()>to maintain backward compatibility. - OrderId
- Legacy type alias for backward compatibility with code using
OrderId.