Skip to main content

Crate extrema_infra

Crate extrema_infra 

Source
Expand description

Event-driven infrastructure for exchange-facing trading systems.

extrema_infra provides the runtime layer used by strategy binaries that need to combine timers, model workers, order execution, public market data, and private account streams in one process. The framework is intentionally small at the strategy boundary: implement Strategy, register the tasks your process needs, and start the environment with EnvBuilder. The builder creates one broadcast stream for every concrete runtime task.

The crate is organized around a few concepts:

  • Strategy is the unit of application logic. A binary can register one strategy module or several independent modules in the same runtime.
  • EventHandler contains async callbacks for schedule ticks, model predictions, order execution intents, trades, LOB updates, candles, private account updates, raw exchange-specific websocket frames, and on_lagged when a slow module lost events. All callbacks default to no-op.
  • CommandEmitter gives a strategy access to task command handles after the runtime has prepared those tasks.
  • TaskInfo declares work that the runtime owns, such as AltTaskInfo for scheduler/model/order-intent work or WsTaskInfo for websocket relays.
  • TaskKey identifies one concrete task stream. Strategies receive every registered task by default, or can bind to selected keys. A key contains a complete AltTaskType or WsChannel plus its task id.
  • prelude re-exports the common types used by strategy binaries.

§Event Model

Runtime tasks own IO, timers, model workers, and order relays. They publish normalized InfraMsg<T> values into their task-local broadcast stream. A task’s lifecycle event and primary business events share that stream.

Strategy modules handle inbound events through async callbacks and send TaskCommand values back through command handles when they need active work such as websocket connect/subscribe or order execution. Register a module with EnvBuilder::with_strategy_module to receive all tasks, or EnvBuilder::with_strategy_module_on to receive selected task streams.

Market data, account updates, timers, model outputs, and order relays can run on independent cadences without forcing one polling loop or one module to own duplicate IO.

Delivery is lossy by design. Each task’s broadcast ring is bounded, and a module that falls behind loses the oldest events instead of stalling the publisher. The runtime then calls EventHandler::on_lagged(key, skipped), coalesced per task stream to at most once per second; use it to resync, for example by re-reading account state over REST.

§Core Trait Responsibilities

Strategy is the lifecycle trait. It has one required startup hook, initialize, which should complete after loading config, initializing clients, or warming local state. Runtime event loops start after this phase.

EventHandler is the inbound event surface. Its methods are callbacks: on_schedule, on_trade, on_candle, on_acc_pos, on_inst_intent, on_order_execution, on_ws_other, and so on. on_lagged(key, skipped) is the one callback that takes no InfraMsg<T>. Every callback defaults to no-op, so modules stay narrow and only implement the events that matter to them. Modules receive every registered task by default; explicit task bindings avoid receiver creation and wakeups for unrelated tasks.

CommandEmitter is the outbound command surface. After tasks are prepared, the runtime supplies a CommandRegistry. Strategy modules store that registry, then find task handles by (task type, task id) or (websocket channel, task id) when they need to send commands.

MarketLobApi, LobPublicRest, LobPrivateRest, and LobWebsocket define exchange-client adapters. Implementations translate exchange-specific REST and websocket details into the normalized data types consumed by strategy modules. LobWsDecoder plugs the websocket frame decoding of a venue implemented outside this crate into the runtime: register it with EnvBuilder::with_ws_decoder and declare its tasks on Market::Custom(..); its events reach the same callbacks as built-in venues.

§Event Flow

A typical runtime flow looks like this:

EnvBuilder
  -> registers TaskInfo values
  -> expands each declaration into concrete TaskKey values
  -> creates one broadcast stream per concrete task
  -> registers Strategy modules
  -> registers LobWsDecoder values for Market::Custom tasks
  -> build() validates tasks, bindings and decoders
  -> EnvMediator::execute()
      -> Strategy::initialize()
      -> prepare AltTask/WsTask workers and command handles
      -> CommandEmitter::command_init()
      -> subscribe strategies and spawn their event loops
      -> spawn prepared task workers
      -> tasks publish InfraMsg<T>
      -> EventHandler callbacks react
      -> strategies send TaskCommand through CommandHandle when needed

InfraMsg<T> always carries the task_id that emitted the event, but not the full TaskKey. The builder therefore rejects id reuse within the same AltTaskType or WsChannel variant, while different variants may reuse an id.

§Minimal scheduler runtime

use std::{sync::Arc, time::Duration};

use extrema_infra::prelude::*;

#[derive(Clone)]
struct Heartbeat {
    registry: Arc<CommandRegistry>,
}

impl Heartbeat {
    fn new() -> Self {
        Self {
            registry: Arc::new(CommandRegistry::default()),
        }
    }
}

impl Strategy for Heartbeat {
    async fn initialize(&mut self) {
        // Load config, warm local state, or initialize API clients here.
    }
}

impl CommandEmitter for Heartbeat {
    fn command_init(&mut self, registry: Arc<CommandRegistry>) {
        self.registry = registry;
    }

    fn command_registry(&self) -> Arc<CommandRegistry> {
        self.registry.clone()
    }
}

impl EventHandler for Heartbeat {
    async fn on_schedule(&mut self, msg: InfraMsg<AltScheduleEvent>) {
        println!("schedule tick from task_id={}", msg.task_id);
    }
}

#[tokio::main]
async fn main() -> InfraResult<()> {
    let schedule = AltTaskInfo {
        alt_task_type: AltTaskType::TimeScheduler(Duration::from_secs(5)),
        chunk: 1,
        task_base_id: Some(1),
    };

    let env = EnvBuilder::new()
        .with_task(schedule)
        .with_strategy_module(Heartbeat::new())
        .build()?;

    env.execute().await;
    Ok(())
}

§Downstream usage shape

Real strategy binaries usually follow the same shape as the minimal example: build AltTaskInfo and WsTaskInfo values from config, register one or more strategy modules, and call EnvMediator::execute. A simple signal process may only need scheduler and intent tasks; an execution orchestrator typically adds private account websocket tasks, order execution tasks, risk/evaluation modules, and several independently bound strategy modules.

§Extrema Infra Usage Guide

This guide covers common runtime wiring for strategy modules, task-local broadcast rings, task bindings, and command handles.

§Runtime Model

An application usually has four layers:

  1. Strategy modules implement business logic.
  2. Tasks own long-running work such as timers, model workers, order-execution relays, and websocket relays.
  3. Task-local broadcast rings carry task output to strategies. Every concrete TaskKey owns one ring, and that task’s lifecycle event and primary events travel together.
  4. Command handles let strategies send commands back to tasks after the runtime has prepared them.

The final binary wires those pieces together with EnvBuilder:

use std::{sync::Arc, time::Duration};

use extrema_infra::prelude::*;

#[derive(Clone)]
struct StrategyModule {
    registry: Arc<CommandRegistry>,
}

impl StrategyModule {
    fn new() -> Self {
        Self {
            registry: Arc::new(CommandRegistry::default()),
        }
    }
}

impl Strategy for StrategyModule {
    async fn initialize(&mut self) {
        // Load config, initialize API clients, warm caches, etc.
    }
}

impl CommandEmitter for StrategyModule {
    fn command_init(&mut self, registry: Arc<CommandRegistry>) {
        self.registry = registry;
    }

    fn command_registry(&self) -> Arc<CommandRegistry> {
        self.registry.clone()
    }
}

impl EventHandler for StrategyModule {
    async fn on_schedule(&mut self, msg: InfraMsg<AltScheduleEvent>) {
        println!("schedule tick: task_id={}", msg.task_id);
    }
}

#[tokio::main]
async fn main() -> InfraResult<()> {
    let schedule_task = AltTaskInfo {
        alt_task_type: AltTaskType::TimeScheduler(Duration::from_secs(30)),
        chunk: 1,
        task_base_id: Some(1),
    };

    let env = EnvBuilder::new()
        .with_task(schedule_task)
        .with_strategy_module(StrategyModule::new())
        .build()?;

    env.execute().await;
    Ok(())
}

EnvBuilder::build() validates task identities, explicit bindings and websocket decoders (unique decoder ids, one registered decoder for every Market::Custom task), creates the task rings, and returns InfraResult<EnvMediator<_, _>>. The with_strategy_module call above subscribes the module to every registered task.

§Prerequisites

Strategy binaries should declare their direct runtime dependencies. Do not rely on transitive dependencies from extrema_infra when using tokio, rustls, or logging crates in your own code. For process-wide TLS provider initialization, see TLS Setup.

[dependencies]
# Local workspace development:
extrema_infra = { path = "../extrema_infra" }
tokio = { version = "1.53", features = ["full"] }
rustls = { version = "0.23", features = ["aws-lc-rs"] }
tracing = "0.1"
tracing-subscriber = "0.3"

Exchange clients and websocket routing are controlled by Cargo features. Enable only the markets used by the binary:

extrema_infra = { path = "../extrema_infra", features = ["binance", "okx"] }

Use features = ["lob_clients"] for the LobClients aggregate helper; it also enables all four exchange features (binance, okx, gate, hyperliquid). Custom venues (Market::Custom with with_ws_decoder) need no exchange feature. Use features = ["model_zmq"] or features = ["model_onnx"] for model prediction task variants; features = ["model_runner"] enables both. Use features = ["polars"] only when downstream code needs the Polars error conversion. Use features = ["all"] for every exchange module, LobClients, both model runners, and Polars support.

§Strategy Module Checklist

Every strategy module should implement three traits:

use std::sync::Arc;

use extrema_infra::prelude::*;

#[derive(Clone)]
struct MyModule {
    registry: Arc<CommandRegistry>,
}

impl Strategy for MyModule {
    async fn initialize(&mut self) {}
}

impl CommandEmitter for MyModule {
    fn command_init(&mut self, registry: Arc<CommandRegistry>) {
        self.registry = registry;
    }

    fn command_registry(&self) -> Arc<CommandRegistry> {
        self.registry.clone()
    }
}

impl EventHandler for MyModule {}

Use initialize for startup-only work. Use command_init only to store the runtime-provided command registry. Implement only the event callbacks your module needs; all other callbacks default to no-op.

Strategy modules receive every registered task by default. See Task Bindings for explicit per-task routing.

§Scheduler and Intent Tasks

AltTaskInfo is used for non-websocket runtime tasks:

ⓘ
use std::time::Duration;

use extrema_infra::prelude::*;

let schedule_task = AltTaskInfo {
    alt_task_type: AltTaskType::TimeScheduler(Duration::from_secs(60)),
    chunk: 1,
    task_base_id: Some(10),
};

let order_execution_task = AltTaskInfo {
    alt_task_type: AltTaskType::OrderExecution,
    chunk: 1,
    task_base_id: Some(20),
};

let env = EnvBuilder::new()
    .with_task(schedule_task)
    .with_task(order_execution_task);

Common AltTaskType values:

  • TimeScheduler(Duration): the duration must be greater than zero. After the task’s approximately five-second startup delay, the first on_schedule callback is immediate; later callbacks use the configured duration.
  • InstIntent: instrument or portfolio target intents delivered to on_inst_intent.
  • OrderExecution: relays order batches to on_order_execution; the receiving application module implements actual exchange submission.
  • ModelPreds(ModelRunner::Zmq(..)): external model process integration. Enable model_zmq, model_runner, or all, to make this variant available.
  • ModelPreds(ModelRunner::Onnx(..)): in-process ONNX inference. Enable model_onnx, model_runner, or all, to make this variant available.

Scheduler tasks publish to on_schedule, intent tasks to on_inst_intent, order-execution relay tasks to on_order_execution, and model prediction tasks to on_preds.

§Public Websocket Task

A public market-data strategy receives a WsTaskInfo event before each connection or reconnection cycle, uses the command handle to connect and subscribe, then consumes normalized events such as trades or candles. The handler must be safe to call repeatedly and must repeat the full initialization sequence each time. LOB updates are available only for exchange relays that implement WsChannel::Lob routing.

ⓘ
use extrema_infra::prelude::*;

const TASK_ID: u64 = 2001;

let trades_task = WsTaskInfo {
    market: Market::BinanceUmFutures,
    ws_channel: WsChannel::Trades(Some(TradesParam::AggTrades)),
    filter_channels: false,
    chunk: 1,
    task_base_id: Some(TASK_ID),
};

let env = EnvBuilder::new()
    .with_task(trades_task);

The corresponding strategy callbacks are:

use extrema_infra::prelude::*;

#[derive(Clone)]
struct MyPublicWsModule;

impl EventHandler for MyPublicWsModule {
    async fn on_ws_event(&mut self, msg: InfraMsg<WsTaskInfo>) {
        // Find the Ws handle, connect, and subscribe.
        let _ = msg;
    }

    async fn on_trade(&mut self, msg: InfraMsg<Vec<WsTrade>>) {
        // Consume normalized trade batches.
        let _ = msg;
    }
}

The strategy owns the connect and subscribe sequence. The Binance UM example below uses its URL-only target with TaskCommand::WsConnect, then sends exchange login/subscription messages as needed:

ⓘ
async fn on_ws_event(&mut self, msg: InfraMsg<WsTaskInfo>) {
    if msg.task_id != TASK_ID {
        return;
    }

    let Some(handle) = self.find_ws_handle(&msg.data.ws_channel, msg.task_id) else {
        return;
    };

    let Ok(ws_url) = exchange_client
        .get_public_connect_msg(&msg.data.ws_channel)
        .await
    else {
        return;
    };

    let (tx, rx) = tokio::sync::oneshot::channel();
    if handle
        .send_command(
            TaskCommand::WsConnect {
                msg: ws_url,
                ack: AckHandle::new(tx),
            },
            Some((AckStatus::WsConnect, rx)),
        )
        .await
        .is_err()
    {
        return;
    }

    let Ok(sub_msg) = exchange_client
        .get_public_sub_msg(&msg.data.ws_channel, Some(&insts))
        .await
    else {
        return;
    };

    let _ = handle
        .send_command(
            TaskCommand::WsMessage {
                msg: sub_msg,
                ack: AckHandle::none(),
            },
            None,
        )
        .await;
}

URL-only clients use get_*_connect_msg with TaskCommand::WsConnect. Clients that require HTTP upgrade metadata, currently including Gate Futures, use get_*_connect_target with TaskCommand::WsConnectWithTarget; the string form cannot carry headers.

§Private Account Websocket Task

Private account streams use the same task model, but publish account-specific payloads:

ⓘ
use extrema_infra::prelude::*;

let positions_task = WsTaskInfo {
    market: Market::Okx,
    ws_channel: WsChannel::AccountPositions,
    filter_channels: false,
    chunk: 1,
    task_base_id: Some(3001),
};

let env = EnvBuilder::new()
    .with_task(positions_task);

Useful callbacks:

  • on_acc_order: private order updates.
  • on_acc_bal_pos: balance and position updates.
  • on_acc_pos: position-only updates.
  • on_ws_other: raw JSON frames from exchange-specific WsChannel::Other(...) tasks.
  • on_lagged: the module fell behind on a task stream and lost events; resync account state over REST (see Task Bindings).

The relay preserves the complete top-level JSON frame and a local receive timestamp. Exchange-specific code can then decode fields such as Hyperliquid’s channel: "post" and data.id; the relay-level AckStatus::WsMessage still means only that the text frame was written locally.

Exchange clients normally need API-key initialization in Strategy::initialize before private websocket login messages are built. Credentials and login flows are exchange-specific; for example, OKX uses a concrete login-message helper, Binance UM/CM futures private streams require listen-key management and periodic renewal, and Binance Spot private streams use the WS API signed subscription helper.

Built-in private clients read credentials from the process environment or a .env file:

ExchangeRequired variables
BinanceBINANCE_API_KEY, BINANCE_SECRET_KEY
OKXOKX_API_KEY, OKX_SECRET_KEY, OKX_PASSPHRASE
GateGATE_API_KEY, GATE_SECRET_KEY, GATE_USER_ID
HyperliquidHYPERLIQUID_OWNER_ADDRESS, HYPERLIQUID_AGENT_PRIVATE_KEY; optional HYPERLIQUID_VAULT_ADDRESS, and HYPERLIQUID_WITHDRAW_PRIVATE_KEY (owner key, required only for withdrawals and sendToEvmWithData)

§Multiple Strategy Modules

Large binaries can register several independent strategy modules in one static runtime. A portfolio process, for example, can combine signal generation, weight allocation, account-state mediation, order execution, and evaluation modules:

ⓘ
use extrema_infra::prelude::*;

// These are your concrete strategy modules. Each one implements Strategy,
// CommandEmitter, EventHandler, and Clone.
let signal_module = build_signal_module();
let allocator_module = build_allocator_module();
let account_state_module = build_account_state_module();
let order_executor_module = build_order_executor_module();
let runtime_tasks = build_runtime_tasks();

let env = EnvBuilder::new()
    .with_tasks(runtime_tasks)
    .with_strategy_module(signal_module)
    .with_strategy_module(allocator_module)
    .with_strategy_module(account_state_module)
    .with_strategy_module(order_executor_module)
    .build()?;

EnvBuilder stores strategy modules in a heterogeneous list. This keeps each module as its concrete type instead of forcing all modules into Box<dyn Strategy>. Each module has its own event loop. In the simple form above, every module subscribes to every task. Use explicit bindings for larger runtimes that partition feeds across modules.

§Task Bindings

Every concrete runtime task owns one broadcast ring. A TaskInfo declaration with chunk = n expands to n TaskKey values and n independent rings, so traffic or lag on one publisher does not write into another publisher’s ring. A selected ring carries both that task’s lifecycle event and primary events.

with_strategy_module and with_strategy_modules subscribe to every task. Their _on variants accept explicit keys and avoid receiver creation and wakeups for unrelated tasks. TaskInfo::task_keys() expands a declaration’s whole chunk into the keys accepted by those methods.

For example, 100 one-task trade declarations can be split across 20 same-type signal strategies, five task streams per strategy:

ⓘ
let trade_tasks: Vec<TaskInfo> = build_100_trade_tasks();

let bound_signal_modules = (0..20)
    .map(|partition| -> InfraResult<_> {
        let task_keys = trade_tasks[partition * 5..(partition + 1) * 5]
            .iter()
            .map(TaskInfo::task_keys)
            .collect::<InfraResult<Vec<_>>>()?
            .into_iter()
            .flatten()
            .collect();
        Ok((build_signal_strategy(partition), task_keys))
    })
    .collect::<InfraResult<Vec<_>>>()?;

let env = EnvBuilder::new()
    .with_tasks(trade_tasks)
    .with_strategy_modules_on(bound_signal_modules)
    .build()?;

Each pair passed to with_strategy_modules_on gets an independent strategy handler loop and only its five receivers. For different strategy Rust types, call with_strategy_module_on once per module.

The event side supports fan-out: several strategies may bind to the same websocket task and receive its lifecycle and primary events. The command side still needs one owner. Designate exactly one strategy to send that task’s connect, login, and subscribe sequence; the remaining consumers must not send duplicate startup commands when they receive on_ws_event.

Every spawned task receives a task_id:

  • If task_base_id is Some(base) and chunk = n, generated task IDs are base, base + 1, …, base + n - 1.
  • If task_base_id is None, generated task IDs start from 1 for that task declaration.

Use stable task IDs when a strategy must route events or command handles by market, account, channel, or model worker.

Both task rings and command handles use TaskKey. TaskKey::Alt stores the complete AltTaskType plus task_id, while TaskKey::Ws stores the complete WsChannel plus task_id. Embedded parameters are part of identity: for example, Trades(AggTrades) and Trades(AllTrades) produce different routing keys, as do schedulers with different durations. Websocket market, chunk, and task_base_id are not part of the key.

The InfraMsg callback envelope carries task_id, not the full TaskKey. EnvBuilder::build() therefore rejects a duplicate task id for the same task type, ignoring embedded parameters: two Trades channels or two schedulers cannot share an id even though their full keys differ. Different task types, such as Trade and LOB, may reuse an id. This check runs during build, not on the message path. Build also rejects an explicit binding to an unregistered key.

Ring capacity is fixed per event type: Schedule 1,024; WS lifecycle, InstIntent, Candle and Other 2,048; Trade, account streams, OrderExecute and ModelPreds 8,192; Lob 16,384; LobMbo 65,536. Total reserved slots therefore scale with publisher count, not receiver count: 100 Trade tasks reserve 819,200 ring slots. Explicit bindings reduce receivers and wakeups, but do not reduce publisher ring capacity.

Delivery is lossy by design. A module that falls behind a full ring loses the oldest events instead of stalling the publisher; other tasks’ rings are unaffected. The runtime then calls EventHandler::on_lagged(key, skipped), coalesced per task stream to at most one call per second, with skipped the total dropped since the previous notice. A drop inside the one-second window is reported only with the next lag on that stream. Treat the notice as a signal to resync, for example by re-reading positions and open orders over REST.

§Custom Venue Websocket Decoder

A venue implemented outside this crate can run on the built-in websocket relay. Implement LobWsDecoder, register it with EnvBuilder::with_ws_decoder, and declare its tasks on Market::Custom(MyVenueWs::ID). No exchange feature is needed.

use extrema_infra::prelude::*;

#[derive(Clone)]
struct MyVenueWs;

impl LobWsDecoder for MyVenueWs {
    const ID: u16 = 42;
    const NAME: &'static str = "my_venue";

    async fn ws_channel<R: WsFrameRunner>(&self, channel: &WsChannel, runner: R) {
        match channel {
            WsChannel::Lob(_) => runner.ws_loop(TaskEvent::Lob, decode_bbo).await,
            WsChannel::Other(_) => runner.ws_loop(TaskEvent::WsOther, decode_raw_ws).await,
            _ => {},
        }
    }
}

let lob_task = WsTaskInfo {
    market: Market::Custom(MyVenueWs::ID),
    ws_channel: WsChannel::Lob(None),
    filter_channels: false,
    chunk: 1,
    task_base_id: Some(1),
};

let env = EnvBuilder::new()
    .with_ws_decoder(MyVenueWs)
    .with_task(lob_task)
    .with_strategy_module(MyStrategy)
    .build()?;
  • ws_channel runs once per connection. It picks a decode function for the task’s channel and passes it to WsFrameRunner::ws_loop with the matching TaskEvent constructor, which decides the callback (TaskEvent::Lob -> on_lob, TaskEvent::WsOther -> on_ws_other, and so on). decode_raw_ws forwards complete frames to on_ws_other.
  • The strategy still connects, authenticates and subscribes in on_ws_event, as with built-in venues. After a disconnect the relay waits 5 s and emits on_ws_event again.
  • The relay sends a websocket Ping frame after 10 s without an inbound frame. Application-level pings must be sent with TaskCommand::WsMessage.
  • Do not declare tasks on channels the decoder ignores: returning without calling the runner closes the connection, and the task reconnects about every 5 s.

tests/custom_ws_decoder.rs is a complete worked example against a local websocket server.

§TLS Setup

When exactly one built-in provider feature is enabled, rustls 0.23 can select it automatically. A final binary can still install a provider explicitly before creating REST or websocket clients to pin the process-wide choice:

rustls::crypto::aws_lc_rs::default_provider()
    .install_default()
    .expect("failed to install rustls crypto provider");

Explicit selection is necessary when the dependency graph enables zero or multiple built-in providers, or when the application uses a custom provider. It belongs in the final binary, not library code, because rustls allows only one process-wide default provider.

§Reference Patterns

Downstream repositories currently exercise these patterns:

  • funding_carry: a single strategy module with scheduler, instrument intent, and state-save tasks.
  • portfolio_orchestrator: several strategy modules in one runtime, including signal allocation, portfolio mediation, order execution, transfer ticks, account websocket streams, and evaluation tasks.
  • api_checkers: small exchange-focused subcommands that demonstrate REST calls, public websocket streams, and private account websocket streams.
  • tests/custom_ws_decoder.rs: a custom venue on the built-in websocket relay.

The examples need the features listed in Cargo.toml, so a bare cargo build --examples only builds empty_strategy_example:

ExampleCommand
The smallest scheduler examplecargo run --example empty_strategy_example
Multiple strategy modules in one runtimecargo run --example multi_strategy_example --features binance
Private account websocket setup, including Binance listen-key renewalcargo run --example websocket_private_account_example --features binance,okx
Read-only Hyperliquid REST usage, including optional balance/position reads by owner addresscargo run --example hyperliquid_api_usage_example --features hyperliquid
OKX trades, account orders, two ZMQ model tasks and an OrderExecution relay with explicit bindingscargo run --example complex_strategy_example --features okx,model_zmq

Modules§

arch
errors
Error and result types used across the crate.
prelude
Common imports for strategy binaries.