Crate trade_aggregation

Trade Aggregation

A high performance, modular and flexible trade aggregation crate, producing Candle data, suitable for low-latency applications and incremental updates. It allows the user to choose the rule dictating how a new candle is created through the AggregationRule trait, e.g: Time, Volume based or some other information driven rule. It also allows the user to choose which type of candle will be created from the aggregation process through the ModularCandle trait. Combined with the Candle macro, it enables the user to flexibly create any type of Candle as long as each component implements the CandleComponent trait. The aggregation process is also generic over the type of input trade data as long as it implements the TakerTrade trait, allowing for greater flexibility for downstream projects.

See MathisWellmann/go_trade_aggregation for a go implementation with less features and performance.

Here is a sample trade series of Bitmex_XBTUSD aggregated into 15 minute candles:

Features:

AggregationRule:

The pre-existing rules in this crate include: ‘AggregationRule’ | Description ––––––––––|———–– TimeRule | Create candles every n seconds AlignedTimeRule | Same as TimeRule but candles are aligned to the start of a period VolumeRule | Create candles every n units traded TickRule | Create candles every n ticks RelativePriceRule | Create candles with every n basis points price movement (Renko)

If these don’t satisfy your desires, just create your own by implementing the AggregationRule trait, and you can plug and play it into the GenericAggregator.

CandleComponent:

These pre-existing ‘CandleComponents’ exist out of the box: ‘CandleComponent’ | Description ––––––––––| — Open | Price at the beginning of a candle High | Maximum price during the candle Low | Minimum price during the candle Close | Price at the end of a candle Volume | The cumulative trading volume NumTrades | The number of trades during the candle AveragePrice | The equally weighted average price WeightedPrice | The volume weighted price StdDevPrices | Keeps track of the standard deviation of prices StdDevSizes | Keeps track of the standard deviation of sizes TimeVelocity | Essentially how fast the candle was created time wise Entropy | Binary Shannon entropy using the trade side as inputs Trades | Just returns the observed trades during that candle

And again, if these don’t satisfy your needs, just bring your own by implementing the CandleComponent trait and you can plug them into your own candle struct.

How to use:

To use this crate in your project, add the following to your Cargo.toml:

[dependencies]
trade_aggregation = "^9"

Lets aggregate all trades into time based 1 minute candles, consisting of open, high, low and close information. Notice how easy it is to specify the ‘AggregationRule’ and ‘ModularCandle’ being used in the process. One can easily plug in their own implementation of those trait for full customization. This examples uses an online style approach to update with each tick:

use trade_aggregation::{
    candle_components::{Close, High, Low, Open},
    *,
};

#[derive(Debug, Default, Clone, Candle)]
struct MyCandle {
    open: Open,
    high: High,
    low: Low,
    close: Close,
}

fn main() {
    let trades = load_trades_from_csv("data/Bitmex_XBTUSD_1M.csv")
        .expect("Could not load trades from file!");

    // specify the aggregation rule to be time based and the resolution each trade timestamp has
    let time_rule = TimeRule::new(M1, TimestampResolution::Millisecond);
    // Notice how the aggregator is generic over the output candle type, 
    // the aggregation rule as well as the input trade data
    let mut aggregator = GenericAggregator::<MyCandle, TimeRule, Trade>::new(time_rule);

    for t in &trades {
        if let Some(candle) = aggregator.update(t) {
            println!(
                "candle created with open: {}, high: {}, low: {}, close: {}",
                candle.open(),
                candle.high(),
                candle.low(),
                candle.close()
            );
        }
    }
}

Notice how the code is calling the ‘open()’, ‘high()’, ‘low()’ and ‘close()’ methods on the ‘MyCandle’ struct. These are getters automatically generated by the Candle macro, that have the same name as the field. In this case the ‘Candle’ derive macro for ‘MyCandle’ expands into this:

use trade_aggregation::{
    candle_components::{Close, High, Low, Open},
    *,
};

#[derive(Debug, Default, Clone)]
struct MyCandle {
    open: Open,
    high: High,
    low: Low,
    close: Close,
}

impl MyCandle {
    fn open(&self) -> f64 {
        self.open.value()
    }
    fn high(&self) -> f64 {
        self.high.value()
    }
    fn low(&self) -> f64 {
        self.low.value()
    }
    fn close(&self) -> f64 {
        self.close.value()
    }
}

impl ModularCandle<Trade> for MyCandle {
    fn update(&mut self, trade: &Trade) {
        self.open.update(trade);
        self.high.update(trade);
        self.low.update(trade);
        self.close.update(trade);
    }
    fn reset(&mut self) {
        self.open.reset();
        self.high.reset();
        self.low.reset();
        self.close.reset();
    }
}

See examples folder for more. Run examples using

cargo run --release --example aggregate_all_ohlc
cargo run --release --example streaming_aggregate_ohlc

Performance:

To run the benchmarks, written using criterion, run:

cargo bench

Here are some results running on a 12th gen Intel Core i7-12800H, aggregating 1 million trades into 1 minute candles:

Candle	Time
Open	1 ms
OHLC	2.5 ms
All	8 ms

The more ’CandleComponent’s you use, the longer it takes obviously.

Features

The serde feature exists which, when enabled, derives Serialize and Deserialize

TODOs:

• Make generic over the data type storing the price (f64, f32, i64, Decimal, etc…)

Donations :moneybag: :money_with_wings:

I you would like to support the development of this crate, feel free to send over a donation:

Monero (XMR) address:

47xMvxNKsCKMt2owkDuN1Bci2KMiqGrAFCQFSLijWLs49ua67222Wu3LZryyopDVPYgYmAnYkSZSz9ZW2buaDwdyKTWGwwb

License

Copyright (C) 2020 <Mathis Wellmann wellmannmathis@gmail.com>

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Commercial License

If you’d like to use this crate legally without the restrictions of the GNU AGPLv3 license, please contact me so we can quickly arrange a custom license. This crate is used for aggregating raw trade data into candles using various methods

Re-exports

• pub use candle_components::CandleComponent;
• pub use candle_components::CandleComponentUpdate;

Modules

• candle_components
  This module contains a bunch of ready to use ‘CandleComponents’ that can easily be combined to create a ‘ModularCandle’ using the ‘Candle’ macro.

Structs

• AlignedTimeRule
  The classic time based aggregation rule, creating a new candle every n seconds. The time trigger is aligned such that the trigger points are starting from a time equals zero. For example, if the first tick comes in a 1:32:00 on a 5 minute candle, that first candle will only contain 3 minutes of trades, representing a 1:30 start.
• GenericAggregator
  An Aggregator that is generic over the type of Candle being produced, as well as by which rule the candle is created
• RelativePriceRule
  Creates Candles once the price changed by a give relative absolute price delta
• TickRule
  Creates candles every n ticks
• TimeRule
  The classic time based aggregation rule, creating a new candle every n seconds
• Trade
  Defines a taker trade
• VolumeRule
  Creates candles every n units of volume traded

Enums

• By
  Defines how to aggregate trade size either by Base currency or Quote Currency assumes trades sizes are denoted in Quote e.g.: buy 10 contracts of BTC would be trade size of 10
• Error
  Enumerate the possible errors in this crate
• TimestampResolution
  The resolution of the “TakerTrade” timestamps

Constants

• D1
  1 Day candle period
• H1
  1 Hour candle period
• H2
  2 Hour candle period
• H4
  4 Hour candle period
• H8
  8 Hour candle period
• H12
  12 Hour candle period
• M1
  1 Minute candle period
• M5
  5 Minute candle period
• M15
  15 Minute candle period
• M30
  30 Minute candle period

Traits

• AggregationRule
  Defines under what conditions one aggregation period is finished Is generic over the type of candle being produced C, as well as the type of input trade T
• Aggregator
  Defines the needed methods for any online Aggregator
• ModularCandle
  A modular candle that can be composed of multiple components Is generic over the type of trade it accepts during the update step, as long as it implements the TakerTrade trait
• TakerTrade
  Trait to enable third party types to be passed into aggregators.

Functions

• aggregate_all_trades
  Apply an aggregator for all trades at once
• candle_volume_from_time_period
  Determine the candle volume which produces the same number of candles as the given time aggregation equivalent
• load_trades_from_csv
  Load trades from csv file

Type Aliases

• Result
  Convenient wrapper for this crates custom Error

Derive Macros

• Candle
  The ‘Candle’ macro takes a named struct, that has multiple fields of type ‘CandleComponent’ to automatically generate a struct that implements the ‘ModularCandle’ trait, which means it can then be used in the aggregation process. It also exposes getter functions for each ‘CandleComponent’ for convenience.