Crate pikmin

An extensible downloader for obtaining trade data simultaneously from exchanges' API.

pikmin is a trade (execution) data downloader for crypto-currency exchanges, such as BitMex, bitFlyer, Liquid, etc. This library provides not only some pre-composed downloaders, but also ability to build a custom downloader for users' demand.

A downloader is composed of ID and Writer. An ID specifies the point where to start downloading, and a Writer outputs trade data to external storage.

There are some pre-composed downloaders in downloader module.

Example

A simple downloader for Liquid with writing to stdout. This program creates /tmp/qn-progress.txt for recording progress, so delete it if you want to run again from the starting point.

use std::path::Path;
use std::thread;
use std::time::Duration;

use chrono::offset::TimeZone;
use chrono::Utc;

use pikmin::{LiquidDownloader, StdOutWriter};
use pikmin::downloader::Downloader;

fn main() {
    // by using thread, you can run multiple downloaders
    let liquid = thread::spawn(move || {
        while {
            // download data from 2019-01-01T01:01:01Z to 2019-01-01T01:03:01Z
            // output the downloaded data to standard out (println!)
            let downloader = LiquidDownloader::new(
                Utc.ymd(2019, 1, 1).and_hms(1, 1, 1),
                Utc.ymd(2019, 1, 1).and_hms(1, 3, 1),
            );

            // Locate progress file to `/tmp/qn-progress.txt`.
            // You can control the starting point of downloading
            // by preparing this file before you run.
            let progress_file = Path::new("/tmp/qn-progress.txt");

            // write out to standard out. simple writer for debugging
            let mut writer = StdOutWriter::default();

            println!("start QnDownloader");

            // run!
            match downloader.run(&mut writer, &progress_file) {
                Ok(_) => {
                    println!("finished");
                    false
                }
                Err(e) => {
                    println!("error: {}", e);
                    println!("retry...");
                    thread::sleep(Duration::from_secs(5));
                    true
                }
            }
        } {}
    });

    match liquid.join() {
        Ok(_) => println!("finish all"),
        Err(_) => println!("threading error"),
    }
}

Re-exports

pub use self::downloader::bf::BfDownloader;

pub use self::downloader::bitmex::MexDownloader;

pub use self::downloader::liquid::LiquidDownloader;

pub use self::writer::db_mysql::MySQLWriter;

pub use self::writer::stdout::StdOutWriter;

Modules

downloader	Composed downloaders.
error	Error utilities.
writer	Writers to output the trade data into external storage.