atelier_data 0.0.15

Data Artifacts and I/O for the atelier-rs engine
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

# atelier-data Examples

## Worker Examples

### data_worker / `run_data_worker`

Raw event ingestion via `DataWorker`. Connects to a live exchange WebSocket
feed, decodes events without synchronisation, and delivers them through
the configured output sinks.

```bash
cargo run -p atelier_data --example run_data_worker \
  -- --config atelier-data/examples/data_worker/data_worker_config.toml
```

### market_worker / `run_market_worker`

Synchronised market snapshots to Parquet via `MarketWorker`. Bins events
onto a time grid using `ClockMode::ExternalClock` and writes
`MarketSnapshot`s to per-datatype Parquet files.

```bash
cargo run -p atelier_data --example run_market_worker --features parquet \
  -- --config atelier-data/examples/market_worker/market_worker_config.toml
```

### market_worker / `read_market_worker`

Reads all Parquet files under a given directory, groups them by symbol
(parsed from the filename), and prints per-symbol statistics for all five
data types (orderbooks, trades, liquidations, funding, open interest).

```bash
cargo run -p atelier_data --example read_market_worker --features parquet \
  -- --dir datasets/collected/bybit
```

## Standalone Exchange Examples

### market_sync / `bybit_markets`

Config-driven Bybit market snapshot collection with OB + trades
synchronisation and Parquet persistence.

```bash
cargo run -p atelier_data --example bybit_markets --features parquet \
  -- --config examples/market_sync/bybit/bybit_config.toml
```

### market_sync / `coinbase_markets`

Config-driven Coinbase market snapshot collection (level2 + market_trades).

```bash
cargo run -p atelier_data --example coinbase_markets --features parquet \
  -- --config examples/market_sync/coinbase/coinbase_config.toml
```

### market_sync / `kraken_markets`

Config-driven Kraken market snapshot collection (book + trade).

```bash
cargo run -p atelier_data --example kraken_markets --features parquet \
  -- --config examples/market_sync/kraken/kraken_config.toml
```

### market_sync / `market_load`

Loads and verifies the most recent Parquet files for orderbooks and trades,
printing record counts, time spans, and spacing uniformity.

```bash
cargo run -p atelier_data --example market_load --features parquet \
  -- --config examples/market_sync/bybit/bybit_config.toml
```

## Raw Stream Examples

### market_strems / `market_fetch`

Multi-exchange raw market stream collector. Connects to Bybit, Coinbase,
and Kraken simultaneously, captures orderbook snapshots and trades without
synchronisation, and flushes to per-exchange per-symbol Parquet files.

```bash
cargo run -p atelier_data --example market_fetch --features parquet
```

## Other

### multi_sync / `multi_sync_workers`

Multi-worker manifest parser (stub). Parses a worker manifest TOML but
does not yet spawn workers.

```bash
cargo run -p atelier_data --example multi_sync_workers \
  -- --config examples/market_sync/bybit/bybit_workers.toml
```