1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
use ;
use Url;
use ;
;
/* DO:
will be split into Hot and Cold parts.
Cold can run on a different port, and even on a different machine. Hot needs to subscribe to Cold. // obviously will allow to have it spawn up Cold in an owned process if needed, without going through subscriptions layer
Hot will have an always up-to-date snapshot state + a way to wait for its update (tick)
Cold will directly own per-day parquet files with data per supported exchange (different files for different exchanges).
Has a method to request entire history for any Timestamp:Timestamp. When requesting, you can filter in/out any pattern like exchanges or depths; but when it's returned, everything is flat and ordered by creation time (NB: not arrival). Trades go in a separate line (cause storage and processing optimizations).
*/
//DO: need a daemon, subscribable over TCP // cause what if multiple things want same data + storage problems
/*DO:
/// want to have distinction over what
enum Data {
Snapshot,
Delta,
Trade,
/// Session close / trading halted randomly / delisted
Close,
}
*/
/// self-managed. One of the oh so few things allowed to have its proper tokio::spawn
//Q: wait, does it have to? If it writes to kernel buffer anyways, could we make it be pull-based as everything else is?
///// provides a channel to Cold. Its step is batch-reading from it
//struct BookHot {
// pub snapshot: UnsafeCell<BookShape>,
// cache: BTreeMap<Timestamp, BookChunk<Duration::from_minutes(15)>,, // needs to be persisting hot loaded areas of the map. In reality, BookHot itself should be a thin wrapper around BookCold I think. Only difference is we warm all the data contained + is always local.
//}
//impl BookHot {
// pub fn new(cold_source: ColdSource) -> Self {
// todo!();
// }
//}