knut-bifrost 0.2.0

Bifröst — watch lakehouse tables (Iceberg via Spark Connect) and materialise them as a FalkorDB property graph
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

//! Sinks: the two places a bifröst round can land.
//!
//! There are two families, and they are siblings rather than one generalised
//! trait. The reason is structural, not stylistic:
//!
//! * [`graph::GraphSink`] takes a [`GraphMutation`](crate::mutation::GraphMutation),
//!   every field of which is a label, an edge type, an id column or an edge
//!   merge-key. Its report counts vertices and edges.
//! * [`table::TableSink`] takes a [`table::TableWindow`] — a snapshot pair plus
//!   already-split upsert/delete row batches. It has no labels and no edges.
//!
//! Folding those into one trait would produce a type whose majority of fields
//! are meaningless to whichever caller you are. So: one file per family, and
//! `mod.rs` re-exports so `crate::sink::GraphSink` / `crate::sink::FalkorDbSink`
//! keep resolving exactly as before the split.
//!
//! See `.nornir/doris-sink-design.md` f2.

pub mod graph;
pub mod table;

/// The Doris [`table::TableSink`] implementation. Off by default; enabling
/// `doris` adds only `reqwest`, which the `airflow` feature already pulls.
#[cfg(feature = "doris")]
pub mod doris;

pub use graph::{FalkorDbSink, GraphSink};
pub use table::{
    AckToken, LoadAck, SnapshotWindow, TableSink, TableWindow, deliver_window,
    is_valid_doris_label, refuse_full_reload, stream_load_label,
};

#[cfg(feature = "doris")]
pub use doris::{
    DorisConfig, DorisSink, LoadFormat, LoadMode, ReqwestTransport, StreamLoadRequest,
    StreamLoadResponse, StreamLoadTransport,
};

// The Cypher row renderer stays crate-internal but must keep its old path
// (`crate::sink::rows_as_cypher_maps`) — `src/tests.rs` calls it.
#[cfg(test)]
pub(crate) use graph::rows_as_cypher_maps;