knut-thund 0.1.1

Þund — a Rust-native, Arrow-centric streaming dataflow engine (batch + streaming) with a pluggable execution backend: native Arrow/DataFusion or lower-to-Spark-Declarative-Pipelines via Spark Connect. The 'Airflow killer' authoring+runtime for knut.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

//! Authoring surfaces — the three front-ends that all lower to one
//! [`crate::ir::Pipeline`].
//!
//! Þund's design decision (see `.nornir/thund.md`) is **all-three-over-one-IR**:
//!
//! 1. **Typed Rust builder** — the [`crate::ir`] types *are* the builder:
//!    `Pipeline::new(..).with_dataset(..).with_flow(..)`. Compile-time typed,
//!    testable, the canonical form. No separate builder type is needed because
//!    the IR's `with_*` chaining already is one.
//! 2. **Thin declarative DSL** ([`dsl`], feature `dsl`) — a RON serialization
//!    of the IR for config-as-code / LLM-authoring / warehouse-historized
//!    defs. It is a *pure (de)serialization* of the same IR, never a second
//!    model, so there is no drift.
//! 3. **facett graph editor** — the visual node-graph editor reuses the
//!    existing `knut-pipelines` facett `PipelineView`; it edits the IR's
//!    `(dataset, flow)` graph directly and emits the same [`crate::ir::Pipeline`].
//!
//! Because all three produce the identical IR, a pipeline can be drawn in
//! facett, exported to RON, hand-tuned in Rust, and run on either backend —
//! the IR is the single contract.

#[cfg(feature = "dsl")]
pub mod dsl;