Skip to main content

Crate fv_streams_sql

Crate fv_streams_sql 

Source
Expand description

§fv-streams-sql — the SQL front door

A streaming SQL statement parses to a small typed logical plan (plan::Plan) and lowers to the same Topology of StageDefs the TOML frontend produces (whose steps are the engine’s own JSON vocabulary). One runtime, two front doors: SQL inherits checkpointing, the exchange, exactness and --join because it emits the identical steps a hand-written pipeline would.

Deliberately not DataFusion’s LogicalPlan/physical layer (design §10): our own IR models streaming (DataFusion’s does not), so there is no fork risk, the errors are streaming-aware and ours, and the IR is the seam a future optimiser plugs into. Scalar expressions lower to the value dialect strings the expression steps already carry, compiled through fv-value-datafusion at build time exactly as the TOML path does.

let topo = fv_streams_sql::compile(
    "INSERT INTO out SELECT auction, price * 0.908 AS eur FROM bids WHERE type = 'bid'",
).unwrap();
assert_eq!(topo.stages.len(), 1);

Modules§

reference
The SQL reference: every statement form the front door compiles, every expression it translates, and every form it refuses — each described once here, with an example that is compiled to write docs/reference/sql.md (what a form becomes is the compiler’s own output, and a refusal’s message is the compiler’s own words). A test fails when the page is stale, when an example stops compiling to the steps its form promises, or when a refused statement starts compiling.

Enums§

Plan
The typed logical plan. 5c-i covers the stateless shape (source → filter → project → sink); 5c-ii adds the windowed aggregate, the updating aggregate, and the joins.

Functions§

compile
Compile one streaming SQL statement into a Topology. Datasets (the FROM/INSERT INTO targets) are referenced by name; their bindings (topic, connector, schema) stay in the pipeline config, exactly as the TOML frontend resolves them. Fails LOUDLY with a streaming-aware message on anything outside the supported subset.