Expand description
§fv-streams-sql — the SQL front door
A streaming SQL statement parses to a small typed logical plan (plan::Plan) and lowers to
the same Topology of StageDefs the TOML frontend produces (whose steps are the engine’s
own JSON vocabulary). One runtime, two front doors: SQL inherits checkpointing, the exchange,
exactness and --join because it emits the identical steps a hand-written pipeline would.
Deliberately not DataFusion’s LogicalPlan/physical layer (design §10): our own IR models
streaming (DataFusion’s does not), so there is no fork risk, the errors are streaming-aware and
ours, and the IR is the seam a future optimiser plugs into. Scalar expressions lower to the value
dialect strings the expression steps already carry, compiled through fv-value-datafusion at
build time exactly as the TOML path does.
let topo = fv_streams_sql::compile(
"INSERT INTO out SELECT auction, price * 0.908 AS eur FROM bids WHERE type = 'bid'",
).unwrap();
assert_eq!(topo.stages.len(), 1);Modules§
- reference
- The SQL reference: every statement form the front door compiles, every expression it translates,
and every form it refuses — each described once here, with an example that is compiled to write
docs/reference/sql.md(what a form becomes is the compiler’s own output, and a refusal’s message is the compiler’s own words). A test fails when the page is stale, when an example stops compiling to the steps its form promises, or when a refused statement starts compiling.
Enums§
- Plan
- The typed logical plan. 5c-i covers the stateless shape (source → filter → project → sink); 5c-ii adds the windowed aggregate, the updating aggregate, and the joins.
Functions§
- compile
- Compile one streaming SQL statement into a
Topology. Datasets (theFROM/INSERT INTOtargets) are referenced by name; their bindings (topic, connector, schema) stay in the pipeline config, exactly as the TOML frontend resolves them. Fails LOUDLY with a streaming-aware message on anything outside the supported subset.