fv-streams-sql
The SQL front door for FusionVault Streams.
A streaming SQL statement parses to a small typed logical plan and lowers to the same
topology the TOML frontend produces (its steps are the engine's own JSON vocabulary). One runtime,
two front doors: SQL inherits checkpointing, the exchange, exactness and --join because it emits
the identical steps a hand-written pipeline would.
let topo = compile?;
Design
- No DataFusion
LogicalPlan/ physical layer, no fork. We lower thesqlparserAST to our own IR, which models streaming (DataFusion's logical plan does not); the errors are streaming-aware and ours, and the IR is the seam a future optimiser plugs into. (Seedesign/…-phase5-…§10.) - Expressions reuse the value dialect. SQL scalar expressions unparse to the value-dialect
strings the
expressionsteps already carry, compiled throughfv-value-datafusionat build time exactly as the TOML path does — one expression layer, so the two front doors cannot drift.
The supported SQL
Every statement form, what each one becomes, how expressions translate, and what is refused (with the
message): docs/reference/sql.md,
generated from fv_streams_sql::reference — each example on it is compiled to write it, and a test
fails when an example stops compiling to the steps its form promises.
Multi-stage worked examples (each e2e-exact against its TOML twin's oracle): q13 — WITH bids AS (SELECT *, auction % N AS key … WHERE type='bid') SELECT * FROM bids JOIN side ON bids.key = side.key
(a computed key, then the lookup join); q5 — a sliding windowed count CTE then a per-window
ROW_NUMBER() <= 1; q4 — four CTEs (two filtered streams, their interval join, a per-auction
max) then an avg per category, five stages with two nested windowed aggregates.
In a pipeline
A pipeline config declares its datasets as usual and puts the query in a sql field instead of
[[transforms]]:
= "enrich"
= "INSERT INTO out SELECT auction, price * 0.908 AS eur FROM bids WHERE type = 'bid'"
[]
= "localhost:9092"
[]
= "bids"
[]
= "out"