# fv-streams-sql
The SQL front door of [FusionVault Streams](https://github.com/FusionVault/streams).
It compiles a streaming SQL statement to the same stages and steps that a TOML pipeline describes.
The engine runs them the same way, so SQL gets checkpoints, exactly-once output and scale-out with
no extra cost.
```rust
let topology = fv_streams_sql::compile(
"INSERT INTO out SELECT auction, price * 0.908 AS eur FROM bids WHERE type = 'bid'",
)?;
```
## In a pipeline
A pipeline file declares its datasets as usual, and puts the statement in `sql` instead of
`[[transforms]]`:
```toml
name = "enrich"
sql = "INSERT INTO out SELECT auction, price * 0.908 AS eur FROM bids WHERE type = 'bid'"
[brokers]
default = "localhost:9092"
[datasets.bids]
topic = "bids"
[datasets.out]
topic = "out"
```
## What SQL it supports
The [SQL reference](https://github.com/FusionVault/streams/blob/main/docs/reference/sql.md) lists
every statement form, what each one becomes, how expressions translate, and what is refused, with
the message.
It is generated from `fv_streams_sql::reference`.
Each example on it is compiled to write the page, and a test fails if one stops compiling to the
steps its form promises.
Some multi-stage statements, each checked against the same query written in TOML:
- **q13** computes a key in a `WITH` clause, then joins it to a lookup table.
- **q5** counts in sliding windows in a `WITH` clause, then keeps the top auction per window with
`ROW_NUMBER() <= 1`.
- **q4** uses four `WITH` clauses: two filtered streams, their interval join, and a `max` per
auction. It then takes an `avg` per category. That is five stages, with two windowed aggregates
one after the other.
## How it works
- **Its own plan, not a database's.** The statement is parsed with `sqlparser`, then turned into a
small typed plan that models streaming. There is no DataFusion logical plan and no fork. The
errors speak about streams, and the plan is where an optimiser can plug in later.
- **One expression language.** SQL expressions become the same expression strings the TOML steps
use, compiled through `fv-value-datafusion` in the same way. So the two front doors cannot drift
apart.
## Licence
Apache 2.0.