Skip to main content

Module streaming_window_plan

Module streaming_window_plan 

Source
Expand description

Compile a windowed streaming SQL query into a WindowExecutionSpec.

Supports the canonical keyed windowed-aggregation shape:

SELECT key, AGG(col) AS out [, ...]
FROM TUMBLE(TABLE src, DESCRIPTOR(ts), <size>)   -- or HOP / SESSION
GROUP BY key, window_start, window_end

§One SQL front door (Phase 60)

Streaming and batch share one front door for the window TVF and one parse of the query:

  1. The window TVF is rewritten to a subquery exactly once by rewrite_window_tvfs (streaming_tvf.rs) — the same rewrite the batch planner consumes, so TUMBLE/HOP/SESSION has a single canonical lowering, not one for batch and another hand-rolled for streaming.
  2. The rewritten SQL is parsed once with the front-door dialect (DuckDbDialect, matching SqlEngine’s sql_parser.dialect). Parsing streaming SQL with a different dialect than batch was the divergence class behind the SUM(CASE WHEN …) 409 in prod; a single dialect closes it structurally.
  3. Everything the operator needs — the window kind/size/slide/gap, the event time column, the grouping key, and the aggregate list — is derived from that one parsed plan. Window recognition is now structural: “does the parsed plan carry a window_start boundary projection over a recognised window function?”, so SqlError::Unsupported means “the planner cannot lower this shape to a continuous plan”, not “a text matcher failed to recognise the SQL”.

The dataflow ContinuousWindowExecutor still computes the aggregation from the resulting WindowExecutionSpec; consuming DataFusion’s own LogicalPlan (rather than the shared sqlparser AST) is the deeper unification that grows with Phase 55’s operator coverage.

Structs§

StreamingWindowPlan
A compiled windowed streaming plan: the operator spec plus the name of the source table the window reads from.

Functions§

compile_streaming_window_sql
Compile a windowed streaming SQL query into a StreamingWindowPlan.
is_windowed_streaming_sql
Returns true when sql contains a TUMBLE/HOP/SESSION window TVF.