Skip to main content

Module stream

Module stream 

Source
Available on crate features source-duckdb and source-rest only.
Expand description

DuckDB source implementation โ€” the one module that performs I/O.

duckdb is a synchronous, embedded engine, so every database call runs inside tokio::task::spawn_blocking. Streaming stays bounded-memory: a dedicated blocking task holds the connection, iterates the result row by row, and hands finished StreamPages to the async side over a small bounded channel โ€” never buffering the whole result set.

Structsยง

DuckdbSource
A source that executes a SQL query against DuckDB and returns rows as JSON.