Skip to main content

Module serial

Module serial 

Source
Expand description

Plans that run on one task, for the browser (#278).

DataFusion runs each input partition of a CoalescePartitionsExec as its own Tokio task, and so does execute_stream over a plan with several partitions. The browser build has no Tokio runtime: its futures are driven by JavaScript’s event loop, so a spawn panics (“there is no reactor running”). target_partitions = 1 keeps scans and joins to one partition, but a UNION still has one partition per input, and a graph’s nodes table is a union.

SingleTask is a physical optimizer rule, run last in the browser build. It replaces each coalesce over several partitions, and a plan whose output has several partitions, with SerialCoalesceExec, which reads the partitions one after another on the task that polls it. A coalesce returns rows in no fixed order (whichever partition is ready first), so reading them in partition order is one of the orders it may produce. Graph inputs from such plans are sorted by KeepEdgeOrder before the index is built, so graph results do not depend on it either.

Structs§

SerialCoalesceExec
One partition holding every row of its input’s partitions, read in partition order on the polling task, with an optional row limit.
SingleTask
The rule that makes every plan run on one task. See the module docs.