Expand description
Plans that run on one task, for the browser (#278).
DataFusion runs each input partition of a CoalescePartitionsExec as its
own Tokio task, and so does execute_stream over a plan with several
partitions. The browser build has no Tokio runtime: its futures are driven
by JavaScript’s event loop, so a spawn panics (“there is no reactor
running”). target_partitions = 1 keeps scans and joins to one partition,
but a UNION still has one partition per input, and a graph’s nodes
table is a union.
SingleTask is a physical optimizer rule, run last in the browser build.
It replaces each coalesce over several partitions, and a plan whose output
has several partitions, with SerialCoalesceExec, which reads the
partitions one after another on the task that polls it. A coalesce returns
rows in no fixed order (whichever partition is ready first), so reading
them in partition order is one of the orders it may produce. Graph inputs
from such plans are sorted by KeepEdgeOrder before the index is built, so
graph results do not depend on it either.
Structs§
- Serial
Coalesce Exec - One partition holding every row of its input’s partitions, read in partition order on the polling task, with an optional row limit.
- Single
Task - The rule that makes every plan run on one task. See the module docs.