Result of running a [TaskEstimator] on a leaf node. It tells the distributed planner hints
about how many tasks should be used in [Stage]s that contain leaf nodes.
Provides contextual information about where a WorkUnitFeedProvider is being executed. When
using WorkUnitFeedProvider in distributed queries, it might be getting executed in the
coordinating stage, or it might be getting executed just locally because the query did not
need any remote execution.
A latency metric that captures only the first recorded value, ignoring all subsequent ones.
Uses 0 as the unset sentinel (valid durations are clamped to at least 1 nanosecond).
ExecutionPlan that coalesces partitions from multiple tasks into a one or more task without
performing any repartition, and maintaining the same partitioning scheme.
TaskData stores state for a single task being executed by this Endpoint. It may be shared
by concurrent requests for the same task which execute separate partitions.
A value that a transport may either leave encoded or materialize in memory.
Users are free to pass MaybeEncoded::Encoded or MaybeEncoded::Decoded at any
moment and Distributed DataFusion’s code will internally know how to handle it.
Defines what shape should the head node of a stage have upon getting executed. Depending
on the NetworkBoundary implementation, the stage below should have different head nodes.
A unit of isolation for a portion of a physical execution plan
that can be executed independently and across a network boundary.
It implements ExecutionPlan and can be executed to produce a
stream of record batches.
Label used to annotate metrics in execution plan nodes with the task in which they were executed.
Note that the same task id may be used in multiple stages.
Extension trait for DataFusion’s metric system that adds support for byte count metrics
that display using human-readable byte sizes (KB, MB, GB) instead of plain count notation.
Allows users to customize the way Worker clients are created. A common use case is to
wrap the client with tower layers or schedule it in an IO-specific tokio runtime.
This trait represents a node that introduces the necessity of a network boundary in the plan.
The distributed planner, upon stepping into one of these, will break the plan and build a stage
out of it.
Gets the WorkerResolver from the SessionConfig’s extensions. Typically called inside
[RouteTasksHandler::route_tasks] to resolve the worker URLs available for distributed tasks.
Rewrites a distributed plan with metrics. Does nothing if the root node is not a DistributedExec.
Returns an error if the distributed plan was not executed.