Module datafusion::physical_plan [−][src]
Expand description
Traits for physical query plan, supporting parallel execution for partitioned relations.
Re-exports
pub use display::DisplayFormatType;pub use self::planner::PhysicalPlanner;Modules
Declaration of built-in (aggregate) functions. This module contains built-in aggregates’ enumeration and metadata.
Array expressions
CoalesceBatchesExec combines small batches into larger batches for more efficient use of vectorized processing by upstream operators.
Defines the merge plan for executing partitions in parallel and then merging the results into a single partition
Defines common code used in execution plans
Defines the cross join plan for loading the left side of the cross join and producing batches in parallel for the right partitions
Crypto expressions
Execution plan for reading CSV files
DateTime expressions
Implementation of physical plan display. See
crate::physical_plan::displayable for examples of how to
format
Implementations for DISTINCT expressions, e.g. COUNT(DISTINCT c)
EmptyRelation execution plan
Defines the EXPLAIN operator
Defines physical expressions that can evaluated at runtime during query execution
FilterExec evaluates a boolean predicate against all input batches to determine which rows to include in its output batches.
Declaration of built-in (scalar) functions. This module contains built-in functions’ enumeration and metadata.
Defines the execution plan for the hash aggregate operation
Defines the join plan for executing partitions in parallel and then joining the results into a set of partitions.
Functionality used both on logical and physical plans
Execution plan for reading line-delimited JSON files
Defines the LIMIT plan
Math expressions
Execution plan for reading in-memory batches of data
Execution plan for reading Parquet files
Physical query planner
Defines the projection execution plan. A projection determines which columns or expressions
are returned from a query. The SQL statement SELECT a, b, a+b FROM t1 is an example
of a projection on table t1 where the expressions a, b, and a+b are the
projection expressions. SELECT without FROM will only evaluate expressions.
Regex expressions
The repartition operator maps N input partitions to M output partitions based on a partitioning scheme.
Defines the SORT plan
Defines the sort preserving merge plan
Contains a Source enum represents where the data comes from.
String expressions
Type coercion rules for functions with multiple valid signatures
This module contains functions and structs supporting user-defined aggregate functions.
UDF support
Unicode expressions
The Union operator combines multiple inputs with the same schema
Window functions provide the ability to perform calculations across sets of rows that are related to the current query row.
Physical expressions for window functions
Structs
EmptyRecordBatchStream can be used to create a RecordBatchStream that will produce no results
SQL metric such as counter (number of input or output rows) or timing information about a physical operator.
Enums
Represents the result from an expression
Distribution schemes
SQL metric type
Partitioning schemes supported by operators.
Traits
An accumulator represents a stateful object that lives throughout the evaluation of multiple rows and generically accumulates values.
An aggregate expression that:
ExecutionPlan represent nodes in the DataFusion Physical Plan.
Trait that implements the Visitor
pattern for a
depth first walk of ExecutionPlan nodes. pre_visit is called
before any children are visited, and then post_visit is called
after all children have been visited.
To use, define a struct that implements this trait and then invoke
[‘accept’].
Expression that can be evaluated against a RecordBatch A Physical expression knows its type, nullability and how to evaluate itself.
Trait for types that stream arrow::record_batch::RecordBatch
A window expression that:
Functions
Visit all children of this plan, according to the order defined on ExecutionPlanVisitor.
Execute the ExecutionPlan and collect the results in memory
Execute the ExecutionPlan and collect the results in memory
Return a wrapper around an
ExecutionPlan which can be displayed in various easier to
understand ways.
Execute the ExecutionPlan and return a single stream of results
Execute the ExecutionPlan and return a vec with one stream per output partition
Recursively gateher all execution metrics from this plan and all of its input plans
Recursively calls pre_visit and post_visit for this node and
all of its children, as described on ExecutionPlanVisitor
Type Definitions
Trait for a stream of record batches.