Module execution_plan

Re-exports

pub use crate::display::DefaultDisplay;

pub use crate::display::DisplayAs;

pub use crate::display::DisplayFormatType;

pub use crate::display::VerboseDisplay;

pub use crate::metrics::Metric;

pub use crate::stream::EmptyRecordBatchStream;

Modules

expressions

Defines physical expressions that can evaluated at runtime during query execution

hash_utils

Functionality used both on logical and physical plans

udf

Macros

internal_err

Macro wraps Err($ERR) to add backtrace feature

Structs

ColumnStatistics

Statistics for a column within a relation

PlanProperties

Stores certain, often expensive to compute, plan properties used in query optimization.

Statistics

Statistics for a relation Fields are optional and can be inexact because the sources sometimes provide approximate estimates for performance reasons and the transformations output are not always predictable.

Enums

Boundedness

Represents whether a stream of data generated by an operator is bounded (finite) or unbounded (infinite).

CardinalityEffect

Indicates the effect an execution plan operator will have on the cardinality of its input stream

ColumnarValue

The result of evaluating an expression.

Distribution

How data is distributed amongst partitions. See Partitioning for more details.

EmissionType

Represents how an operator emits its output records.

InputOrderMode

Specifies how the input to an aggregation or window operator is ordered relative to their GROUP BY or PARTITION BY expressions.

InvariantLevel

ExecutionPlan Invariant Level

Partitioning

Output partitioning supported by ExecutionPlans.

Traits

Accumulator

Tracks an aggregate function’s state.

ExecutionPlan

Represent nodes in the DataFusion Physical Plan.

ExecutionPlanProperties

Extension trait provides an easy API to fetch various properties of ExecutionPlan objects based on ExecutionPlan::properties.

PhysicalExpr

PhysicalExprs represent expressions such as A + 1 or CAST(c1 AS int).

RecordBatchStream

Trait for types that stream RecordBatch

WindowExpr

Common trait for window function implementations

Functions

check_not_null_constraints

Checks a RecordBatch for not null constraints on specified columns.

collect

Execute the ExecutionPlan and collect the results in memory

collect_partitioned

Execute the ExecutionPlan and collect the results in memory

displayable

Return a DisplayableExecutionPlan wrapper around an ExecutionPlan which can be displayed in various easier to understand ways.

execute_input_stream

Executes an input stream and ensures that the resulting stream adheres to the not null constraints specified in the sink_schema.

execute_stream

Execute the ExecutionPlan and return a single stream of RecordBatches.

execute_stream_partitioned

Execute the ExecutionPlan and return a vec with one stream per output partition

get_plan_string

Utility function yielding a string representation of the given ExecutionPlan.

need_data_exchange

Indicate whether a data exchange is needed for the input of plan, which will be very helpful especially for the distributed engine to judge whether need to deal with shuffling. Currently there are 3 kinds of execution plan which needs data exchange 1. RepartitionExec for changing the partition number between two ExecutionPlans 2. CoalescePartitionsExec for collapsing all of the partitions into one without ordering guarantee 3. SortPreservingMergeExec for collapsing all of the sorted partitions into one with ordering guarantee

project_schema

Applies an optional projection to a SchemaRef, returning the projected schema

with_new_children_if_necessary

Returns a copy of this plan if we change any child according to the pointer comparison. The size of children must be equal to the size of ExecutionPlan::children().

Type Aliases

SendableRecordBatchStream

Trait for a Stream of RecordBatches that can be passed between threads