Module datafusion::logical_expr
source · Expand description
re-export of datafusion_expr
crate
Modules§
- Aggregate function module contains all built-in aggregate functions definitions
- Conditional expressions
- Logical Expressions:
Expr
- Functions for creating logical expressions
- Expression rewriter
- Utility functions for complex field access
- Function module contains typing and signature for built-in and user defined functions.
- Vectorized
GroupsAccumulator
- Interval arithmetic library
- Structs and traits to provide the information needed for expression simplification.
- Tree node implementation for logical expr
- Type coercion rules for DataFusion
- Expression utilities
- Variable provider
- Window frame module
- Structures used to hold window function state (for implementing WindowUDFs)
Macros§
Structs§
- Aggregates its input based on a set of grouping and aggregate expressions (e.g. SUM).
- Logical representation of a user-defined aggregate function (UDAF).
- Runs the actual plan, and then prints the physical plan with with execution metrics.
- BETWEEN expression
- Binary expression
- CASE expression
- Cast expression
- Creates a catalog (aka “Database”).
- Creates a schema.
- Creates an external table.
- Arguments passed to
CREATE FUNCTION
- Creates an in memory table.
- Creates a view.
- Apply Cross Join to two logical plans
- Describe the schema of table
- Removes duplicate rows from the input
- The operator that modifies the content of a database (adapted from substrait WriteRel)
- Drops a schema
- Drops a table.
- Drops a view.
- Produces no rows: An empty relation with an empty schema
- Produces a relation with string representations of various parts of the plan
- Extension operator defined outside of DataFusion
- Filters rows from its input that do not match an expression (essentially a WHERE clause with a predicate expression).
- Returns the field of a
arrow::array::ListArray
orarrow::array::StructArray
bykey
. SeeGetFieldAccess
for details. - Join two logical plans on one or more join columns
- LIKE expression
- Produces the first
n
tuples from its input and discards the rest. - Builder for logical plans
- Prepare a statement but do not execute it. Prepare statements can have 0 or more
Expr::Placeholder
expressions that are filled in during execution - Evaluates an arbitrary list of expressions (essentially a SELECT with an expression list) on its input.
- A variadic query operation, Recursive CTE.
- Repartition the plan based on a partitioning scheme.
- Logical representation of a Scalar User Defined Function.
- Set a Variable’s value – value in
ConfigOptions
- Defines the supported argument types (
TypeSignature
) andVolatility
for a function. - Implements
AggregateUDFImpl
for functions that have a single signature and return type. - Implements
ScalarUDFImpl
for functions that have a single signature and return type. - Implements
WindowUDFImpl
for functions that have a single signature and return type. - Sorts its input according to a list of sort expressions.
- Represents some sort of execution plan, in String form
- Subquery
- Aliased subquery
- Produces rows from a table provider by reference or from the context
- Indicator that any current transaction should be terminated
- Indicator that the following statements should be committed or rolled back atomically
- TryCast Expression
- Union multiple inputs
- Unnest a column that contains a nested list type. See
UnnestOptions
for more details. - Values expression. See Postgres VALUES documentation for more details.
- Window its input based on a set of window spec and window function (e.g. SUM or RANK)
- The frame specification determines which output rows are read by an aggregate window function. The ending frame boundary can be omitted if the
BETWEEN
andAND
keywords that surround the starting frame boundary are also omitted, in which case the ending frame boundary defaults toCURRENT ROW
. - Logical representation of a user-defined window function (UDWF) A UDWF is different from a UDF in that it is stateful across batches.
Enums§
- Enum of all built-in aggregate functions
- A window function built in to DataFusion
- The result of evaluating an expression.
- Various types of DDL (CREATE / DROP) catalog manipulation
- Removes duplicate rows from the input
- Describes how many rows should be emitted during grouping.
Expr
is a central struct of DataFusion’s query API, and represent logical expressions such asA + 1
, orCAST(c1 AS int)
.- Access a sub field of a nested type, such as
Field
orList
- Grouping sets See https://www.postgresql.org/docs/current/queries-table-expressions.html#QUERIES-GROUPING-SETS for Postgres definition. See https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-groupby.html for Apache Spark definition.
- Join constraint
- Join type
- A LogicalPlan represents the different types of relational operators (such as Projection, Filter, etc) and can be created by the SQL query planner and the DataFrame API.
- Operators applied to expressions
- Logical partitioning schemes supported by the repartition operator.
- Represents which type of plan, when storing multiple for use in EXPLAIN plans
- Defines which implementation of a function for DataFusion to call.
- Various types of Statements.
- Indicates how a filter expression is handled by
TableProvider::scan
. - Indicates the type of this table for metadata/catalog purposes.
- Indicates if this transaction is allowed to write
- Indicates if a transaction was committed or aborted
- Indicates ANSI transaction isolation level
- A function’s type signature defines the types of arguments the function supports.
- A function’s volatility, which defines the functions eligibility for certain optimizations
- There are five ways to describe starting and ending frame boundaries:
- There are three frame types: ROWS, GROUPS, and RANGE. The frame type determines how the starting and ending boundaries of the frame are measured.
- WindowFunction Defines which implementation of an aggregate function DataFusion should call.
Constants§
- Constant that is used as a placeholder for any valid timezone. This is used where a function can accept a timestamp type with any valid timezone, it exists to avoid the need to enumerate all possible timezones. See
TypeSignature
for more details. - Default table name for unnamed table
Traits§
- Tracks an aggregate function’s state.
- Trait for implementing
AggregateUDF
. - trait to allow expr to typable with respect to a schema
GroupAccumulator
implements a single aggregate (e.g. AVG) and stores the state for all groups internally.- Trait for converting a type to a
Literal
literal expression. - Partition evaluator for Window Functions
- Trait for implementing
ScalarUDF
. - The TableSource trait is used during logical query planning and optimizations and provides access to schema information and filter push-down capabilities. This trait provides a subset of the functionality of the TableProvider trait in the core datafusion crate. The TableProvider trait provides additional capabilities needed for physical query execution (such as the ability to perform a scan). The reason for having two separate traits is to avoid having the logical plan code be dependent on the DataFusion execution engine. Other projects may want to use DataFusion’s logical plans and have their own execution engine.
- Trait for converting a type to a literal timestamp
- Trait for something that can be formatted as a stringified plan
- This defines the interface for
LogicalPlan
nodes that can be used to extend DataFusion with custom relational operators. - This trait facilitates implementation of the
UserDefinedLogicalNode
. - Trait for implementing
WindowUDF
.
Functions§
- Return a new expression with a logical AND
- Returns the approximate number of distinct input values. This function provides an approximation of count(DISTINCT x). Zero is returned if all input values are null. This function should produce a standard error of 0.81%, which is the standard deviation of the (approximately normal) error distribution over all possible sets. It does not guarantee an upper bound on the error for any specific input set.
- Calculate an approximation of the median for
expr
. - Calculate an approximation of the specified
percentile
forexpr
. - Calculate an approximation of the specified
percentile
forexpr
andweight_expr
. - Create an expression to represent the array_agg() aggregate function
- Create an expression to represent the avg() aggregate function
- Return a new expression
left <op> right
- Return a new expression with bitwise AND
- Return a new expression with bitwise OR
- Return a new expression with bitwise SHIFT LEFT
- Return a new expression with bitwise SHIFT RIGHT
- Return a new expression with bitwise XOR
- Creates a schema for a join operation. The fields from the left side are first
- Create a CASE WHEN statement with literal WHEN expressions for comparison to the base expression.
- Create a cast expression
- Create a column expression based on a qualified or unqualified column name. Will normalize unquoted identifiers according to SQL rules (identifiers will become lowercase).
- Create an expression to represent the count() aggregate function
- Create an expression to represent the count(distinct) aggregate function
- Creates a new UDAF with a specific signature, state type and return type. The signature and state type must match the
Accumulator's implementation
. - Convenience method to create a new user defined scalar function (UDF) with a specific signature and specific return type.
- Creates a new UDWF with a specific signature, state type and return type.
- Create a grouping set for all combination of
exprs
- Print the schema in a compact representation to
buf
- Create an EXISTS subquery expression
- Create a grouping set
- Create an unqualified column expression from the provided name, without normalizing the column.
- Create an in_list expression
- Create an IN subquery expression
- Create is false expression
- Create is not false expression
- Create is not true expression
- Create is not unknown expression
- Create is null expression
- Create is true expression
- Create is unknown expression
- Create a literal expression
- Create a literal timestamp expression
- Create an expression to represent the max() aggregate function
- Calculate the median for
expr
. - Create an expression to represent the min() aggregate function
- Return a new expression with a logical NOT
- Create a NOT EXISTS subquery expression
- Create a NOT IN subquery expression
- Return a new expression with a logical OR
- Create an out reference column which hold a reference that has been resolved to a field outside of the current plan.
- Create placeholder value that will be filled in (such as
$1
) - Computes the schema of the result produced by applying a projection to the input logical plan.
- Create a grouping set for rollup
- Create a scalar subquery expression
- Create an expression to represent the stddev() aggregate function
- Create an expression to represent the sum() aggregate function
- Create a LogicalPlanBuilder representing a scan of a table with the provided name and schema. This is mostly used for testing and documentation.
- Create a try cast expression
- Union two logical plans.
- Create a CASE WHEN statement with boolean WHEN expressions and no base expression.
- Create an ‘*’
Expr::Wildcard
expression that matches all columns - Wrap projection for a plan, if the join keys contains normal expression.
Type Aliases§
- Factory that returns an accumulator for the given aggregate function.
- Monotonicity of the
ScalarFunctionExpr
with respect to its arguments. Each element of this vector corresponds to an argument and indicates whether the function’s behavior is monotonic, or non-monotonic/unknown for that argument, namely: - Factory that creates a PartitionEvaluator for the given window function
- Factory that returns the functions’s return type given the input argument types
- Scalar function
- Factory that returns the types used by an aggregator to serialize its state, given its return datatype.