Crate datafusion_expr

DataFusion is an extensible query execution framework that uses Apache Arrow as its in-memory format.

This crate is a submodule of DataFusion that provides types representing logical query plans (LogicalPlan) and logical expressions (Expr) as well as utilities for working with these types.

The expr_fn module contains functions for creating expressions.

Re-exports

• pub use aggregate_function::AggregateFunction;
• pub use expr::Between;
• pub use expr::BinaryExpr;
• pub use expr::Case;
• pub use expr::Cast;
• pub use expr::Expr;
• pub use expr::GetIndexedField;
• pub use expr::GroupingSet;
• pub use expr::Like;
• pub use expr::TryCast;
• pub use expr_schema::ExprSchemable;
• pub use function::AccumulatorFunctionImplementation;
• pub use function::ReturnTypeFunction;
• pub use function::ScalarFunctionImplementation;
• pub use function::StateTypeFunction;
• pub use logical_plan::builder::build_join_schema;
• pub use logical_plan::builder::union;
• pub use logical_plan::builder::wrap_projection_for_join_if_necessary;
• pub use logical_plan::builder::UNNAMED_TABLE;
• pub use logical_plan::LogicalPlanBuilder;
• pub use window_frame::WindowFrame;
• pub use window_frame::WindowFrameBound;
• pub use window_frame::WindowFrameUnits;
• pub use window_function::BuiltInWindowFunction;
• pub use window_function::WindowFunction;
• pub use expr_fn::*;

Modules

• aggregate_function
  Aggregate function module contains all built-in aggregate functions definitions
• array_expressions
• conditional_expressions
• expr
  Expr module contains core type definition for Expr.
• expr_fn
  Functions for creating logical expressions
• expr_rewriter
  Expression rewriter
• expr_schema
• expr_visitor
  Expression visitor
• field_util
  Utility functions for complex field access
• function
  Function module contains typing and signature for built-in and user defined functions.
• logical_plan
• struct_expressions
• type_coercion
  Type coercion rules for DataFusion
• utils
  Expression utilities
• window_frame
  Window frame module
• window_function
  Window functions provide the ability to perform calculations across sets of rows that are related to the current query row.

Structs

• Aggregate
  Aggregates its input based on a set of grouping and aggregate expressions (e.g. SUM).
• AggregateUDF
  Logical representation of a user-defined aggregate function (UDAF) A UDAF is different from a UDF in that it is stateful across batches.
• CreateCatalog
  Creates a catalog (aka “Database”).
• CreateCatalogSchema
  Creates a schema.
• CreateExternalTable
  Creates an external table.
• CreateMemoryTable
  Creates an in memory table.
• CreateView
  Creates a view.
• CrossJoin
  Apply Cross Join to two logical plans
• DescribeTable
  Describe the schema of table
• Distinct
  Removes duplicate rows from the input
• DmlStatement
  The operator that modifies the content of a database (adapted from substrait WriteRel)
• DropTable
  Drops a table.
• DropView
  Drops a view.
• EmptyRelation
  Produces no rows: An empty relation with an empty schema
• Explain
  Produces a relation with string representations of various parts of the plan
• Extension
  Extension operator defined outside of DataFusion
• Filter
  Filters rows from its input that do not match an expression (essentially a WHERE clause with a predicate expression).
• Join
  Join two logical plans on one or more join columns
• Limit
  Produces the first n tuples from its input and discards the rest.
• Projection
  Evaluates an arbitrary list of expressions (essentially a SELECT with an expression list) on its input.
• Repartition
  Repartition the plan based on a partitioning scheme.
• ScalarUDF
  Logical representation of a UDF.
• SetVariable
  Set a Variable’s value – value in ConfigOptions
• Signature
  The Signature of a function defines its supported input types as well as its volatility.
• Sort
  Sorts its input according to a list of sort expressions.
• StringifiedPlan
  Represents some sort of execution plan, in String form
• Subquery
  Subquery
• SubqueryAlias
  Aliased subquery
• TableScan
  Produces rows from a table provider by reference or from the context
• Union
  Union multiple inputs
• Unnest
  Unnest a column that contains a nested list type.
• Values
  Values expression. See Postgres VALUES documentation for more details.
• Window
  Window its input based on a set of window spec and window function (e.g. SUM or RANK)

Enums

• BuiltinScalarFunction
  Enum of all built-in scalar functions
• ColumnarValue
  Represents the result of evaluating an expression: either a single ScalarValue or an [ArrayRef].
• JoinConstraint
  Join constraint
• JoinType
  Join type
• LogicalPlan
  A LogicalPlan represents the different types of relational operators (such as Projection, Filter, etc) and can be created by the SQL query planner and the DataFrame API.
• Operator
  Operators applied to expressions
• Partitioning
  Logical partitioning schemes supported by the repartition operator.
• PlanType
  Represents which type of plan, when storing multiple for use in EXPLAIN plans
• TableProviderFilterPushDown
  ! Table source Indicates whether and how a filter expression can be handled by a TableProvider for table scans.
• TableType
  Indicates the type of this table for metadata/catalog purposes.
• TypeSignature
  A function’s type signature, which defines the function’s supported argument types.
• Volatility
  A function’s volatility, which defines the functions eligibility for certain optimizations
• WriteOp

Statics

• SUPPORTED_NULLIF_TYPES
  Currently supported types by the nullif function. The order of these types correspond to the order on which coercion applies This should thus be from least informative to most informative

Traits

• Accumulator
  An accumulator represents a stateful object that lives throughout the evaluation of multiple rows and generically accumulates values.
• Literal
  Trait for converting a type to a Literal literal expression.
• PlanVisitor
  Trait that implements the Visitor pattern for a depth first walk of LogicalPlan nodes. pre_visit is called before any children are visited, and then post_visit is called after all children have been visited. To use, define a struct that implements this trait and then invoke LogicalPlan::accept.
• TableSource
  The TableSource trait is used during logical query planning and optimizations and provides access to schema information and filter push-down capabilities. This trait provides a subset of the functionality of the TableProvider trait in the core datafusion crate. The TableProvider trait provides additional capabilities needed for physical query execution (such as the ability to perform a scan). The reason for having two separate traits is to avoid having the logical plan code be dependent on the DataFusion execution engine. Other projects may want to use DataFusion’s logical plans and have their own execution engine.
• TimestampLiteral
  Trait for converting a type to a literal timestamp
• ToStringifiedPlan
  Trait for something that can be formatted as a stringified plan
• UserDefinedLogicalNode
  This defines the interface for LogicalPlan nodes that can be used to extend DataFusion with custom relational operators.

Functions

• lit
  Create a literal expression
• lit_timestamp_nano
  Create a literal timestamp expression