Module expressions

Expand description

Definitions and functions to create and manipulate kernel expressions

Re-exports§

col: Convenience macro that builds an Expression column reference by forwarding all args to column_name!:
column_expr_ref
column_name: Creates a nested column name whose field names are all simple column names (containing only alphanumeric characters and underscores).
column_pred
joined_column_expr
joined_column_name: Joins two column names together, when one or both inputs might be literal strings representing simple (non-nested) column names. For example:

ArrayData
BinaryExpression
BinaryPredicate
CastExpression: An expression that casts a child expression to a target type, following SQL CAST semantics: a value that cannot be represented in the target type evaluates to NULL rather than erroring.
ColumnName: A (possibly nested) column name.
DecimalData
JunctionPredicate
MapData
MapToStructExpression: Transforms a Map<String, String> column into a struct whose schema is provided by the evaluator’s output type (via result_type). Each row in the map column becomes one row in the output struct column: a key -> value mapping in the map means the struct field named key receives value, parsed into the field’s target type via PrimitiveType::parse_scalar. An empty-string value is the exception (aligning with Spark): it casts to itself for string, to empty bytes for binary, and to null for every other type. This empty-string rule is specific to this operator; ParseJsonExpression does not share it.
OpaqueExpression
OpaquePredicate
ParseJsonExpression: An expression that parses a JSON string column into a struct column of output_schema, the inverse of UnaryExpressionOp::ToJson except for the sub-millisecond timestamp precision that operator discards.
StructData
UnaryExpression
UnaryPredicate
VariadicExpression

BinaryExpressionOp: A binary arithmetic operator over two numeric operands.
BinaryPredicateOp: A binary predicate operator.
Expression: A SQL expression.
JunctionPredicateOp: A junction (AND/OR) predicate operator over N child predicates.
Predicate: A SQL predicate.
Scalar: A single value, which can be null. Used for representing literal values in Expressions.
UnaryExpressionOp: A unary expression operator.
UnaryPredicateOp: A unary predicate operator.
VariadicExpressionOp: A variadic expression operator.

OpaqueExpressionOp: An opaque expression operation (ie defined and implemented by the engine).
OpaquePredicateOp: An opaque predicate operation (ie defined and implemented by the engine).

lit: Build an Expression::Literal from anything that converts into a Scalar.
null_lit: Build a typed NULL Expression::Literal.

ExpressionRef
ExpressionStructPatchBuilder: A StructPatchBuilder whose emitted items are expressions, lowered into an ExpressionStructPatch that can be embedded in an Expression.
OpaqueExpressionOpRef: A shared reference to an OpaqueExpressionOp instance.
OpaquePredicateOpRef: A shared reference to an OpaquePredicateOp instance.
PredicateRef
ScalarExpressionEvaluator: A kernel-supplied scalar expression evaluator which in particular can convert column references (i.e. Expression::Column) to Scalar values. OpaqueExpressionOp::eval_expr_scalar and OpaquePredicateOp::eval_pred_scalar rely on this evaluator.