Enum datafusion::prelude::Expr

source · [−]
pub enum Expr {

Show 36 variants    Alias(Box<Expr, Global>, String),
    Column(Column),
    ScalarVariable(DataType, Vec<String, Global>),
    Literal(ScalarValue),
    BinaryExpr {
        left: Box<Expr, Global>,
        op: Operator,
        right: Box<Expr, Global>,
    },
    Like {
        negated: bool,
        expr: Box<Expr, Global>,
        pattern: Box<Expr, Global>,
        escape_char: Option<char>,
    },
    ILike {
        negated: bool,
        expr: Box<Expr, Global>,
        pattern: Box<Expr, Global>,
        escape_char: Option<char>,
    },
    SimilarTo {
        negated: bool,
        expr: Box<Expr, Global>,
        pattern: Box<Expr, Global>,
        escape_char: Option<char>,
    },
    Not(Box<Expr, Global>),
    IsNotNull(Box<Expr, Global>),
    IsNull(Box<Expr, Global>),
    IsTrue(Box<Expr, Global>),
    IsFalse(Box<Expr, Global>),
    IsUnknown(Box<Expr, Global>),
    IsNotTrue(Box<Expr, Global>),
    IsNotFalse(Box<Expr, Global>),
    IsNotUnknown(Box<Expr, Global>),
    Negative(Box<Expr, Global>),
    GetIndexedField {
        expr: Box<Expr, Global>,
        key: ScalarValue,
    },
    Between {
        expr: Box<Expr, Global>,
        negated: bool,
        low: Box<Expr, Global>,
        high: Box<Expr, Global>,
    },
    Case {
        expr: Option<Box<Expr, Global>>,
        when_then_expr: Vec<(Box<Expr, Global>, Box<Expr, Global>), Global>,
        else_expr: Option<Box<Expr, Global>>,
    },
    Cast {
        expr: Box<Expr, Global>,
        data_type: DataType,
    },
    TryCast {
        expr: Box<Expr, Global>,
        data_type: DataType,
    },
    Sort {
        expr: Box<Expr, Global>,
        asc: bool,
        nulls_first: bool,
    },
    ScalarFunction {
        fun: BuiltinScalarFunction,
        args: Vec<Expr, Global>,
    },
    ScalarUDF {
        fun: Arc<ScalarUDF>,
        args: Vec<Expr, Global>,
    },
    AggregateFunction {
        fun: AggregateFunction,
        args: Vec<Expr, Global>,
        distinct: bool,
        filter: Option<Box<Expr, Global>>,
    },
    WindowFunction {
        fun: WindowFunction,
        args: Vec<Expr, Global>,
        partition_by: Vec<Expr, Global>,
        order_by: Vec<Expr, Global>,
        window_frame: Option<WindowFrame>,
    },
    AggregateUDF {
        fun: Arc<AggregateUDF>,
        args: Vec<Expr, Global>,
        filter: Option<Box<Expr, Global>>,
    },
    InList {
        expr: Box<Expr, Global>,
        list: Vec<Expr, Global>,
        negated: bool,
    },
    Exists {
        subquery: Subquery,
        negated: bool,
    },
    InSubquery {
        expr: Box<Expr, Global>,
        subquery: Subquery,
        negated: bool,
    },
    ScalarSubquery(Subquery),
    Wildcard,
    QualifiedWildcard {
        qualifier: String,
    },
    GroupingSet(GroupingSet),

}
Expand description

Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).

An Expr can compute its DataType and nullability, and has functions for building up complex expressions.

Examples

Create an expression c1 referring to column named “c1”

let expr = col("c1");
assert_eq!(expr, Expr::Column(Column::from_name("c1")));

Create the expression c1 + c2 to add columns “c1” and “c2” together

let expr = col("c1") + col("c2");

assert!(matches!(expr, Expr::BinaryExpr { ..} ));
if let Expr::BinaryExpr { left, right, op } = expr {
  assert_eq!(*left, col("c1"));
  assert_eq!(*right, col("c2"));
  assert_eq!(op, Operator::Plus);
}

Create expression c1 = 42 to compare the value in column “c1” to the literal value 42

let expr = col("c1").eq(lit(42_i32));

assert!(matches!(expr, Expr::BinaryExpr { .. } ));
if let Expr::BinaryExpr { left, right, op } = expr {
  assert_eq!(*left, col("c1"));
  let scalar = ScalarValue::Int32(Some(42));
  assert_eq!(*right, Expr::Literal(scalar));
  assert_eq!(op, Operator::Eq);
}

Variants

Alias(Box<Expr, Global>, String)

An expression with a specific name.

Column(Column)

A named reference to a qualified filed in a schema.

ScalarVariable(DataType, Vec<String, Global>)

A named reference to a variable in a registry.

Literal(ScalarValue)

A constant value.

BinaryExpr

Fields

left: Box<Expr, Global>

Left-hand side of the expression

op: Operator

The comparison operator

right: Box<Expr, Global>

Right-hand side of the expression

A binary expression such as “age > 21”

Like

Fields

negated: bool
expr: Box<Expr, Global>
pattern: Box<Expr, Global>
escape_char: Option<char>

LIKE expression

ILike

Fields

negated: bool
expr: Box<Expr, Global>
pattern: Box<Expr, Global>
escape_char: Option<char>

Case-insensitive LIKE expression

SimilarTo

Fields

negated: bool
expr: Box<Expr, Global>
pattern: Box<Expr, Global>
escape_char: Option<char>

LIKE expression that uses regular expressions

Not(Box<Expr, Global>)

Negation of an expression. The expression’s type must be a boolean to make sense.

IsNotNull(Box<Expr, Global>)

Whether an expression is not Null. This expression is never null.

IsNull(Box<Expr, Global>)

Whether an expression is Null. This expression is never null.

IsTrue(Box<Expr, Global>)

Whether an expression is True. Boolean operation

IsFalse(Box<Expr, Global>)

Whether an expression is False. Boolean operation

IsUnknown(Box<Expr, Global>)

Whether an expression is Unknown. Boolean operation

IsNotTrue(Box<Expr, Global>)

Whether an expression is not True. Boolean operation

IsNotFalse(Box<Expr, Global>)

Whether an expression is not False. Boolean operation

IsNotUnknown(Box<Expr, Global>)

Whether an expression is not Unknown. Boolean operation

Negative(Box<Expr, Global>)

arithmetic negation of an expression, the operand must be of a signed numeric data type

GetIndexedField

Fields

expr: Box<Expr, Global>

the expression to take the field from

key: ScalarValue

The name of the field to take

Returns the field of a [arrow::array::ListArray] or [arrow::array::StructArray] by key

Between

Fields

expr: Box<Expr, Global>

The value to compare

negated: bool

Whether the expression is negated

low: Box<Expr, Global>

The low end of the range

high: Box<Expr, Global>

The high end of the range

Whether an expression is between a given range.

Case

Fields

expr: Option<Box<Expr, Global>>

Optional base expression that can be compared to literal values in the “when” expressions

when_then_expr: Vec<(Box<Expr, Global>, Box<Expr, Global>), Global>

One or more when/then expressions

else_expr: Option<Box<Expr, Global>>

Optional “else” expression

The CASE expression is similar to a series of nested if/else and there are two forms that can be used. The first form consists of a series of boolean “when” expressions with corresponding “then” expressions, and an optional “else” expression.

CASE WHEN condition THEN result [WHEN …] [ELSE result] END

The second form uses a base expression and then a series of “when” clauses that match on a literal value.

CASE expression WHEN value THEN result [WHEN …] [ELSE result] END

Cast

Fields

expr: Box<Expr, Global>

The expression being cast

data_type: DataType

The DataType the expression will yield

Casts the expression to a given type and will return a runtime error if the expression cannot be cast. This expression is guaranteed to have a fixed type.

TryCast

Fields

expr: Box<Expr, Global>

The expression being cast

data_type: DataType

The DataType the expression will yield

Casts the expression to a given type and will return a null value if the expression cannot be cast. This expression is guaranteed to have a fixed type.

Sort

Fields

expr: Box<Expr, Global>

The expression to sort on

asc: bool

The direction of the sort

nulls_first: bool

Whether to put Nulls before all other data values

A sort expression, that can be used to sort values.

ScalarFunction

Fields

fun: BuiltinScalarFunction

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

Represents the call of a built-in scalar function with a set of arguments.

ScalarUDF

Fields

fun: Arc<ScalarUDF>

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

Represents the call of a user-defined scalar function with arguments.

AggregateFunction

Fields

fun: AggregateFunction

Name of the function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

distinct: bool

Whether this is a DISTINCT aggregation or not

filter: Option<Box<Expr, Global>>

Optional filter

Represents the call of an aggregate built-in function with arguments.

WindowFunction

Fields

fun: WindowFunction

Name of the function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

partition_by: Vec<Expr, Global>

List of partition by expressions

order_by: Vec<Expr, Global>

List of order by expressions

window_frame: Option<WindowFrame>

Window frame

Represents the call of a window function with arguments.

AggregateUDF

Fields

fun: Arc<AggregateUDF>

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

filter: Option<Box<Expr, Global>>

Optional filter applied prior to aggregating

aggregate function

InList

Fields

expr: Box<Expr, Global>

The expression to compare

list: Vec<Expr, Global>

A list of values to compare against

negated: bool

Whether the expression is negated

Returns whether the list contains the expr value.

Exists

Fields

subquery: Subquery

subquery that will produce a single column of data

negated: bool

Whether the expression is negated

EXISTS subquery

InSubquery

Fields

expr: Box<Expr, Global>

The expression to compare

subquery: Subquery

subquery that will produce a single column of data to compare against

negated: bool

Whether the expression is negated

IN subquery

ScalarSubquery(Subquery)

Scalar subquery

Wildcard

Represents a reference to all fields in a schema.

QualifiedWildcard

Fields

qualifier: String

Represents a reference to all fields in a specific schema.

GroupingSet(GroupingSet)

List of grouping set expressions. Only valid in the context of an aggregate GROUP BY expression list

Implementations

Returns the name of this expression as it should appear in a schema. This name will not include any CAST expressions.

Returns a full and complete string representation of this expression.

Return String representation of the variant represented by self Useful for non-rust based bindings

Return self == other

Return self != other

Return self > other

Return self >= other

Return self < other

Return self <= other

Return self && other

Return self || other

Return !self

Calculate the modulus of two expressions. Return self % other

Return self LIKE other

Return self NOT LIKE other

Return self AS name alias expression

Return self IN <list> if negated is false, otherwise return self NOT IN <list>.a

Return `IsNull(Box(self))

Return `IsNotNull(Box(self))

Create a sort expression from an existing expression.

let sort_expr = col("foo").sort(true, true); // SORT ASC NULLS_FIRST

Return IsTrue(Box(self))

Return IsNotTrue(Box(self))

Return IsFalse(Box(self))

Return IsNotFalse(Box(self))

Return IsUnknown(Box(self))

Return IsNotUnknown(Box(self))

Trait Implementations

The resulting type after applying the + operator.
Performs the + operation. Read more
Returns a copy of the value. Read more
Performs copy-assignment from source. Read more

Format expressions for display as part of a logical plan. In many cases, this will produce similar output to Expr.name() except that column names will be prefixed with ‘#’.

Formats the value using the given formatter. Read more

Format expressions for display as part of a logical plan. In many cases, this will produce similar output to Expr.name() except that column names will be prefixed with ‘#’.

Formats the value using the given formatter. Read more
The resulting type after applying the / operator.
Performs the / operation. Read more

Performs a depth first walk of an expression and its children to rewrite an expression, consuming self producing a new Expr.

Implements a modified version of the visitor pattern to separate algorithms from the structure of the Expr tree and make it easier to write new, efficient expression transformation algorithms.

For an expression tree such as

BinaryExpr (GT)
   left: Column("foo")
   right: Column("bar")

The nodes are visited using the following order

pre_visit(BinaryExpr(GT))
pre_visit(Column("foo"))
mutatate(Column("foo"))
pre_visit(Column("bar"))
mutate(Column("bar"))
mutate(BinaryExpr(GT))

If an Err result is returned, recursion is stopped immediately

If false is returned on a call to pre_visit, no children of that expression are visited, nor is mutate called on that expression

Invoked before any children of expr are rewritten / visited. Default implementation returns Ok(RewriteRecursion::Continue) Read more
Invoked after all children of expr have been mutated and returns a potentially modified expr. Read more

Returns the [arrow::datatypes::DataType] of the expression based on ExprSchema

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its [arrow::datatypes::DataType]. This happens when e.g. the expression refers to a column that does not exist in the schema, or when the expression is incorrectly typed (e.g. [utf8] + [bool]).

Returns the nullability of the expression based on ExprSchema.

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its nullability. This happens when the expression refers to a column that does not exist in the schema.

Returns a [arrow::datatypes::Field] compatible with this expression.

Wraps this expression in a cast to a target [arrow::datatypes::DataType].

Errors

This function errors when it is impossible to cast the expression to the target [arrow::datatypes::DataType].

Simplifies this Expr`s as much as possible, evaluating constants and applying algebraic simplifications

Example:

b > 2 AND b > 2 can be written to b > 2

use datafusion_expr::{col, lit, Expr};
use datafusion_common::Result;
use datafusion_physical_expr::execution_props::ExecutionProps;
use datafusion_optimizer::expr_simplifier::{SimplifyInfo, ExprSimplifiable};

/// Simple implementation that provides `Simplifier` the information it needs
#[derive(Default)]
struct Info {
  execution_props: ExecutionProps,
};

impl SimplifyInfo for Info {
  fn is_boolean_type(&self, expr: &Expr) -> Result<bool> {
    Ok(false)
  }
  fn nullable(&self, expr: &Expr) -> Result<bool> {
    Ok(true)
  }
  fn execution_props(&self) -> &ExecutionProps {
    &self.execution_props
  }
}

// b < 2
let b_lt_2 = col("b").gt(lit(2));

// (b < 2) OR (b < 2)
let expr = b_lt_2.clone().or(b_lt_2.clone());

// (b < 2) OR (b < 2) --> (b < 2)
let expr = expr.simplify(&Info::default()).unwrap();
assert_eq!(expr, b_lt_2);

Performs a depth first walk of an expression and its children, calling ExpressionVisitor::pre_visit and visitor.post_visit.

Implements the visitor pattern to separate expression algorithms from the structure of the Expr tree and make it easier to add new types of expressions and algorithms that walk the tree.

For an expression tree such as

BinaryExpr (GT)
   left: Column("foo")
   right: Column("bar")

The nodes are visited using the following order

pre_visit(BinaryExpr(GT))
pre_visit(Column("foo"))
post_visit(Column("foo"))
pre_visit(Column("bar"))
post_visit(Column("bar"))
post_visit(BinaryExpr(GT))

If an Err result is returned, recursion is stopped immediately

If Recursion::Stop is returned on a call to pre_visit, no children of that expression are visited, nor is post_visit called on that expression

Feeds this value into the given Hasher. Read more
Feeds a slice of this type into the given Hasher. Read more
The resulting type after applying the * operator.
Performs the * operation. Read more
The resulting type after applying the ! operator.
Performs the unary ! operation. Read more
This method tests for self and other values to be equal, and is used by ==. Read more
This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason. Read more
This method returns an ordering between self and other values if one exists. Read more
This method tests less than (for self and other) and is used by the < operator. Read more
This method tests less than or equal to (for self and other) and is used by the <= operator. Read more
This method tests greater than (for self and other) and is used by the > operator. Read more
This method tests greater than or equal to (for self and other) and is used by the >= operator. Read more
The resulting type after applying the % operator.
Performs the % operation. Read more
The resulting type after applying the - operator.
Performs the - operation. Read more

Auto Trait Implementations

Blanket Implementations

Gets the TypeId of self. Read more
Immutably borrows from an owned value. Read more
Mutably borrows from an owned value. Read more
Compare self to key and return true if they are equal.

Returns the argument unchanged.

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Should always be Self
The resulting type after obtaining ownership.
Creates owned data from borrowed data, usually by cloning. Read more
Uses borrowed data to replace owned data, usually by cloning. Read more
Converts the given value to a String. Read more
The type returned in the event of a conversion error.
Performs the conversion.
The type returned in the event of a conversion error.
Performs the conversion.