Enum datafusion::prelude::Expr

pub enum Expr {
    Alias(Box<Expr, Global>, String),
    Column(Column),
    ScalarVariable(DataType, Vec<String, Global>),
    Literal(ScalarValue),
    BinaryExpr {
        left: Box<Expr, Global>,
        op: Operator,
        right: Box<Expr, Global>,
    },
    Not(Box<Expr, Global>),
    IsNotNull(Box<Expr, Global>),
    IsNull(Box<Expr, Global>),
    Negative(Box<Expr, Global>),
    GetIndexedField {
        expr: Box<Expr, Global>,
        key: ScalarValue,
    },
    Between {
        expr: Box<Expr, Global>,
        negated: bool,
        low: Box<Expr, Global>,
        high: Box<Expr, Global>,
    },
    Case {
        expr: Option<Box<Expr, Global>>,
        when_then_expr: Vec<(Box<Expr, Global>, Box<Expr, Global>), Global>,
        else_expr: Option<Box<Expr, Global>>,
    },
    Cast {
        expr: Box<Expr, Global>,
        data_type: DataType,
    },
    TryCast {
        expr: Box<Expr, Global>,
        data_type: DataType,
    },
    Sort {
        expr: Box<Expr, Global>,
        asc: bool,
        nulls_first: bool,
    },
    ScalarFunction {
        fun: BuiltinScalarFunction,
        args: Vec<Expr, Global>,
    },
    ScalarUDF {
        fun: Arc<ScalarUDF>,
        args: Vec<Expr, Global>,
    },
    AggregateFunction {
        fun: AggregateFunction,
        args: Vec<Expr, Global>,
        distinct: bool,
    },
    WindowFunction {
        fun: WindowFunction,
        args: Vec<Expr, Global>,
        partition_by: Vec<Expr, Global>,
        order_by: Vec<Expr, Global>,
        window_frame: Option<WindowFrame>,
    },
    AggregateUDF {
        fun: Arc<AggregateUDF>,
        args: Vec<Expr, Global>,
    },
    InList {
        expr: Box<Expr, Global>,
        list: Vec<Expr, Global>,
        negated: bool,
    },
    Exists {
        subquery: Subquery,
        negated: bool,
    },
    InSubquery {
        expr: Box<Expr, Global>,
        subquery: Subquery,
        negated: bool,
    },
    ScalarSubquery(Subquery),
    Wildcard,
    QualifiedWildcard {
        qualifier: String,
    },
    GroupingSet(GroupingSet),
}

Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).

An Expr can compute its DataType and nullability, and has functions for building up complex expressions.

Examples

Create an expression c1 referring to column named “c1”

let expr = col("c1");
assert_eq!(expr, Expr::Column(Column::from_name("c1")));

Create the expression c1 + c2 to add columns “c1” and “c2” together

let expr = col("c1") + col("c2");

assert!(matches!(expr, Expr::BinaryExpr { ..} ));
if let Expr::BinaryExpr { left, right, op } = expr {
  assert_eq!(*left, col("c1"));
  assert_eq!(*right, col("c2"));
  assert_eq!(op, Operator::Plus);
}

Create expression c1 = 42 to compare the value in column “c1” to the literal value 42

let expr = col("c1").eq(lit(42_i32));

assert!(matches!(expr, Expr::BinaryExpr { .. } ));
if let Expr::BinaryExpr { left, right, op } = expr {
  assert_eq!(*left, col("c1"));
  let scalar = ScalarValue::Int32(Some(42));
  assert_eq!(*right, Expr::Literal(scalar));
  assert_eq!(op, Operator::Eq);
}

Variants

Alias(Box<Expr, Global>, String)

An expression with a specific name.

Column(Column)

A named reference to a qualified filed in a schema.

ScalarVariable(DataType, Vec<String, Global>)

A named reference to a variable in a registry.

Literal(ScalarValue)

A constant value.

BinaryExpr

Fields

left: Box<Expr, Global>

Left-hand side of the expression

op: Operator

The comparison operator

right: Box<Expr, Global>

Right-hand side of the expression

A binary expression such as “age > 21”

Not(Box<Expr, Global>)

Negation of an expression. The expression’s type must be a boolean to make sense.

IsNotNull(Box<Expr, Global>)

Whether an expression is not Null. This expression is never null.

IsNull(Box<Expr, Global>)

Whether an expression is Null. This expression is never null.

Negative(Box<Expr, Global>)

arithmetic negation of an expression, the operand must be of a signed numeric data type

GetIndexedField

Fields

expr: Box<Expr, Global>

the expression to take the field from

key: ScalarValue

The name of the field to take

Returns the field of a arrow::array::ListArray or arrow::array::StructArray by key

Between

Fields

expr: Box<Expr, Global>

The value to compare

negated: bool

Whether the expression is negated

low: Box<Expr, Global>

The low end of the range

high: Box<Expr, Global>

The high end of the range

Whether an expression is between a given range.

Case

Fields

expr: Option<Box<Expr, Global>>

Optional base expression that can be compared to literal values in the “when” expressions

when_then_expr: Vec<(Box<Expr, Global>, Box<Expr, Global>), Global>

One or more when/then expressions

else_expr: Option<Box<Expr, Global>>

Optional “else” expression

The CASE expression is similar to a series of nested if/else and there are two forms that can be used. The first form consists of a series of boolean “when” expressions with corresponding “then” expressions, and an optional “else” expression.

CASE WHEN condition THEN result [WHEN …] [ELSE result] END

The second form uses a base expression and then a series of “when” clauses that match on a literal value.

CASE expression WHEN value THEN result [WHEN …] [ELSE result] END

Cast

Fields

expr: Box<Expr, Global>

The expression being cast

data_type: DataType

The DataType the expression will yield

Casts the expression to a given type and will return a runtime error if the expression cannot be cast. This expression is guaranteed to have a fixed type.

TryCast

Fields

expr: Box<Expr, Global>

The expression being cast

data_type: DataType

The DataType the expression will yield

Casts the expression to a given type and will return a null value if the expression cannot be cast. This expression is guaranteed to have a fixed type.

Sort

Fields

expr: Box<Expr, Global>

The expression to sort on

asc: bool

The direction of the sort

nulls_first: bool

Whether to put Nulls before all other data values

A sort expression, that can be used to sort values.

ScalarFunction

Fields

fun: BuiltinScalarFunction

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

Represents the call of a built-in scalar function with a set of arguments.

ScalarUDF

Fields

fun: Arc<ScalarUDF>

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

Represents the call of a user-defined scalar function with arguments.

AggregateFunction

Fields

fun: AggregateFunction

Name of the function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

distinct: bool

Whether this is a DISTINCT aggregation or not

Represents the call of an aggregate built-in function with arguments.

WindowFunction

Fields

fun: WindowFunction

Name of the function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

partition_by: Vec<Expr, Global>

List of partition by expressions

order_by: Vec<Expr, Global>

List of order by expressions

window_frame: Option<WindowFrame>

Window frame

Represents the call of a window function with arguments.

AggregateUDF

Fields

fun: Arc<AggregateUDF>

The function

args: Vec<Expr, Global>

List of expressions to feed to the functions as arguments

aggregate function

InList

Fields

expr: Box<Expr, Global>

The expression to compare

list: Vec<Expr, Global>

A list of values to compare against

negated: bool

Whether the expression is negated

Returns whether the list contains the expr value.

Exists

Fields

subquery: Subquery

subquery that will produce a single column of data

negated: bool

Whether the expression is negated

EXISTS subquery

InSubquery

Fields

expr: Box<Expr, Global>

The expression to compare

subquery: Subquery

subquery that will produce a single column of data to compare against

negated: bool

Whether the expression is negated

IN subquery

ScalarSubquery(Subquery)

Scalar subquery

Wildcard

Represents a reference to all fields in a schema.

QualifiedWildcard

Fields

qualifier: String

Represents a reference to all fields in a specific schema.

GroupingSet(GroupingSet)

List of grouping set expressions. Only valid in the context of an aggregate GROUP BY expression list

Implementations

impl Expr

pub fn name(&self, input_schema: &DFSchema) -> Result<String, DataFusionError>

Returns the name of this expression based on datafusion_common::DFSchema.

This represents how a column with this expression is named when no alias is chosen

pub fn variant_name(&self) -> &str

Return String representation of the variant represented by self Useful for non-rust based bindings

pub fn eq(self, other: Expr) -> Expr

Return self == other

pub fn not_eq(self, other: Expr) -> Expr

Return self != other

pub fn gt(self, other: Expr) -> Expr

Return self > other

pub fn gt_eq(self, other: Expr) -> Expr

Return self >= other

pub fn lt(self, other: Expr) -> Expr

Return self < other

pub fn lt_eq(self, other: Expr) -> Expr

Return self <= other

pub fn and(self, other: Expr) -> Expr

Return self && other

pub fn or(self, other: Expr) -> Expr

Return self || other

pub fn not(self) -> Expr

Return !self

pub fn modulus(self, other: Expr) -> Expr

Calculate the modulus of two expressions. Return self % other

pub fn like(self, other: Expr) -> Expr

Return self LIKE other

pub fn not_like(self, other: Expr) -> Expr

Return self NOT LIKE other

pub fn alias(self, name: &str) -> Expr

Return self AS name alias expression

pub fn in_list(self, list: Vec<Expr, Global>, negated: bool) -> Expr

Return self IN <list> if negated is false, otherwise return self NOT IN <list>.a

pub fn is_null(self) -> Expr

Return `IsNull(Box(self))

pub fn is_not_null(self) -> Expr

Return `IsNotNull(Box(self))

pub fn sort(self, asc: bool, nulls_first: bool) -> Expr

Create a sort expression from an existing expression.

let sort_expr = col("foo").sort(true, true); // SORT ASC NULLS_FIRST

Trait Implementations

impl Add<Expr> for Expr

type Output = Expr

The resulting type after applying the + operator.

fn add(self, rhs: Expr) -> Expr

Performs the + operation.

impl Clone for Expr

fn clone(&self) -> Expr

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Expr

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Display for Expr

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Div<Expr> for Expr

type Output = Expr

The resulting type after applying the / operator.

fn div(self, rhs: Expr) -> Expr

Performs the / operation.

impl ExprRewritable for Expr

fn rewrite<R>(self, rewriter: &mut R) -> Result<Expr, DataFusionError> where
    R: ExprRewriter<Expr>, 

Performs a depth first walk of an expression and its children to rewrite an expression, consuming self producing a new Expr.

Implements a modified version of the visitor pattern to separate algorithms from the structure of the Expr tree and make it easier to write new, efficient expression transformation algorithms.

For an expression tree such as

BinaryExpr (GT)
   left: Column("foo")
   right: Column("bar")

The nodes are visited using the following order

pre_visit(BinaryExpr(GT))
pre_visit(Column("foo"))
mutatate(Column("foo"))
pre_visit(Column("bar"))
mutate(Column("bar"))
mutate(BinaryExpr(GT))

If an Err result is returned, recursion is stopped immediately

If false is returned on a call to pre_visit, no children of that expression are visited, nor is mutate called on that expression

impl<'a> ExprRewriter<Expr> for ConstEvaluator<'a>

fn pre_visit(&mut self, expr: &Expr) -> Result<RewriteRecursion, DataFusionError>

Invoked before any children of expr are rewritten / visited. Default implementation returns Ok(RewriteRecursion::Continue)

fn mutate(&mut self, expr: Expr) -> Result<Expr, DataFusionError>

Invoked after all children of expr have been mutated and returns a potentially modified expr.

impl ExprSchemable for Expr

fn get_type<S>(&self, schema: &S) -> Result<DataType, DataFusionError> where
    S: ExprSchema, 

Returns the arrow::datatypes::DataType of the expression based on ExprSchema

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its arrow::datatypes::DataType. This happens when e.g. the expression refers to a column that does not exist in the schema, or when the expression is incorrectly typed (e.g. [utf8] + [bool]).

fn nullable<S>(&self, input_schema: &S) -> Result<bool, DataFusionError> where
    S: ExprSchema, 

Returns the nullability of the expression based on ExprSchema.

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its nullability. This happens when the expression refers to a column that does not exist in the schema.

fn to_field(&self, input_schema: &DFSchema) -> Result<DFField, DataFusionError>

Returns a arrow::datatypes::Field compatible with this expression.

fn cast_to<S>(
    self,
    cast_to_type: &DataType,
    schema: &S
) -> Result<Expr, DataFusionError> where
    S: ExprSchema, 

Wraps this expression in a cast to a target arrow::datatypes::DataType.

Errors

This function errors when it is impossible to cast the expression to the target arrow::datatypes::DataType.

impl ExprSimplifiable for Expr

fn simplify<S>(self, info: &S) -> Result<Expr, DataFusionError> where
    S: SimplifyInfo, 

Simplifies this Expr`s as much as possible, evaluating constants and applying algebraic simplifications

Example:

b > 2 AND b > 2 can be written to b > 2

use datafusion_expr::{col, lit, Expr};
use datafusion_common::Result;
use datafusion_physical_expr::execution_props::ExecutionProps;
use datafusion_optimizer::expr_simplifier::{SimplifyInfo, ExprSimplifiable};

/// Simple implementation that provides `Simplifier` the information it needs
#[derive(Default)]
struct Info {
  execution_props: ExecutionProps,
};

impl SimplifyInfo for Info {
  fn is_boolean_type(&self, expr: &Expr) -> Result<bool> {
    Ok(false)
  }
  fn nullable(&self, expr: &Expr) -> Result<bool> {
    Ok(true)
  }
  fn execution_props(&self) -> &ExecutionProps {
    &self.execution_props
  }
}

// b < 2
let b_lt_2 = col("b").gt(lit(2));

// (b < 2) OR (b < 2)
let expr = b_lt_2.clone().or(b_lt_2.clone());

// (b < 2) OR (b < 2) --> (b < 2)
let expr = expr.simplify(&Info::default()).unwrap();
assert_eq!(expr, b_lt_2);

impl ExprVisitable for Expr

fn accept<V>(&self, visitor: V) -> Result<V, DataFusionError> where
    V: ExpressionVisitor<Expr>, 

Performs a depth first walk of an expression and its children, calling ExpressionVisitor::pre_visit and visitor.post_visit.

Implements the visitor pattern to separate expression algorithms from the structure of the Expr tree and make it easier to add new types of expressions and algorithms that walk the tree.

For an expression tree such as

BinaryExpr (GT)
   left: Column("foo")
   right: Column("bar")

The nodes are visited using the following order

pre_visit(BinaryExpr(GT))
pre_visit(Column("foo"))
post_visit(Column("foo"))
pre_visit(Column("bar"))
post_visit(Column("bar"))
post_visit(BinaryExpr(GT))

If an Err result is returned, recursion is stopped immediately

If Recursion::Stop is returned on a call to pre_visit, no children of that expression are visited, nor is post_visit called on that expression

impl Hash for Expr

fn hash<__H>(&self, state: &mut __H) where
    __H: Hasher, 

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl Mul<Expr> for Expr

type Output = Expr

The resulting type after applying the * operator.

fn mul(self, rhs: Expr) -> Expr

Performs the * operation.

impl Not for Expr

type Output = Expr

The resulting type after applying the ! operator.

fn not(self) -> <Expr as Not>::Output

Performs the unary ! operation.

impl PartialEq<Expr> for Expr

fn eq(&self, other: &Expr) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Expr) -> bool

This method tests for !=.

impl PartialOrd<Expr> for Expr

fn partial_cmp(&self, other: &Expr) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl Rem<Expr> for Expr

type Output = Expr

The resulting type after applying the % operator.

fn rem(self, rhs: Expr) -> Expr

Performs the % operation.

impl Sub<Expr> for Expr

type Output = Expr

The resulting type after applying the - operator.

fn sub(self, rhs: Expr) -> Expr

Performs the - operation.

impl StructuralPartialEq for Expr