Enum datafusion::logical_expr::Expr

pub enum Expr {
    Alias(Alias),
    Column(Column),
    ScalarVariable(DataType, Vec<String>),
    Literal(ScalarValue),
    BinaryExpr(BinaryExpr),
    Like(Like),
    SimilarTo(Like),
    Not(Box<Expr>),
    IsNotNull(Box<Expr>),
    IsNull(Box<Expr>),
    IsTrue(Box<Expr>),
    IsFalse(Box<Expr>),
    IsUnknown(Box<Expr>),
    IsNotTrue(Box<Expr>),
    IsNotFalse(Box<Expr>),
    IsNotUnknown(Box<Expr>),
    Negative(Box<Expr>),
    GetIndexedField(GetIndexedField),
    Between(Between),
    Case(Case),
    Cast(Cast),
    TryCast(TryCast),
    Sort(Sort),
    ScalarFunction(ScalarFunction),
    AggregateFunction(AggregateFunction),
    WindowFunction(WindowFunction),
    InList(InList),
    Exists(Exists),
    InSubquery(InSubquery),
    ScalarSubquery(Subquery),
    Wildcard {
        qualifier: Option<String>,
    },
    GroupingSet(GroupingSet),
    Placeholder(Placeholder),
    OuterReferenceColumn(DataType, Column),
    Unnest(Unnest),
}

Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).

An Expr can compute its DataType and nullability, and has functions for building up complex expressions.

Examples

Create an expression c1 referring to column named “c1”

let expr = col("c1");
assert_eq!(expr, Expr::Column(Column::from_name("c1")));

Create the expression c1 + c2 to add columns “c1” and “c2” together

let expr = col("c1") + col("c2");

assert!(matches!(expr, Expr::BinaryExpr { ..} ));
if let Expr::BinaryExpr(binary_expr) = expr {
  assert_eq!(*binary_expr.left, col("c1"));
  assert_eq!(*binary_expr.right, col("c2"));
  assert_eq!(binary_expr.op, Operator::Plus);
}

Create expression c1 = 42 to compare the value in column “c1” to the literal value 42

let expr = col("c1").eq(lit(42_i32));

assert!(matches!(expr, Expr::BinaryExpr { .. } ));
if let Expr::BinaryExpr(binary_expr) = expr {
  assert_eq!(*binary_expr.left, col("c1"));
  let scalar = ScalarValue::Int32(Some(42));
  assert_eq!(*binary_expr.right, Expr::Literal(scalar));
  assert_eq!(binary_expr.op, Operator::Eq);
}

Return a list of Expr::Column from a schema’s columns

let arrow_schema = Schema::new(vec![
   Field::new("c1", DataType::Int32, false),
   Field::new("c2", DataType::Float64, false),
]);
let df_schema = DFSchema::try_from_qualified_schema("t1", &arrow_schema).unwrap();

// Form a list of expressions for each item in the schema
let exprs: Vec<_> = df_schema.iter()
  .map(Expr::from)
  .collect();

assert_eq!(exprs, vec![
  Expr::from(Column::from_qualified_name("t1.c1")),
  Expr::from(Column::from_qualified_name("t1.c2")),
]);

Variants

Alias(Alias)

An expression with a specific name.

Column(Column)

A named reference to a qualified filed in a schema.

ScalarVariable(DataType, Vec<String>)

A named reference to a variable in a registry.

Literal(ScalarValue)

A constant value.

BinaryExpr(BinaryExpr)

A binary expression such as “age > 21”

Like(Like)

LIKE expression

SimilarTo(Like)

LIKE expression that uses regular expressions

Not(Box<Expr>)

Negation of an expression. The expression’s type must be a boolean to make sense.

IsNotNull(Box<Expr>)

True if argument is not NULL, false otherwise. This expression itself is never NULL.

IsNull(Box<Expr>)

True if argument is NULL, false otherwise. This expression itself is never NULL.

IsTrue(Box<Expr>)

True if argument is true, false otherwise. This expression itself is never NULL.

IsFalse(Box<Expr>)

True if argument is false, false otherwise. This expression itself is never NULL.

IsUnknown(Box<Expr>)

True if argument is NULL, false otherwise. This expression itself is never NULL.

IsNotTrue(Box<Expr>)

True if argument is FALSE or NULL, false otherwise. This expression itself is never NULL.

IsNotFalse(Box<Expr>)

True if argument is TRUE OR NULL, false otherwise. This expression itself is never NULL.

IsNotUnknown(Box<Expr>)

True if argument is TRUE or FALSE, false otherwise. This expression itself is never NULL.

Negative(Box<Expr>)

arithmetic negation of an expression, the operand must be of a signed numeric data type

GetIndexedField(GetIndexedField)

Returns the field of a arrow::array::ListArray or arrow::array::StructArray by index or range

Between(Between)

Whether an expression is between a given range.

Case(Case)

The CASE expression is similar to a series of nested if/else and there are two forms that can be used. The first form consists of a series of boolean “when” expressions with corresponding “then” expressions, and an optional “else” expression.

CASE WHEN condition THEN result [WHEN …] [ELSE result] END

The second form uses a base expression and then a series of “when” clauses that match on a literal value.

CASE expression WHEN value THEN result [WHEN …] [ELSE result] END

Cast(Cast)

Casts the expression to a given type and will return a runtime error if the expression cannot be cast. This expression is guaranteed to have a fixed type.

TryCast(TryCast)

Casts the expression to a given type and will return a null value if the expression cannot be cast. This expression is guaranteed to have a fixed type.

Sort(Sort)

A sort expression, that can be used to sort values.

ScalarFunction(ScalarFunction)

Represents the call of a scalar function with a set of arguments.

AggregateFunction(AggregateFunction)

Represents the call of an aggregate built-in function with arguments.

WindowFunction(WindowFunction)

Represents the call of a window function with arguments.

InList(InList)

Returns whether the list contains the expr value.

Exists(Exists)

EXISTS subquery

InSubquery(InSubquery)

IN subquery

ScalarSubquery(Subquery)

Scalar subquery

Wildcard

Represents a reference to all available fields in a specific schema, with an optional (schema) qualifier.

This expr has to be resolved to a list of columns before translating logical plan into physical plan.

Fields

qualifier: Option<String>

GroupingSet(GroupingSet)

List of grouping set expressions. Only valid in the context of an aggregate GROUP BY expression list

Placeholder(Placeholder)

A place holder for parameters in a prepared statement (e.g. $foo or $1)

OuterReferenceColumn(DataType, Column)

A place holder which hold a reference to a qualified field in the outer query, used for correlated sub queries.

Unnest(Unnest)

Unnest expression

Implementations

impl Expr

pub fn display_name(&self) -> Result<String, DataFusionError>

Returns the name of this expression as it should appear in a schema. This name will not include any CAST expressions.

pub fn canonical_name(&self) -> String

Returns a full and complete string representation of this expression.

pub fn variant_name(&self) -> &str

Return String representation of the variant represented by self Useful for non-rust based bindings

pub fn eq(self, other: Expr) -> Expr

Return self == other

pub fn not_eq(self, other: Expr) -> Expr

Return self != other

pub fn gt(self, other: Expr) -> Expr

Return self > other

pub fn gt_eq(self, other: Expr) -> Expr

Return self >= other

pub fn lt(self, other: Expr) -> Expr

Return self < other

pub fn lt_eq(self, other: Expr) -> Expr

Return self <= other

pub fn and(self, other: Expr) -> Expr

Return self && other

pub fn or(self, other: Expr) -> Expr

Return self || other

pub fn like(self, other: Expr) -> Expr

Return self LIKE other

pub fn not_like(self, other: Expr) -> Expr

Return self NOT LIKE other

pub fn ilike(self, other: Expr) -> Expr

Return self ILIKE other

pub fn not_ilike(self, other: Expr) -> Expr

Return self NOT ILIKE other

pub fn name_for_alias(&self) -> Result<String, DataFusionError>

Return the name to use for the specific Expr, recursing into Expr::Sort as appropriate

pub fn alias_if_changed( self, original_name: String ) -> Result<Expr, DataFusionError>

Ensure expr has the name as original_name by adding an alias if necessary.

pub fn alias(self, name: impl Into<String>) -> Expr

Return self AS name alias expression

pub fn alias_qualified( self, relation: Option<impl Into<TableReference>>, name: impl Into<String> ) -> Expr

Return self AS name alias expression with a specific qualifier

pub fn unalias(self) -> Expr

Remove an alias from an expression if one exists.

pub fn in_list(self, list: Vec<Expr>, negated: bool) -> Expr

Return self IN <list> if negated is false, otherwise return self NOT IN <list>.a

pub fn is_null(self) -> Expr

Return `IsNull(Box(self))

pub fn is_not_null(self) -> Expr

Return `IsNotNull(Box(self))

pub fn sort(self, asc: bool, nulls_first: bool) -> Expr

Create a sort expression from an existing expression.

let sort_expr = col("foo").sort(true, true); // SORT ASC NULLS_FIRST

pub fn is_true(self) -> Expr

Return IsTrue(Box(self))

pub fn is_not_true(self) -> Expr

Return IsNotTrue(Box(self))

pub fn is_false(self) -> Expr

Return IsFalse(Box(self))

pub fn is_not_false(self) -> Expr

Return IsNotFalse(Box(self))

pub fn is_unknown(self) -> Expr

Return IsUnknown(Box(self))

pub fn is_not_unknown(self) -> Expr

Return IsNotUnknown(Box(self))

pub fn between(self, low: Expr, high: Expr) -> Expr

return self BETWEEN low AND high

pub fn not_between(self, low: Expr, high: Expr) -> Expr

return self NOT BETWEEN low AND high

pub fn field(self, name: impl Into<String>) -> Expr

Return access to the named field. Example expr["name"]

Access field “my_field” from column “c1”

For example if column “c1” holds documents like this

{
  "my_field": 123.34,
  "other_field": "Boston",
}

You can access column “my_field” with

let expr = col("c1")
   .field("my_field");
assert_eq!(expr.display_name().unwrap(), "c1[my_field]");

pub fn index(self, key: Expr) -> Expr

Return access to the element field. Example expr["name"]

Example Access element 2 from column “c1”

For example if column “c1” holds documents like this

[10, 20, 30, 40]

You can access the value “30” with

let expr = col("c1")
   .index(lit(3));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(3)]");

pub fn range(self, start: Expr, stop: Expr) -> Expr

Return elements between 1 based start and stop, for example expr[1:3]

Example: Access element 2, 3, 4 from column “c1”

For example if column “c1” holds documents like this

[10, 20, 30, 40]

You can access the value [20, 30, 40] with

let expr = col("c1")
   .range(lit(2), lit(4));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(2):Int32(4):Int64(1)]");

pub fn try_into_col(&self) -> Result<Column, DataFusionError>

pub fn to_columns(&self) -> Result<HashSet<Column>, DataFusionError>

Return all referenced columns of this expression.

pub fn contains_outer(&self) -> bool

Return true when the expression contains out reference(correlated) expressions.

pub fn is_volatile(&self) -> Result<bool, DataFusionError>

Returns true if the expression is volatile, i.e. whether it can return different results when evaluated multiple times with the same input.

pub fn infer_placeholder_types( self, schema: &DFSchema ) -> Result<Expr, DataFusionError>

Recursively find all Expr::Placeholder expressions, and to infer their DataType from the context of their use.

For example, gicen an expression like <int32> = $0 will infer $0 to have type int32.

pub fn short_circuits(&self) -> bool

Returns true if some of this exprs subexpressions may not be evaluated and thus any side effects (like divide by zero) may not be encountered

Trait Implementations

impl Add for Expr

Support <expr> + <expr> fluent style

type Output = Expr

The resulting type after applying the + operator.

fn add(self, rhs: Expr) -> Expr

Performs the + operation.

impl BitAnd for Expr

Support <expr> & <expr> fluent style

type Output = Expr

The resulting type after applying the & operator.

fn bitand(self, rhs: Expr) -> Expr

Performs the & operation.

impl BitOr for Expr

Support <expr> | <expr> fluent style

type Output = Expr

The resulting type after applying the | operator.

fn bitor(self, rhs: Expr) -> Expr

Performs the | operation.

impl BitXor for Expr

Support <expr> ^ <expr> fluent style

type Output = Expr

The resulting type after applying the ^ operator.

fn bitxor(self, rhs: Expr) -> Expr

Performs the ^ operation.

impl Clone for Expr

fn clone(&self) -> Expr

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Expr

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for Expr

fn default() -> Expr

Returns the “default value” for a type.

impl Display for Expr

Format expressions for display as part of a logical plan. In many cases, this will produce similar output to Expr.name() except that column names will be prefixed with ‘#’.

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Div for Expr

Support <expr> / <expr> fluent style

type Output = Expr

The resulting type after applying the / operator.

fn div(self, rhs: Expr) -> Expr

Performs the / operation.

impl ExprSchemable for Expr

fn get_type(&self, schema: &dyn ExprSchema) -> Result<DataType, DataFusionError>

Returns the arrow::datatypes::DataType of the expression based on ExprSchema

Note: DFSchema implements ExprSchema.

Examples

Get the type of an expression that adds 2 columns. Adding an Int32

and Float32 results in Float32 type

fn main() {
  let expr = col("c1") + col("c2");
  let schema = DFSchema::from_unqualifed_fields(
    vec![
      Field::new("c1", DataType::Int32, true),
      Field::new("c2", DataType::Float32, true),
      ].into(),
      HashMap::new(),
  ).unwrap();
  assert_eq!("Float32", format!("{}", expr.get_type(&schema).unwrap()));
}

Errors

This function errors when it is not possible to compute its arrow::datatypes::DataType. This happens when e.g. the expression refers to a column that does not exist in the schema, or when the expression is incorrectly typed (e.g. [utf8] + [bool]).

fn nullable( &self, input_schema: &dyn ExprSchema ) -> Result<bool, DataFusionError>

Returns the nullability of the expression based on ExprSchema.

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its nullability. This happens when the expression refers to a column that does not exist in the schema.

fn data_type_and_nullable( &self, schema: &dyn ExprSchema ) -> Result<(DataType, bool), DataFusionError>

Returns the datatype and nullability of the expression based on ExprSchema.

Note: DFSchema implements ExprSchema.

Errors

This function errors when it is not possible to compute its datatype or nullability.

fn to_field( &self, input_schema: &dyn ExprSchema ) -> Result<(Option<TableReference>, Arc<Field>), DataFusionError>

Returns a arrow::datatypes::Field compatible with this expression.

So for example, a projected expression col(c1) + col(c2) is placed in an output field named col(“c1 + c2”)

fn cast_to( self, cast_to_type: &DataType, schema: &dyn ExprSchema ) -> Result<Expr, DataFusionError>

Wraps this expression in a cast to a target arrow::datatypes::DataType.

Errors

This function errors when it is impossible to cast the expression to the target arrow::datatypes::DataType.

fn metadata( &self, schema: &dyn ExprSchema ) -> Result<HashMap<String, String>, DataFusionError>

given a schema, return the expr’s optional metadata

impl<'a> From<(Option<&'a TableReference>, &'a Arc<Field>)> for Expr

Create an Expr from an optional qualifier and a FieldRef. This is useful for creating Expr from a DFSchema.

See example on Expr

fn from(value: (Option<&'a TableReference>, &'a Arc<Field>)) -> Expr

Converts to this type from the input type.

impl From<Column> for Expr

Create an Expr from a Column

fn from(value: Column) -> Expr

Converts to this type from the input type.

impl Hash for Expr

fn hash<__H>(&self, state: &mut __H)

where __H: Hasher,

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Mul for Expr

Support <expr> * <expr> fluent style

type Output = Expr

The resulting type after applying the * operator.

fn mul(self, rhs: Expr) -> Expr

Performs the * operation.

impl Neg for Expr

Support - <expr> fluent style

type Output = Expr

The resulting type after applying the - operator.

fn neg(self) -> <Expr as Neg>::Output

Performs the unary - operation.

impl Not for Expr

Support NOT <expr> fluent style

type Output = Expr

The resulting type after applying the ! operator.

fn not(self) -> <Expr as Not>::Output

Performs the unary ! operation.

impl PartialEq for Expr

fn eq(&self, other: &Expr) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for Expr

fn partial_cmp(&self, other: &Expr) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl Rem for Expr

Support <expr> % <expr> fluent style

type Output = Expr

The resulting type after applying the % operator.

fn rem(self, rhs: Expr) -> Expr

Performs the % operation.

impl Shl for Expr

Support <expr> << <expr> fluent style

type Output = Expr

The resulting type after applying the << operator.

fn shl(self, rhs: Expr) -> <Expr as Shl>::Output

Performs the << operation.

impl Shr for Expr

Support <expr> >> <expr> fluent style

type Output = Expr

The resulting type after applying the >> operator.

fn shr(self, rhs: Expr) -> <Expr as Shr>::Output

Performs the >> operation.

impl Sub for Expr

Support <expr> - <expr> fluent style

type Output = Expr

The resulting type after applying the - operator.

fn sub(self, rhs: Expr) -> Expr

Performs the - operation.

impl TreeNode for Expr

fn apply_children<F>(&self, f: F) -> Result<TreeNodeRecursion, DataFusionError>

where F: FnMut(&Expr) -> Result<TreeNodeRecursion, DataFusionError>,

Low-level API used to implement other APIs.

fn map_children<F>(self, f: F) -> Result<Transformed<Expr>, DataFusionError>

where F: FnMut(Expr) -> Result<Transformed<Expr>, DataFusionError>,

Low-level API used to implement other APIs.

fn visit<V>( &self, visitor: &mut V ) -> Result<TreeNodeRecursion, DataFusionError>

where V: TreeNodeVisitor<Node = Self>,

Visit the tree node with a TreeNodeVisitor, performing a depth-first walk of the node and its children.

fn rewrite<R>( self, rewriter: &mut R ) -> Result<Transformed<Self>, DataFusionError>

where R: TreeNodeRewriter<Node = Self>,

Rewrite the tree node with a TreeNodeRewriter, performing a depth-first walk of the node and its children.

fn apply<F>(&self, f: F) -> Result<TreeNodeRecursion, DataFusionError>

where F: FnMut(&Self) -> Result<TreeNodeRecursion, DataFusionError>,

Applies f to the node then each of its children, recursively (a top-down, pre-order traversal).

fn transform<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>

where F: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

Recursively rewrite the node’s children and then the node using f (a bottom-up post-order traversal).

fn transform_down<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>

where F: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

Recursively rewrite the tree using f in a top-down (pre-order) fashion.

fn transform_down_mut<F>( self, f: &mut F ) -> Result<Transformed<Self>, DataFusionError>

where F: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

👎Deprecated since 38.0.0: Use transform_down instead

Same as Self::transform_down but with a mutable closure.

fn transform_up<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>

where F: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

Recursively rewrite the node using f in a bottom-up (post-order) fashion.

fn transform_up_mut<F>( self, f: &mut F ) -> Result<Transformed<Self>, DataFusionError>

where F: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

👎Deprecated since 38.0.0: Use transform_up instead

Same as Self::transform_up but with a mutable closure.

fn transform_down_up<FD, FU>( self, f_down: FD, f_up: FU ) -> Result<Transformed<Self>, DataFusionError>

where FD: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>, FU: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,

Transforms the node using f_down while traversing the tree top-down (pre-order), and using f_up while traversing the tree bottom-up (post-order).

fn exists<F>(&self, f: F) -> Result<bool, DataFusionError>

where F: FnMut(&Self) -> Result<bool, DataFusionError>,

Returns true if f returns true for any node in the tree.

impl Eq for Expr

impl StructuralPartialEq for Expr