Enum datafusion_expr::expr::Expr
source · pub enum Expr {
Show 35 variants
Alias(Alias),
Column(Column),
ScalarVariable(DataType, Vec<String>),
Literal(ScalarValue),
BinaryExpr(BinaryExpr),
Like(Like),
SimilarTo(Like),
Not(Box<Expr>),
IsNotNull(Box<Expr>),
IsNull(Box<Expr>),
IsTrue(Box<Expr>),
IsFalse(Box<Expr>),
IsUnknown(Box<Expr>),
IsNotTrue(Box<Expr>),
IsNotFalse(Box<Expr>),
IsNotUnknown(Box<Expr>),
Negative(Box<Expr>),
GetIndexedField(GetIndexedField),
Between(Between),
Case(Case),
Cast(Cast),
TryCast(TryCast),
Sort(Sort),
ScalarFunction(ScalarFunction),
AggregateFunction(AggregateFunction),
WindowFunction(WindowFunction),
InList(InList),
Exists(Exists),
InSubquery(InSubquery),
ScalarSubquery(Subquery),
Wildcard {
qualifier: Option<String>,
},
GroupingSet(GroupingSet),
Placeholder(Placeholder),
OuterReferenceColumn(DataType, Column),
Unnest(Unnest),
}
Expand description
Expr
is a central struct of DataFusion’s query API, and
represent logical expressions such as A + 1
, or CAST(c1 AS int)
.
An Expr
can compute its DataType
and nullability, and has functions for building up complex
expressions.
§Examples
§Create an expression c1
referring to column named “c1”
let expr = col("c1");
assert_eq!(expr, Expr::Column(Column::from_name("c1")));
§Create the expression c1 + c2
to add columns “c1” and “c2” together
let expr = col("c1") + col("c2");
assert!(matches!(expr, Expr::BinaryExpr { ..} ));
if let Expr::BinaryExpr(binary_expr) = expr {
assert_eq!(*binary_expr.left, col("c1"));
assert_eq!(*binary_expr.right, col("c2"));
assert_eq!(binary_expr.op, Operator::Plus);
}
§Create expression c1 = 42
to compare the value in column “c1” to the literal value 42
let expr = col("c1").eq(lit(42_i32));
assert!(matches!(expr, Expr::BinaryExpr { .. } ));
if let Expr::BinaryExpr(binary_expr) = expr {
assert_eq!(*binary_expr.left, col("c1"));
let scalar = ScalarValue::Int32(Some(42));
assert_eq!(*binary_expr.right, Expr::Literal(scalar));
assert_eq!(binary_expr.op, Operator::Eq);
}
§Return a list of Expr::Column
from a schema’s columns
let arrow_schema = Schema::new(vec![
Field::new("c1", DataType::Int32, false),
Field::new("c2", DataType::Float64, false),
]);
let df_schema = DFSchema::try_from_qualified_schema("t1", &arrow_schema).unwrap();
// Form a list of expressions for each item in the schema
let exprs: Vec<_> = df_schema.iter()
.map(Expr::from)
.collect();
assert_eq!(exprs, vec![
Expr::from(Column::from_qualified_name("t1.c1")),
Expr::from(Column::from_qualified_name("t1.c2")),
]);
Variants§
Alias(Alias)
An expression with a specific name.
Column(Column)
A named reference to a qualified filed in a schema.
ScalarVariable(DataType, Vec<String>)
A named reference to a variable in a registry.
Literal(ScalarValue)
A constant value.
BinaryExpr(BinaryExpr)
A binary expression such as “age > 21”
Like(Like)
LIKE expression
SimilarTo(Like)
LIKE expression that uses regular expressions
Not(Box<Expr>)
Negation of an expression. The expression’s type must be a boolean to make sense.
IsNotNull(Box<Expr>)
True if argument is not NULL, false otherwise. This expression itself is never NULL.
IsNull(Box<Expr>)
True if argument is NULL, false otherwise. This expression itself is never NULL.
IsTrue(Box<Expr>)
True if argument is true, false otherwise. This expression itself is never NULL.
IsFalse(Box<Expr>)
True if argument is false, false otherwise. This expression itself is never NULL.
IsUnknown(Box<Expr>)
True if argument is NULL, false otherwise. This expression itself is never NULL.
IsNotTrue(Box<Expr>)
True if argument is FALSE or NULL, false otherwise. This expression itself is never NULL.
IsNotFalse(Box<Expr>)
True if argument is TRUE OR NULL, false otherwise. This expression itself is never NULL.
IsNotUnknown(Box<Expr>)
True if argument is TRUE or FALSE, false otherwise. This expression itself is never NULL.
Negative(Box<Expr>)
arithmetic negation of an expression, the operand must be of a signed numeric data type
GetIndexedField(GetIndexedField)
Returns the field of a arrow::array::ListArray
or
arrow::array::StructArray
by index or range
Between(Between)
Whether an expression is between a given range.
Case(Case)
The CASE expression is similar to a series of nested if/else and there are two forms that can be used. The first form consists of a series of boolean “when” expressions with corresponding “then” expressions, and an optional “else” expression.
CASE WHEN condition THEN result [WHEN …] [ELSE result] END
The second form uses a base expression and then a series of “when” clauses that match on a literal value.
CASE expression WHEN value THEN result [WHEN …] [ELSE result] END
Cast(Cast)
Casts the expression to a given type and will return a runtime error if the expression cannot be cast. This expression is guaranteed to have a fixed type.
TryCast(TryCast)
Casts the expression to a given type and will return a null value if the expression cannot be cast. This expression is guaranteed to have a fixed type.
Sort(Sort)
A sort expression, that can be used to sort values.
ScalarFunction(ScalarFunction)
Represents the call of a scalar function with a set of arguments.
AggregateFunction(AggregateFunction)
Represents the call of an aggregate built-in function with arguments.
WindowFunction(WindowFunction)
Represents the call of a window function with arguments.
InList(InList)
Returns whether the list contains the expr value.
Exists(Exists)
EXISTS subquery
InSubquery(InSubquery)
IN subquery
ScalarSubquery(Subquery)
Scalar subquery
Wildcard
Represents a reference to all available fields in a specific schema, with an optional (schema) qualifier.
This expr has to be resolved to a list of columns before translating logical plan into physical plan.
GroupingSet(GroupingSet)
List of grouping set expressions. Only valid in the context of an aggregate GROUP BY expression list
Placeholder(Placeholder)
A place holder for parameters in a prepared statement
(e.g. $foo
or $1
)
OuterReferenceColumn(DataType, Column)
A place holder which hold a reference to a qualified field in the outer query, used for correlated sub queries.
Unnest(Unnest)
Unnest expression
Implementations§
source§impl Expr
impl Expr
sourcepub fn display_name(&self) -> Result<String>
pub fn display_name(&self) -> Result<String>
Returns the name of this expression as it should appear in a schema. This name will not include any CAST expressions.
sourcepub fn canonical_name(&self) -> String
pub fn canonical_name(&self) -> String
Returns a full and complete string representation of this expression.
sourcepub fn variant_name(&self) -> &str
pub fn variant_name(&self) -> &str
Return String representation of the variant represented by self
Useful for non-rust based bindings
sourcepub fn name_for_alias(&self) -> Result<String>
pub fn name_for_alias(&self) -> Result<String>
Return the name to use for the specific Expr, recursing into
Expr::Sort
as appropriate
sourcepub fn alias_if_changed(self, original_name: String) -> Result<Expr>
pub fn alias_if_changed(self, original_name: String) -> Result<Expr>
Ensure expr
has the name as original_name
by adding an
alias if necessary.
sourcepub fn alias_qualified(
self,
relation: Option<impl Into<TableReference>>,
name: impl Into<String>
) -> Expr
pub fn alias_qualified( self, relation: Option<impl Into<TableReference>>, name: impl Into<String> ) -> Expr
Return self AS name
alias expression with a specific qualifier
sourcepub fn in_list(self, list: Vec<Expr>, negated: bool) -> Expr
pub fn in_list(self, list: Vec<Expr>, negated: bool) -> Expr
Return self IN <list>
if negated
is false, otherwise
return self NOT IN <list>
.a
sourcepub fn is_not_null(self) -> Expr
pub fn is_not_null(self) -> Expr
Return `IsNotNull(Box(self))
sourcepub fn sort(self, asc: bool, nulls_first: bool) -> Expr
pub fn sort(self, asc: bool, nulls_first: bool) -> Expr
Create a sort expression from an existing expression.
let sort_expr = col("foo").sort(true, true); // SORT ASC NULLS_FIRST
sourcepub fn is_not_true(self) -> Expr
pub fn is_not_true(self) -> Expr
Return IsNotTrue(Box(self))
sourcepub fn is_not_false(self) -> Expr
pub fn is_not_false(self) -> Expr
Return IsNotFalse(Box(self))
sourcepub fn is_unknown(self) -> Expr
pub fn is_unknown(self) -> Expr
Return IsUnknown(Box(self))
sourcepub fn is_not_unknown(self) -> Expr
pub fn is_not_unknown(self) -> Expr
Return IsNotUnknown(Box(self))
sourcepub fn not_between(self, low: Expr, high: Expr) -> Expr
pub fn not_between(self, low: Expr, high: Expr) -> Expr
return self NOT BETWEEN low AND high
sourcepub fn field(self, name: impl Into<String>) -> Self
pub fn field(self, name: impl Into<String>) -> Self
Return access to the named field. Example expr["name"]
§Access field “my_field” from column “c1”
For example if column “c1” holds documents like this
{
"my_field": 123.34,
"other_field": "Boston",
}
You can access column “my_field” with
let expr = col("c1")
.field("my_field");
assert_eq!(expr.display_name().unwrap(), "c1[my_field]");
sourcepub fn index(self, key: Expr) -> Self
pub fn index(self, key: Expr) -> Self
Return access to the element field. Example expr["name"]
§Example Access element 2 from column “c1”
For example if column “c1” holds documents like this
[10, 20, 30, 40]
You can access the value “30” with
let expr = col("c1")
.index(lit(3));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(3)]");
sourcepub fn range(self, start: Expr, stop: Expr) -> Self
pub fn range(self, start: Expr, stop: Expr) -> Self
Return elements between 1
based start
and stop
, for
example expr[1:3]
§Example: Access element 2, 3, 4 from column “c1”
For example if column “c1” holds documents like this
[10, 20, 30, 40]
You can access the value [20, 30, 40]
with
let expr = col("c1")
.range(lit(2), lit(4));
assert_eq!(expr.display_name().unwrap(), "c1[Int32(2):Int32(4):Int64(1)]");
pub fn try_into_col(&self) -> Result<Column>
sourcepub fn to_columns(&self) -> Result<HashSet<Column>>
pub fn to_columns(&self) -> Result<HashSet<Column>>
Return all referenced columns of this expression.
sourcepub fn contains_outer(&self) -> bool
pub fn contains_outer(&self) -> bool
Return true when the expression contains out reference(correlated) expressions.
sourcepub fn is_volatile(&self) -> Result<bool>
pub fn is_volatile(&self) -> Result<bool>
Returns true if the expression is volatile, i.e. whether it can return different results when evaluated multiple times with the same input.
sourcepub fn infer_placeholder_types(self, schema: &DFSchema) -> Result<Expr>
pub fn infer_placeholder_types(self, schema: &DFSchema) -> Result<Expr>
Recursively find all Expr::Placeholder
expressions, and
to infer their DataType
from the context of their use.
For example, gicen an expression like <int32> = $0
will infer $0
to
have type int32
.
sourcepub fn short_circuits(&self) -> bool
pub fn short_circuits(&self) -> bool
Returns true if some of this exprs
subexpressions may not be evaluated
and thus any side effects (like divide by zero) may not be encountered
Trait Implementations§
source§impl Display for Expr
impl Display for Expr
Format expressions for display as part of a logical plan. In many cases, this will produce
similar output to Expr.name()
except that column names will be prefixed with ‘#’.
source§impl ExprSchemable for Expr
impl ExprSchemable for Expr
source§fn get_type(&self, schema: &dyn ExprSchema) -> Result<DataType>
fn get_type(&self, schema: &dyn ExprSchema) -> Result<DataType>
Returns the arrow::datatypes::DataType of the expression based on ExprSchema
Note: DFSchema
implements ExprSchema.
§Examples
§Get the type of an expression that adds 2 columns. Adding an Int32
§and Float32 results in Float32 type
fn main() {
let expr = col("c1") + col("c2");
let schema = DFSchema::from_unqualifed_fields(
vec![
Field::new("c1", DataType::Int32, true),
Field::new("c2", DataType::Float32, true),
].into(),
HashMap::new(),
).unwrap();
assert_eq!("Float32", format!("{}", expr.get_type(&schema).unwrap()));
}
§Errors
This function errors when it is not possible to compute its
arrow::datatypes::DataType. This happens when e.g. the
expression refers to a column that does not exist in the
schema, or when the expression is incorrectly typed
(e.g. [utf8] + [bool]
).
source§fn nullable(&self, input_schema: &dyn ExprSchema) -> Result<bool>
fn nullable(&self, input_schema: &dyn ExprSchema) -> Result<bool>
Returns the nullability of the expression based on ExprSchema.
Note: DFSchema
implements ExprSchema.
§Errors
This function errors when it is not possible to compute its nullability. This happens when the expression refers to a column that does not exist in the schema.
source§fn data_type_and_nullable(
&self,
schema: &dyn ExprSchema
) -> Result<(DataType, bool)>
fn data_type_and_nullable( &self, schema: &dyn ExprSchema ) -> Result<(DataType, bool)>
Returns the datatype and nullability of the expression based on ExprSchema.
Note: DFSchema
implements ExprSchema.
§Errors
This function errors when it is not possible to compute its datatype or nullability.
source§fn to_field(
&self,
input_schema: &dyn ExprSchema
) -> Result<(Option<TableReference>, Arc<Field>)>
fn to_field( &self, input_schema: &dyn ExprSchema ) -> Result<(Option<TableReference>, Arc<Field>)>
Returns a arrow::datatypes::Field compatible with this expression.
So for example, a projected expression col(c1) + col(c2)
is
placed in an output field named col(“c1 + c2”)
source§fn cast_to(
self,
cast_to_type: &DataType,
schema: &dyn ExprSchema
) -> Result<Expr>
fn cast_to( self, cast_to_type: &DataType, schema: &dyn ExprSchema ) -> Result<Expr>
Wraps this expression in a cast to a target arrow::datatypes::DataType.
§Errors
This function errors when it is impossible to cast the expression to the target arrow::datatypes::DataType.
source§impl<'a> From<(Option<&'a TableReference>, &'a Arc<Field>)> for Expr
impl<'a> From<(Option<&'a TableReference>, &'a Arc<Field>)> for Expr
source§impl PartialEq for Expr
impl PartialEq for Expr
source§impl PartialOrd for Expr
impl PartialOrd for Expr
1.0.0 · source§fn le(&self, other: &Rhs) -> bool
fn le(&self, other: &Rhs) -> bool
self
and other
) and is used by the <=
operator. Read moresource§impl TreeNode for Expr
impl TreeNode for Expr
source§fn apply_children<F: FnMut(&Self) -> Result<TreeNodeRecursion>>(
&self,
f: F
) -> Result<TreeNodeRecursion>
fn apply_children<F: FnMut(&Self) -> Result<TreeNodeRecursion>>( &self, f: F ) -> Result<TreeNodeRecursion>
source§fn map_children<F: FnMut(Self) -> Result<Transformed<Self>>>(
self,
f: F
) -> Result<Transformed<Self>>
fn map_children<F: FnMut(Self) -> Result<Transformed<Self>>>( self, f: F ) -> Result<Transformed<Self>>
source§fn visit<V>(
&self,
visitor: &mut V
) -> Result<TreeNodeRecursion, DataFusionError>where
V: TreeNodeVisitor<Node = Self>,
fn visit<V>(
&self,
visitor: &mut V
) -> Result<TreeNodeRecursion, DataFusionError>where
V: TreeNodeVisitor<Node = Self>,
TreeNodeVisitor
, performing a
depth-first walk of the node and its children. Read moresource§fn rewrite<R>(
self,
rewriter: &mut R
) -> Result<Transformed<Self>, DataFusionError>where
R: TreeNodeRewriter<Node = Self>,
fn rewrite<R>(
self,
rewriter: &mut R
) -> Result<Transformed<Self>, DataFusionError>where
R: TreeNodeRewriter<Node = Self>,
TreeNodeRewriter
, performing a
depth-first walk of the node and its children. Read moresource§fn apply<F>(&self, f: F) -> Result<TreeNodeRecursion, DataFusionError>
fn apply<F>(&self, f: F) -> Result<TreeNodeRecursion, DataFusionError>
f
to the node then each of its children, recursively (a
top-down, pre-order traversal). Read moresource§fn transform<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
fn transform<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
f
(a bottom-up post-order traversal). Read moresource§fn transform_down<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
fn transform_down<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
f
in a top-down (pre-order)
fashion. Read moresource§fn transform_down_mut<F>(
self,
f: &mut F
) -> Result<Transformed<Self>, DataFusionError>
fn transform_down_mut<F>( self, f: &mut F ) -> Result<Transformed<Self>, DataFusionError>
transform_down
insteadSelf::transform_down
but with a mutable closure.source§fn transform_up<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
fn transform_up<F>(self, f: F) -> Result<Transformed<Self>, DataFusionError>
f
in a bottom-up (post-order)
fashion. Read moresource§fn transform_up_mut<F>(
self,
f: &mut F
) -> Result<Transformed<Self>, DataFusionError>
fn transform_up_mut<F>( self, f: &mut F ) -> Result<Transformed<Self>, DataFusionError>
transform_up
insteadSelf::transform_up
but with a mutable closure.source§fn transform_down_up<FD, FU>(
self,
f_down: FD,
f_up: FU
) -> Result<Transformed<Self>, DataFusionError>where
FD: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,
FU: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,
fn transform_down_up<FD, FU>(
self,
f_down: FD,
f_up: FU
) -> Result<Transformed<Self>, DataFusionError>where
FD: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,
FU: FnMut(Self) -> Result<Transformed<Self>, DataFusionError>,
f_down
while traversing the tree top-down
(pre-order), and using f_up
while traversing the tree bottom-up
(post-order). Read moreimpl Eq for Expr
impl StructuralPartialEq for Expr
Auto Trait Implementations§
impl Freeze for Expr
impl !RefUnwindSafe for Expr
impl Send for Expr
impl Sync for Expr
impl Unpin for Expr
impl !UnwindSafe for Expr
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.