Module expr 

Vortex’s expression language: scalar operations over arrays.

An Expression is a tree of scalar operations rooted at a scope (see root). Expressions are the common currency of scans: a scan takes a filter expression that resolves to a boolean and a projection expression that shapes the output. All expressions are serializable and own their own wire format, so they can be pushed down to remote sources and reconstructed on workers.

Scalar functions

Each node references a scalar function defined by a ScalarFnVTable. The vtable declares the function signature, properties such as null-sensitivity, and the logic that executes it over input arrays. Built-in functions live in crate::scalar_fn; integration and plugin crates supply additional, use-case-specific functions.

Deferred execution

Applying an expression to an array does not compute the result eagerly. Instead it builds a ScalarFnArray representing the deferred application, letting downstream encodings push the computation into compressed data, or fuse several expressions together, before any data is materialized. The deferred tree is executed toward canonical form only when a result is actually required.

Typing and coercion

Expressions are strictly typed: an input array’s dtype must match the function signature exactly, so callers perform any required type coercion themselves before building the expression (see the transform passes). The one relaxation is null-coercion — for example, equality may compare a u32 against a u32?, but never a u32 against an i32.

Filter expressions are decomposed into independent conjuncts with split_conjunction so that scans can evaluate and reorder the most selective predicates first.

The implementation takes inspiration from Postgres and Apache Datafusion.

Re-exports

pub use analysis::*;

Modules

aliases

analysis

arbitrary

display

forms

proto

stats

test_harness

transform

A collection of transformations that can be applied to a crate::expr::Expression.

traversal

Datafusion inspired tree traversal logic.

Structs

ExactExpr

An expression wrapper that performs pointer equality on child expressions.

Expression

A node in a Vortex expression tree.

Traits

VortexExprExt

Functions

and

Create a new Binary using the And operator.

and_collect

Collects a list of anded values into a single expression using a balanced tree.

between

Creates an expression that checks if values are between two bounds.

byte_length

Creates an expression that computes the byte length of each element. This is akin to ANSI SQL OCTET_LENGTH(), or DuckDB’s strlen().

case_when

Creates a CASE WHEN expression with one WHEN/THEN pair and an ELSE value.

case_when_no_else

Creates a CASE WHEN expression with one WHEN/THEN pair and no ELSE value.

cast

Creates an expression that casts values to a target data type.

checked_add

Create a new Binary using the Add operator.

col

Creates an expression that accesses a field from the root array.

dynamic

Creates a dynamic comparison expression.

eq

Create a new Binary using the Eq operator.

ext_storage

Creates an expression that extracts the storage values from an extension array.

fill_null

Creates an expression that replaces null values with a fill value.

get_item

Creates an expression that extracts a named field from a struct expression.

gt

Create a new Binary using the Gt operator.

gt_eq

Create a new Binary using the Gte operator.

ilike

Creates a case-insensitive SQL ILIKE expression.

is_not_null

Creates an expression that checks for non-null values.

is_null

Creates an expression that checks for null values.

is_root

Return whether the expression is a root expression.

like

Creates a SQL LIKE expression.

list_contains

Creates an expression that checks if a value is contained in a list.

list_length

Creates an expression that computes the number of elements in each list for List and FixedSizeList inputs. This is akin to ANSI SQL CARDINALITY(), or DuckDB’s len()/array_length().

lit

Create a new Literal expression from a type that coerces to Scalar.

lt

Create a new Binary using the Lt operator.

lt_eq

Create a new Binary using the Lte operator.

mask

Creates a mask expression that applies the given boolean mask to the input array.

merge

Creates an expression that merges struct expressions into a single struct.

merge_opts

Creates a merge expression with explicit duplicate handling.

nested_case_when

Creates an n-ary CASE WHEN expression from WHEN/THEN pairs and an optional ELSE value.

not

Creates an expression that logically inverts boolean values.

not_eq

Create a new Binary using the NotEq operator.

not_ilike

Creates a negated case-insensitive SQL NOT ILIKE expression.

not_like

Creates a negated SQL NOT LIKE expression.

or

Create a new Binary using the Or operator.

or_collect

Collects a list of ored values into a single expression using a balanced tree.

pack

Creates an expression that packs values into a struct with named fields.

root

Creates an expression that references the root scope.

select

Creates an expression that selects (includes) specific fields from an array.

select_exclude

Creates an expression that excludes specific fields from an array.

split_conjunction

Splits top level and operations into separate expressions.

variant_get

Creates an expression that extracts a path from a Variant expression.

zip_expr

Creates a zip expression that conditionally selects between two arrays.