Module datafusion::prelude

A “prelude” for users of the datafusion crate.

Like the standard library’s prelude, this module simplifies importing of common items. Unlike the standard prelude, the contents of this module must be imported manually:

use datafusion::prelude::*;

Re-exports

pub use crate::dataframe::DataFrame;

pub use crate::execution::context::SessionConfig;

pub use crate::execution::context::SessionContext;

pub use crate::execution::options::AvroReadOptions;

pub use crate::execution::options::CsvReadOptions;

pub use crate::execution::options::NdJsonReadOptions;

pub use crate::execution::options::ParquetReadOptions;

Structs

Column

A named reference to a qualified field in a schema.

Enums

Expr

Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).

JoinType

Join type

Partitioning

Logical partitioning schemes supported by the repartition operator.

Functions

abs

absolute value

acos

inverse cosine

and

Return a new expression with a logical AND

approx_distinct

Returns the approximate number of distinct input values. This function provides an approximation of count(DISTINCT x). Zero is returned if all input values are null. This function should produce a standard error of 0.81%, which is the standard deviation of the (approximately normal) error distribution over all possible sets. It does not guarantee an upper bound on the error for any specific input set.

approx_median

Calculate an approximation of the median for expr.

approx_percentile_cont

Calculate an approximation of the specified percentile for expr.

approx_percentile_cont_with_weight

Calculate an approximation of the specified percentile for expr and weight_expr.

array

returns an array of fixed size with each argument on it.

arrow_typeof

data type

ascii

ASCII code value of the character

asin

inverse sine

atan

inverse tangent

atan2

inverse tangent of a division given in the argument

avg

Create an expression to represent the avg() aggregate function

binary_expr

Return a new expression left <op> right

bit_length

the number of bits in the string

btrim

removes all characters, spaces by default, from both sides of a string

call_fn

Calls a named built in function

case

Create a CASE WHEN statement with literal WHEN expressions for comparison to the base expression.

cast

Create a cast expression

ceil

nearest integer greater than or equal to argument

character_length

the number of characters in the string

chr

converts the Unicode code point to a UTF8 character

coalesce

returns coalesce(args...), which evaluates to the value of the first Expr which is not NULL

col

Create a column expression based on a qualified or unqualified column name

concat

Concatenates the text representations of all the arguments. NULL arguments are ignored.

concat_expr

concatenates several strings

concat_ws

Concatenates all but the first argument, with separators. The first argument is used as the separator. NULL arguments in values are ignored.

concat_ws_expr

concatenates several strings, placing a seperator between each one

cos

cosine

count

Create an expression to represent the count() aggregate function

count_distinct

Create an expression to represent the count(distinct) aggregate function

create_udaf

Creates a new UDAF with a specific signature, state type and return type. The signature and state type must match the Accumulator's implementation.

create_udf

Creates a new UDF with a specific signature and specific return type. This is a helper function to create a new UDF. The function create_udf returns a subset of all possible ScalarFunction:

cube

Create a grouping set for all combination of exprs

current_date

returns current UTC date as a [DataType::Date32] value

current_time

returns current UTC time as a [DataType::Time64] value

date_bin

coerces an arbitrary timestamp to the start of the nearest specified interval

date_part

extracts a subfield from the date

date_trunc

truncates the date to a specified level of precision

digest

compute the binary hash of input, using the algorithm

exists

Create an EXISTS subquery expression

exp

exponential

floor

nearest integer less than or equal to argument

from_unixtime

returns the unix time in format

grouping_set

Create a grouping set

in_list

Create an in_list expression

in_subquery

Create an IN subquery expression

initcap

converts the first letter of each word in string in uppercase and the remaining characters in lowercase

is_false

Create is false expression

is_not_false

Create is not false expression

is_not_true

Create is not true expression

is_not_unknown

Create is not unknown expression

is_null

Create is null expression

is_true

Create is true expression

is_unknown

Create is unknown expression

left

returns the first n characters in the string

lit

Create a literal expression

lit_timestamp_nano

Create a literal timestamp expression

ln

natural logarithm

log2

base 2 logarithm

log10

base 10 logarithm

lower

convert the string to lower case

lpad

fill up a string to the length by prepending the characters

ltrim

removes all characters, spaces by default, from the beginning of a string

max

Create an expression to represent the max() aggregate function

md5

returns the MD5 hash of a string

min

Create an expression to represent the min() aggregate function

not_exists

Create a NOT EXISTS subquery expression

not_in_subquery

Create a NOT IN subquery expression

now

returns current timestamp in nanoseconds, using the same value for all instances of now() in same statement

nullif

returns NULL if value1 equals value2; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE expression.

octet_length

returns the number of bytes of a string

or

Return a new expression with a logical OR

power

base raised to the power of exponent

random

Returns a random value in the range 0.0 <= x < 1.0

regexp_match

matches a regular expression against a string and returns matched substrings.

regexp_replace

replace strings that match a regular expression

repeat

repeats the string to n times

replace

replaces all occurrences of from with to in the string

reverse

reverses the string

right

returns the last n characters in the string

rollup

Create a grouping set for rollup

round

round to nearest integer

rpad

fill up a string to the length by appending the characters

rtrim

removes all characters, spaces by default, from the end of a string

scalar_subquery

Create a scalar subquery expression

sha224

SHA-224 hash

sha256

SHA-256 hash

sha384

SHA-384 hash

sha512

SHA-512 hash

signum

sign of the argument (-1, 0, +1)

sin

sine

split_part

splits a string based on a delimiter and picks out the desired field based on the index.

sqrt

square root of a number

starts_with

whether the string starts with the prefix

strpos

finds the position from where the substring matchs the string

substr

substring from the position to the end

substring

substring from the position with length characters

sum

Create an expression to represent the sum() aggregate function

tan

tangent

to_hex

returns the hexdecimal representation of an integer

to_timestamp_micros

converts a string to a Timestamp(Microseconds, None)

to_timestamp_millis

converts a string to a Timestamp(Milliseconds, None)

to_timestamp_seconds

converts a string to a Timestamp(Seconds, None)

translate

replaces the characters in from with the counterpart in to

trim

removes all characters, space by default from the string

trunc

truncate toward zero

try_cast

Create a try cast expression

upper

converts the string to upper case

uuid

Returns uuid v4 as a string value

when

Create a CASE WHEN statement with boolean WHEN expressions and no base expression.