Module datafusion::prelude

DataFusion “prelude” to simplify importing common types.

Like the standard library’s prelude, this module simplifies importing of common items. Unlike the standard prelude, the contents of this module must be imported manually:

use datafusion::prelude::*;

Re-exports

• pub use crate::dataframe::DataFrame;
• pub use crate::execution::context::SessionContext;
• pub use crate::execution::options::AvroReadOptions;
• pub use crate::execution::options::CsvReadOptions;
• pub use crate::execution::options::NdJsonReadOptions;
• pub use crate::execution::options::ParquetReadOptions;

Structs

• Column
  A named reference to a qualified field in a schema.
• SessionConfig
  Configuration options for Execution context

Enums

• Expr
  Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).
• JoinType
  Join type
• Partitioning
  Logical partitioning schemes supported by the repartition operator.

Functions

• abs
  absolute value
• acos
  inverse cosine
• acosh
  inverse hyperbolic cosine
• and
  Return a new expression with a logical AND
• approx_distinct
  Returns the approximate number of distinct input values. This function provides an approximation of count(DISTINCT x). Zero is returned if all input values are null. This function should produce a standard error of 0.81%, which is the standard deviation of the (approximately normal) error distribution over all possible sets. It does not guarantee an upper bound on the error for any specific input set.
• approx_median
  Calculate an approximation of the median for expr.
• approx_percentile_cont
  Calculate an approximation of the specified percentile for expr.
• approx_percentile_cont_with_weight
  Calculate an approximation of the specified percentile for expr and weight_expr.
• array
  returns an Arrow array using the specified input expressions.
• array_append
  appends an element to the end of an array.
• array_concat
  concatenates arrays.
• array_dims
  returns an array of the array’s dimensions.
• array_fill
  returns an array filled with copies of the given value.
• array_length
  returns the length of the array dimension.
• array_ndims
  returns the number of dimensions of the array.
• array_position
  searches for an element in the array, returns first occurrence.
• array_positions
  searches for an element in the array, returns all occurrences.
• array_prepend
  prepends an element to the beginning of an array.
• array_remove
  removes all elements equal to the given value from the array.
• array_replace
  replaces a specified element with another specified element.
• array_to_string
  converts each element to its text representation.
• arrow_typeof
  data type
• ascii
  ASCII code value of the character
• asin
  inverse sine
• asinh
  inverse hyperbolic sine
• atan
  inverse tangent
• atan2
  inverse tangent of a division given in the argument
• atanh
  inverse hyperbolic tangent
• avg
  Create an expression to represent the avg() aggregate function
• binary_expr
  Return a new expression left <op> right
• bit_length
  the number of bits in the string
• bitwise_and
  Return a new expression with bitwise AND
• bitwise_or
  Return a new expression with bitwise OR
• bitwise_shift_left
  Return a new expression with bitwise SHIFT LEFT
• bitwise_shift_right
  Return a new expression with bitwise SHIFT RIGHT
• bitwise_xor
  Return a new expression with bitwise XOR
• btrim
  removes all characters, spaces by default, from both sides of a string
• call_fn
  Calls a named built in function
• cardinality
  returns the total number of elements in the array.
• case
  Create a CASE WHEN statement with literal WHEN expressions for comparison to the base expression.
• cast
  Create a cast expression
• cbrt
  cube root of a number
• ceil
  nearest integer greater than or equal to argument
• character_length
  the number of characters in the string
• chr
  converts the Unicode code point to a UTF8 character
• coalesce
  returns coalesce(args...), which evaluates to the value of the first Expr which is not NULL
• col
  Create a column expression based on a qualified or unqualified column name. Will normalize unquoted identifiers according to SQL rules (identifiers will become lowercase).
• concat
  Concatenates the text representations of all the arguments. NULL arguments are ignored.
• concat_expr
  concatenates several strings
• concat_ws
  Concatenates all but the first argument, with separators. The first argument is used as the separator. NULL arguments in values are ignored.
• concat_ws_expr
  concatenates several strings, placing a seperator between each one
• cos
  cosine
• cosh
  hyperbolic cosine
• count
  Create an expression to represent the count() aggregate function
• count_distinct
  Create an expression to represent the count(distinct) aggregate function
• create_udaf
  Creates a new UDAF with a specific signature, state type and return type. The signature and state type must match the Accumulator's implementation.
• create_udf
  Creates a new UDF with a specific signature and specific return type. This is a helper function to create a new UDF. The function create_udf returns a subset of all possible ScalarFunction:
• create_udwf
  Creates a new UDWF with a specific signature, state type and return type.
• cube
  Create a grouping set for all combination of exprs
• current_date
  returns current UTC date as a DataType::Date32 value
• current_time
  returns current UTC time as a DataType::Time64 value
• date_bin
  coerces an arbitrary timestamp to the start of the nearest specified interval
• date_part
  extracts a subfield from the date
• date_trunc
  truncates the date to a specified level of precision
• degrees
  converts radians to degrees
• digest
  compute the binary hash of input, using the algorithm
• exists
  Create an EXISTS subquery expression
• exp
  exponential
• factorial
  factorial
• floor
  nearest integer less than or equal to argument
• from_unixtime
  returns the unix time in format
• gcd
  greatest common divisor
• grouping_set
  Create a grouping set
• ident
  Create an unqualified column expression from the provided name, without normalizing the column.
• in_list
  Create an in_list expression
• in_subquery
  Create an IN subquery expression
• initcap
  converts the first letter of each word in string in uppercase and the remaining characters in lowercase
• is_false
  Create is false expression
• is_not_false
  Create is not false expression
• is_not_true
  Create is not true expression
• is_not_unknown
  Create is not unknown expression
• is_null
  Create is null expression
• is_true
  Create is true expression
• is_unknown
  Create is unknown expression
• lcm
  least common multiple
• left
  returns the first n characters in the string
• lit
  Create a literal expression
• lit_timestamp_nano
  Create a literal timestamp expression
• ln
  natural logarithm
• log
  logarithm of a x for a particular base
• log2
  base 2 logarithm
• log10
  base 10 logarithm
• lower
  convert the string to lower case
• lpad
  fill up a string to the length by prepending the characters
• ltrim
  removes all characters, spaces by default, from the beginning of a string
• max
  Create an expression to represent the max() aggregate function
• md5
  returns the MD5 hash of a string
• median
  Calculate the median for expr.
• min
  Create an expression to represent the min() aggregate function
• not_exists
  Create a NOT EXISTS subquery expression
• not_in_subquery
  Create a NOT IN subquery expression
• now
  returns current timestamp in nanoseconds, using the same value for all instances of now() in same statement
• nullif
  returns NULL if value1 equals value2; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE expression.
• octet_length
  returns the number of bytes of a string
• or
  Return a new expression with a logical OR
• out_ref_col
  Create an out reference column which hold a reference that has been resolved to a field outside of the current plan.
• pi
  Returns an approximate value of π
• power
  base raised to the power of exponent
• radians
  converts degrees to radians
• random
  Returns a random value in the range 0.0 <= x < 1.0
• regexp_match
  matches a regular expression against a string and returns matched substrings.
• regexp_replace
  replace strings that match a regular expression
• repeat
  repeats the string to n times
• replace
  replaces all occurrences of from with to in the string
• reverse
  reverses the string
• right
  returns the last n characters in the string
• rollup
  Create a grouping set for rollup
• round
  round to nearest integer
• rpad
  fill up a string to the length by appending the characters
• rtrim
  removes all characters, spaces by default, from the end of a string
• scalar_subquery
  Create a scalar subquery expression
• sha224
  SHA-224 hash
• sha256
  SHA-256 hash
• sha384
  SHA-384 hash
• sha512
  SHA-512 hash
• signum
  sign of the argument (-1, 0, +1)
• sin
  sine
• sinh
  hyperbolic sine
• split_part
  splits a string based on a delimiter and picks out the desired field based on the index.
• sqrt
  square root of a number
• starts_with
  whether the string starts with the prefix
• stddev
  Create an expression to represent the stddev() aggregate function
• strpos
  finds the position from where the substring matches the string
• substr
  substring from the position to the end
• substring
  substring from the position with length characters
• sum
  Create an expression to represent the sum() aggregate function
• tan
  tangent
• tanh
  hyperbolic tangent
• to_hex
  returns the hexdecimal representation of an integer
• to_timestamp_micros
  converts a string to a Timestamp(Microseconds, None)
• to_timestamp_millis
  converts a string to a Timestamp(Milliseconds, None)
• to_timestamp_seconds
  converts a string to a Timestamp(Seconds, None)
• translate
  replaces the characters in from with the counterpart in to
• trim
  removes all characters, space by default from the string
• trim_array
  removes the last n elements from the array.
• trunc
  truncate toward zero
• try_cast
  Create a try cast expression
• upper
  converts the string to upper case
• uuid
  Returns uuid v4 as a string value
• when
  Create a CASE WHEN statement with boolean WHEN expressions and no base expression.