Module datafusion::prelude

DataFusion “prelude” to simplify importing common types.

Like the standard library’s prelude, this module simplifies importing of common items. Unlike the standard prelude, the contents of this module must be imported manually:

use datafusion::prelude::*;

Re-exports

• pub use crate::dataframe::DataFrame;
• pub use crate::execution::context::SQLOptions;
• pub use crate::execution::context::SessionContext;
• pub use crate::execution::options::AvroReadOptions;
• pub use crate::execution::options::CsvReadOptions;
• pub use crate::execution::options::NdJsonReadOptions;
• pub use crate::execution::options::ParquetReadOptions;

Structs

• Column
  A named reference to a qualified field in a schema.
• SessionConfig
  Configuration options for SessionContext.
• SimpleAggregateUDF
  Implements AggregateUDFImpl for functions that have a single signature and return type.
• SimpleScalarUDF
  Implements ScalarUDFImpl for functions that have a single signature and return type.
• SimpleWindowUDF
  Implements WindowUDFImpl for functions that have a single signature and return type.

Enums

• Expr
  Expr is a central struct of DataFusion’s query API, and represent logical expressions such as A + 1, or CAST(c1 AS int).
• JoinType
  Join type
• Partitioning
  Logical partitioning schemes supported by the repartition operator.

Traits

• Add
  The addition operator +.
• BitAnd
  The bitwise AND operator &.
• BitOr
  The bitwise OR operator |.
• BitXor
  The bitwise XOR operator ^.
• Div
  The division operator /.
• Mul
  The multiplication operator *.
• Neg
  The unary negation operator -.
• Not
  The unary logical negation operator !.
• Rem
  The remainder operator %.
• Shl
  The left shift operator <<. Note that because this trait is implemented for all integer types with multiple right-hand-side types, Rust’s type checker has special handling for _ << _, setting the result type for integer operations to the type of the left-hand-side operand. This means that though a << b and a.shl(b) are one and the same from an evaluation standpoint, they are different when it comes to type inference.
• Shr
  The right shift operator >>. Note that because this trait is implemented for all integer types with multiple right-hand-side types, Rust’s type checker has special handling for _ >> _, setting the result type for integer operations to the type of the left-hand-side operand. This means that though a >> b and a.shr(b) are one and the same from an evaluation standpoint, they are different when it comes to type inference.
• Sub
  The subtraction operator -.

Functions

• abs
  returns the absolute value of a given number
• acos
  returns the arc cosine or inverse cosine of a number
• acosh
  returns inverse hyperbolic cosine
• and
  Return a new expression with a logical AND
• approx_distinct
  Returns the approximate number of distinct input values. This function provides an approximation of count(DISTINCT x). Zero is returned if all input values are null. This function should produce a standard error of 0.81%, which is the standard deviation of the (approximately normal) error distribution over all possible sets. It does not guarantee an upper bound on the error for any specific input set.
• approx_median
  Calculate an approximation of the median for expr.
• approx_percentile_cont
  Calculate an approximation of the specified percentile for expr.
• approx_percentile_cont_with_weight
  Calculate an approximation of the specified percentile for expr and weight_expr.
• array_agg
  Create an expression to represent the array_agg() aggregate function
• array_append
  appends an element to the end of an array.
• array_concat
  Concatenates arrays.
• array_dims
  returns an array of the array’s dimensions.
• array_distinct
  returns distinct values from the array after removing duplicates.
• array_element
  extracts the element with the index n from the array.
• array_empty
  returns true for an empty array or false for a non-empty array.
• array_except
  returns an array of the elements that appear in the first array but not in the second.
• array_has
  returns true, if the element appears in the first array, otherwise false.
• array_has_all
  returns true if each element of the second array appears in the first array; otherwise, it returns false.
• array_has_any
  returns true if at least one element of the second array appears in the first array; otherwise, it returns false.
• array_intersect
  returns an array of the elements in the intersection of array1 and array2.
• array_length
  returns the length of the array dimension.
• array_ndims
  returns the number of dimensions of the array.
• array_pop_back
  returns the array without the last element.
• array_pop_front
  returns the array without the first element.
• array_position
  searches for an element in the array, returns first occurrence.
• array_positions
  searches for an element in the array, returns all occurrences.
• array_prepend
  Prepends an element to the beginning of an array.
• array_remove
  removes the first element from the array equal to the given value.
• array_remove_all
  removes all elements from the array equal to the given value.
• array_remove_n
  removes the first max elements from the array equal to the given value.
• array_repeat
  returns an array containing element count times.
• array_replace
  replaces the first occurrence of the specified element with another specified element.
• array_replace_all
  replaces all occurrences of the specified element with another specified element.
• array_replace_n
  replaces the first max occurrences of the specified element with another specified element.
• array_resize
  returns an array with the specified size filled with the given value.
• array_reverse
  reverses the order of elements in the array.
• array_slice
  returns a slice of the array.
• array_sort
  returns sorted array.
• array_to_string
  converts each element to its text representation.
• array_union
  returns an array of the elements in the union of array1 and array2 without duplicates.
• arrow_cast
  returns value1 cast to the arrow_type given the second argument. This can be used to cast to a specific arrow_type.
• arrow_typeof
  Returns the Arrow type of the input expression.
• ascii
  Returns the numeric code of the first character of the argument.
• asin
  returns the arc sine or inverse sine of a number
• asinh
  returns inverse hyperbolic sine
• atan
  returns inverse tangent
• atan2
  returns inverse tangent of a division given in the argument
• atanh
  returns inverse hyperbolic tangent
• avg
  Create an expression to represent the avg() aggregate function
• binary_expr
  Return a new expression left <op> right
• bit_length
  Returns the number of bits in the string
• bitwise_and
  Return a new expression with bitwise AND
• bitwise_or
  Return a new expression with bitwise OR
• bitwise_shift_left
  Return a new expression with bitwise SHIFT LEFT
• bitwise_shift_right
  Return a new expression with bitwise SHIFT RIGHT
• bitwise_xor
  Return a new expression with bitwise XOR
• btrim
  Removes all characters, spaces by default, from both sides of a string
• cardinality
  returns the total number of elements in the array.
• case
  Create a CASE WHEN statement with literal WHEN expressions for comparison to the base expression.
• cast
  Create a cast expression
• cbrt
  cube root of a number
• ceil
  nearest integer greater than or equal to argument
• char_length
  the number of characters in the string
• character_length
  the number of characters in the string
• chr
  Converts the Unicode code point to a UTF8 character
• coalesce
  Returns coalesce(args...), which evaluates to the value of the first expr which is not NULL
• col
  Create a column expression based on a qualified or unqualified column name. Will normalize unquoted identifiers according to SQL rules (identifiers will become lowercase).
• concat
  Concatenates the text representations of all the arguments. NULL arguments are ignored
• concat_ws
  Concatenates all but the first argument, with separators. The first argument is used as the separator string, and should not be NULL. Other NULL arguments are ignored.
• cos
  cosine
• cosh
  hyperbolic cosine
• cot
  cotangent of a number
• count
  Create an expression to represent the count() aggregate function
• count_distinct
  Create an expression to represent the count(distinct) aggregate function
• create_udaf
  Creates a new UDAF with a specific signature, state type and return type. The signature and state type must match the Accumulator's implementation.
• create_udf
  Convenience method to create a new user defined scalar function (UDF) with a specific signature and specific return type.
• create_udwf
  Creates a new UDWF with a specific signature, state type and return type.
• cube
  Create a grouping set for all combination of exprs
• current_date
  returns current UTC date as a Date32 value
• current_time
  returns current UTC time as a Time64 value
• date_bin
  coerces an arbitrary timestamp to the start of the nearest specified interval
• date_part
  extracts a subfield from the date
• date_trunc
  truncates the date to a specified level of precision
• decode
  decode the input, using the encoding. encoding can be base64 or hex Return $name(arg)
• degrees
  converts radians to degrees
• digest
  Computes the binary hash of an expression using the specified algorithm. Return $name(arg)
• encode
  encode the input, using the encoding. encoding can be base64 or hex Return $name(arg)
• ends_with
  Returns true if the string ends with the suffix, false otherwise.
• exists
  Create an EXISTS subquery expression
• exp
  exponential
• factorial
  factorial
• find_in_set
  Returns a value in the range of 1 to N if the string str is in the string list strlist consisting of N substrings
• flatten
  flattens an array of arrays into a single array.
• floor
  nearest integer less than or equal to argument
• from_unixtime
  converts an integer to RFC3339 timestamp format string
• gcd
  greatest common divisor
• gen_series
  create a list of values in the range between start and stop, include upper bound
• get_field
  Returns the value of the field with the given name from the struct
• grouping_set
  Create a grouping set
• ident
  Create an unqualified column expression from the provided name, without normalizing the column.
• in_list
  Create an in_list expression
• in_subquery
  Create an IN subquery expression
• initcap
  Converts the first letter of each word in string in uppercase and the remaining characters in lowercase
• instr
  finds the position from where the substring matches the string
• is_false
  Create is false expression
• is_not_false
  Create is not false expression
• is_not_true
  Create is not true expression
• is_not_unknown
  Create is not unknown expression
• is_null
  Create is null expression
• is_true
  Create is true expression
• is_unknown
  Create is unknown expression
• isnan
  returns true if a given number is +NaN or -NaN otherwise returns false
• iszero
  returns true if a given number is +0.0 or -0.0 otherwise returns false
• lcm
  least common multiple
• left
  returns the first n characters in the string
• length
  the number of characters in the string
• levenshtein
  Returns the Levenshtein distance between the two given strings
• lit
  Create a literal expression
• lit_timestamp_nano
  Create a literal timestamp expression
• ln
  natural logarithm (base e) of a number
• log
  logarithm of a number for a particular base
• log2
  base 2 logarithm of a number
• log10
  base 10 logarithm of a number
• lower
  Converts a string to lowercase.
• lpad
  fill up a string to the length by prepending the characters
• ltrim
  Removes all characters, spaces by default, from the beginning of a string
• make_array
  Returns an Arrow array using the specified input expressions.
• make_date
  make a date from year, month and day component parts
• max
  Create an expression to represent the max() aggregate function
• md5
  Computes an MD5 128-bit checksum for a string expression. Return $name(arg)
• median
  Calculate the median for expr.
• min
  Create an expression to represent the min() aggregate function
• named_struct
  Returns a struct with the given names and arguments pairs
• nanvl
  returns x if x is not NaN otherwise returns y
• not
  Return a new expression with a logical NOT
• not_exists
  Create a NOT EXISTS subquery expression
• not_in_subquery
  Create a NOT IN subquery expression
• now
  returns the current timestamp in nanoseconds, using the same value for all instances of now() in same statement
• nullif
  returns NULL if value1 equals value2; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE expression
• nvl
  Returns value2 if value1 is NULL; otherwise it returns value1
• nvl2
  Returns value2 if value1 is not NULL; otherwise, it returns value3.
• octet_length
  returns the number of bytes of a string
• or
  Return a new expression with a logical OR
• out_ref_col
  Create an out reference column which hold a reference that has been resolved to a field outside of the current plan.
• overlay
  replace the substring of string that starts at the start’th character and extends for count characters with new substring
• pi
  Returns an approximate value of π
• placeholder
  Create placeholder value that will be filled in (such as $1)
• position
  finds the position from where the substring matches the string
• power
  base raised to the power of exponent
• radians
  converts degrees to radians
• random
  Returns a random value in the range 0.0 <= x < 1.0
• range
  create a list of values in the range between start and stop
• regexp_like
  Returns true if a has at least one match in a string,false otherwise. Return $name(arg)
• regexp_match
  returns a list of regular expression matches in a string. Return $name(arg)
• regexp_replace
  Replaces substrings in a string that match Return $name(arg)
• repeat
  Repeats the string to n times
• replace
  Replaces all occurrences of from with to in the string
• reverse
  reverses the string
• right
  returns the last n characters in the string
• rollup
  Create a grouping set for rollup
• round
  round to nearest integer
• rpad
  fill up a string to the length by appending the characters
• rtrim
  Removes all characters, spaces by default, from the end of a string
• scalar_subquery
  Create a scalar subquery expression
• sha224
  Computes the SHA-224 hash of a binary string. Return $name(arg)
• sha256
  Computes the SHA-256 hash of a binary string. Return $name(arg)
• sha384
  Computes the SHA-384 hash of a binary string. Return $name(arg)
• sha512
  Computes the SHA-512 hash of a binary string. Return $name(arg)
• signum
  sign of the argument (-1, 0, +1)
• sin
  sine
• sinh
  hyperbolic sine
• split_part
  Splits a string based on a delimiter and picks out the desired field based on the index.
• sqrt
  square root of a number
• starts_with
  Returns true if string starts with prefix.
• stddev
  Create an expression to represent the stddev() aggregate function
• string_to_array
  splits a string based on a delimiter and returns an array of parts. Any parts matching the optional null_string will be replaced with NULL
• strpos
  finds the position from where the substring matches the string
• struct
  Returns a struct with the given arguments
• substr
  substring from the position to the end
• substr_index
  Returns the substring from str before count occurrences of the delimiter
• substring
  substring from the position with length characters
• sum
  Create an expression to represent the sum() aggregate function
• tan
  returns the tangent of a number
• tanh
  returns the hyperbolic tangent of a number
• to_char
  Returns a string representation of a date, time, timestamp or duration based on a Chrono pattern.
• to_date
• to_hex
  Converts an integer to a hexadecimal string.
• to_timestamp
  converts a string and optional formats to a Timestamp(Nanoseconds, None)
• to_timestamp_micros
  converts a string and optional formats to a Timestamp(Microseconds, None)
• to_timestamp_millis
  converts a string and optional formats to a Timestamp(Milliseconds, None)
• to_timestamp_nanos
  converts a string and optional formats to a Timestamp(Nanoseconds, None)
• to_timestamp_seconds
  converts a string and optional formats to a Timestamp(Seconds, None)
• to_unixtime
  converts a string and optional formats to a Unixtime
• translate
  replaces the characters in from with the counterpart in to
• trim
  Removes all characters, spaces by default, from both sides of a string
• trunc
  truncate toward zero, with optional precision
• try_cast
  Create a try cast expression
• upper
  Converts a string to uppercase.
• uuid
  returns uuid v4 as a string value
• when
  Create a CASE WHEN statement with boolean WHEN expressions and no base expression.
• wildcard
  Create an ‘*’ Expr::Wildcard expression that matches all columns