Module deltalake::arrow::compute

Computation kernels on Arrow Arrays

Modules

• kernels
  Computation kernels on Arrow Arrays

Structs

• CastOptions
  CastOptions provides a way to override the default cast behaviors
• FilterBuilder
  A builder to construct FilterPredicate
• FilterPredicate
  A filtering predicate that can be applied to an Array
• LexicographicalComparator
  A lexicographical comparator that wraps given array data (columns) and can lexicographically compare data at given two indices. The lifetime is the same at the data wrapped.
• Partitions
  A computed set of partitions, see partition
• SlicesIterator
  An iterator of (usize, usize) each representing an interval [start, end) whose slots of a bitmap Buffer are true. Each interval corresponds to a contiguous region of memory to be “taken” from an array to be filtered.
• SortColumn
  One column to be used in lexicographical sort
• SortOptions
  Options that define the sort order of a given column
• TakeOptions
  Options that define how take should behave

Enums

• DatePart
  Valid parts to extract from date/time/timestamp arrays.

Functions

• and
  Performs AND operation on two arrays. If either left or right value is null then the result is also null.
• and_kleene
  Logical ‘and’ boolean values with Kleene logic
• and_not
  Performs AND_NOT operation on two arrays. If either left or right value is null then the result is also null.
• binary
  Given two arrays of length len, calls op(a[i], b[i]) for i in 0..len, collecting the results in a PrimitiveArray. If any index is null in either a or b, the corresponding index in the result will also be null
• binary_mut
  Given two arrays of length len, calls op(a[i], b[i]) for i in 0..len, mutating the mutable PrimitiveArray a. If any index is null in either a or b, the corresponding index in the result will also be null.
• bit_and
  Returns the bitwise and of all non-null input values.
• bit_or
  Returns the bitwise or of all non-null input values.
• bit_xor
  Returns the bitwise xor of all non-null input values.
• bool_and
  Returns true if all non-null input values are true, otherwise false.
• bool_or
  Returns true if any non-null input value is true, otherwise false.
• build_filterDeprecated
  Returns a prepared function optimized to filter multiple arrays. Creating this function requires time, but using it is faster than filter when the same filter needs to be applied to multiple arrays (e.g. a multi-column RecordBatch). WARNING: the nulls of filter are ignored and the value on its slot is considered. Therefore, it is considered undefined behavior to pass filter with null values.
• can_cast_types
  Return true if a value of type from_type can be cast into a value of to_type.
• cast
  Cast array to the provided data type and return a new Array with type to_type, if possible.
• cast_with_options
  Cast array to the provided data type and return a new Array with type to_type, if possible.
• concat
  Concatenate multiple Array of the same type into a single ArrayRef.
• concat_batches
  Concatenates batches together into a single RecordBatch.
• contains
  Perform SQL CONTAINS(left, right)
• date_part
  Given an array, return a new array with the extracted DatePart as signed 32-bit integer values.
• dayDeprecated
  Extracts the day of a given temporal primitive array as an array of integers
• day_dynDeprecated
  Extracts the day of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• doyDeprecated
  Extracts the day of year of a given temporal primitive array as an array of integers The day of year that ranges from 1 to 366
• doy_dynDeprecated
  Extracts the day of year of a given temporal array as an array of integers The day of year that ranges from 1 to 366. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• ends_with
  Perform SQL ENDSWITH(left, right)
• eqDeprecated
  Perform left == right operation on two PrimitiveArrays.
• eq_binaryDeprecated
  Perform left == right operation on BinaryArray / LargeBinaryArray.
• eq_binary_scalarDeprecated
  Perform left == right operation on BinaryArray / LargeBinaryArray and a scalar
• eq_boolDeprecated
  Perform left == right operation on BooleanArray
• eq_bool_scalarDeprecated
  Perform left == right operation on BooleanArray and a scalar
• eq_dynDeprecated
  Perform left == right operation on two (dynamic) Arrays.
• eq_dyn_binary_scalarDeprecated
  Perform left == right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• eq_dyn_bool_scalarDeprecated
  Perform left == right operation on an array and a numeric scalar value.
• eq_dyn_scalarDeprecated
  Perform left == right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• eq_dyn_utf8_scalarDeprecated
  Perform left == right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• eq_scalarDeprecated
  Perform left == right operation on a PrimitiveArray and a scalar value.
• eq_utf8Deprecated
  Perform left == right operation on StringArray / LargeStringArray.
• eq_utf8_scalarDeprecated
  Perform left == right operation on StringArray / LargeStringArray and a scalar.
• filter
  Filters an Array, returning elements matching the filter (i.e. where the values are true).
• filter_record_batch
  Returns a new RecordBatch with arrays containing only values matching the filter.
• gtDeprecated
  Perform left > right operation on two PrimitiveArrays. Non-null values are greater than null values.
• gt_binaryDeprecated
  Perform left > right operation on BinaryArray / LargeBinaryArray.
• gt_binary_scalarDeprecated
  Perform left > right operation on BinaryArray / LargeBinaryArray and a scalar.
• gt_boolDeprecated
  Perform left > right operation on BooleanArray
• gt_bool_scalarDeprecated
  Perform left > right operation on BooleanArray and a scalar
• gt_dynDeprecated
  Perform left > right operation on two (dynamic) Arrays.
• gt_dyn_binary_scalarDeprecated
  Perform left > right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• gt_dyn_bool_scalarDeprecated
  Perform left > right operation on an array and a numeric scalar value. Supports BooleanArrays.
• gt_dyn_scalarDeprecated
  Perform left > right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• gt_dyn_utf8_scalarDeprecated
  Perform left > right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• gt_eqDeprecated
  Perform left >= right operation on two PrimitiveArrays. Non-null values are greater than null values.
• gt_eq_binaryDeprecated
  Perform left >= right operation on BinaryArray / LargeBinaryArray.
• gt_eq_binary_scalarDeprecated
  Perform left >= right operation on BinaryArray / LargeBinaryArray and a scalar.
• gt_eq_boolDeprecated
  Perform left >= right operation on BooleanArray
• gt_eq_bool_scalarDeprecated
  Perform left >= right operation on BooleanArray and a scalar
• gt_eq_dynDeprecated
  Perform left >= right operation on two (dynamic) Arrays.
• gt_eq_dyn_binary_scalarDeprecated
  Perform left >= right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• gt_eq_dyn_bool_scalarDeprecated
  Perform left >= right operation on an array and a numeric scalar value. Supports BooleanArrays.
• gt_eq_dyn_scalarDeprecated
  Perform left >= right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• gt_eq_dyn_utf8_scalarDeprecated
  Perform left >= right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• gt_eq_scalarDeprecated
  Perform left >= right operation on a PrimitiveArray and a scalar value. Non-null values are greater than null values.
• gt_eq_utf8Deprecated
  Perform left >= right operation on StringArray / LargeStringArray.
• gt_eq_utf8_scalarDeprecated
  Perform left >= right operation on StringArray / LargeStringArray and a scalar.
• gt_scalarDeprecated
  Perform left > right operation on a PrimitiveArray and a scalar value. Non-null values are greater than null values.
• gt_utf8Deprecated
  Perform left > right operation on StringArray / LargeStringArray.
• gt_utf8_scalarDeprecated
  Perform left > right operation on StringArray / LargeStringArray and a scalar.
• hourDeprecated
  Extracts the hours of a given temporal primitive array as an array of integers within the range of [0, 23].
• hour_dynDeprecated
  Extracts the hours of a given array as an array of integers within the range of [0, 23]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• ilike
  Perform SQL left ILIKE right
• in_list
  Checks if a GenericListArray contains a value in the PrimitiveArray
• in_list_utf8
  Checks if a GenericListArray contains a value in the GenericStringArray
• interleave
  Takes elements by index from a list of Array, creating a new Array from those values.
• is_not_null
  Returns a non-null BooleanArray with whether each value of the array is not null.
• is_null
  Returns a non-null BooleanArray with whether each value of the array is null.
• lexicographical_partition_rangesDeprecated
  Use partition instead. Given a list of already sorted columns, find partition ranges that would partition lexicographically equal values across columns.
• lexsort
  Sort a list of ArrayRef using SortOptions provided for each array.
• lexsort_to_indices
  Sort elements lexicographically from a list of ArrayRef into an unsigned integer (UInt32Array) of indices.
• like
  Perform SQL left LIKE right
• ltDeprecated
  Perform left < right operation on two PrimitiveArrays. Null values are less than non-null values.
• lt_binaryDeprecated
  Perform left < right operation on BinaryArray / LargeBinaryArray.
• lt_binary_scalarDeprecated
  Perform left < right operation on BinaryArray / LargeBinaryArray and a scalar.
• lt_boolDeprecated
  Perform left < right operation on BooleanArray
• lt_bool_scalarDeprecated
  Perform left < right operation on BooleanArray and a scalar
• lt_dynDeprecated
  Perform left < right operation on two (dynamic) Arrays.
• lt_dyn_binary_scalarDeprecated
  Perform left < right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• lt_dyn_bool_scalarDeprecated
  Perform left < right operation on an array and a numeric scalar value. Supports BooleanArrays.
• lt_dyn_scalarDeprecated
  Perform left < right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• lt_dyn_utf8_scalarDeprecated
  Perform left < right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• lt_eqDeprecated
  Perform left <= right operation on two PrimitiveArrays. Null values are less than non-null values.
• lt_eq_binaryDeprecated
  Perform left <= right operation on BinaryArray / LargeBinaryArray.
• lt_eq_binary_scalarDeprecated
  Perform left <= right operation on BinaryArray / LargeBinaryArray and a scalar.
• lt_eq_boolDeprecated
  Perform left <= right operation on BooleanArray
• lt_eq_bool_scalarDeprecated
  Perform left <= right operation on BooleanArray and a scalar
• lt_eq_dynDeprecated
  Perform left <= right operation on two (dynamic) Arrays.
• lt_eq_dyn_binary_scalarDeprecated
  Perform left <= right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• lt_eq_dyn_bool_scalarDeprecated
  Perform left <= right operation on an array and a numeric scalar value. Supports BooleanArrays.
• lt_eq_dyn_scalarDeprecated
  Perform left <= right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• lt_eq_dyn_utf8_scalarDeprecated
  Perform left <= right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• lt_eq_scalarDeprecated
  Perform left <= right operation on a PrimitiveArray and a scalar value. Null values are less than non-null values.
• lt_eq_utf8Deprecated
  Perform left <= right operation on StringArray / LargeStringArray.
• lt_eq_utf8_scalarDeprecated
  Perform left <= right operation on StringArray / LargeStringArray and a scalar.
• lt_scalarDeprecated
  Perform left < right operation on a PrimitiveArray and a scalar value. Null values are less than non-null values.
• lt_utf8Deprecated
  Perform left < right operation on StringArray / LargeStringArray.
• lt_utf8_scalarDeprecated
  Perform left < right operation on StringArray / LargeStringArray and a scalar.
• max
  Returns the maximum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value
• max_array
  Returns the max of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.
• max_binary
  Returns the maximum value in the binary array, according to the natural order.
• max_boolean
  Returns the maximum value in the boolean array
• max_string
  Returns the maximum value in the string array, according to the natural order.
• microsecondDeprecated
  Extracts the microseconds of a given temporal primitive array as an array of integers
• microsecond_dynDeprecated
  Extracts the microseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• millisecondDeprecated
  Extracts the milliseconds of a given temporal primitive array as an array of integers
• millisecond_dynDeprecated
  Extracts the milliseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• min
  Returns the minimum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value
• min_array
  Returns the min of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.
• min_binary
  Returns the minimum value in the binary array, according to the natural order.
• min_boolean
  Returns the minimum value in the boolean array.
• min_string
  Returns the minimum value in the string array, according to the natural order.
• minuteDeprecated
  Extracts the minutes of a given temporal primitive array as an array of integers
• minute_dynDeprecated
  Extracts the minutes of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• monthDeprecated
  Extracts the month of a given temporal primitive array as an array of integers within the range of [1, 12].
• month_dynDeprecated
  Extracts the month of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• multiply_fixed_point
  Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.
• multiply_fixed_point_checked
  Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.
• multiply_fixed_point_dyn
  Perform left * right operation on two decimal arrays. If either left or right value is null then the result is also null.
• nanosecondDeprecated
  Extracts the nanoseconds of a given temporal primitive array as an array of integers
• nanosecond_dynDeprecated
  Extracts the nanoseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• neqDeprecated
  Perform left != right operation on two PrimitiveArrays.
• neq_binaryDeprecated
  Perform left != right operation on BinaryArray / LargeBinaryArray.
• neq_binary_scalarDeprecated
  Perform left != right operation on BinaryArray / LargeBinaryArray and a scalar.
• neq_boolDeprecated
  Perform left != right operation on BooleanArray
• neq_bool_scalarDeprecated
  Perform left != right operation on BooleanArray and a scalar
• neq_dynDeprecated
  Perform left != right operation on two (dynamic) Arrays.
• neq_dyn_binary_scalarDeprecated
  Perform left != right operation on an array and a numeric scalar value. Supports BinaryArray and LargeBinaryArray
• neq_dyn_bool_scalarDeprecated
  Perform left != right operation on an array and a numeric scalar value. Supports BooleanArrays.
• neq_dyn_scalarDeprecated
  Perform left != right operation on an array and a numeric scalar value. Supports PrimitiveArrays, and DictionaryArrays that have primitive values.
• neq_dyn_utf8_scalarDeprecated
  Perform left != right operation on an array and a numeric scalar value. Supports StringArrays, and DictionaryArrays that have string values
• neq_scalarDeprecated
  Perform left != right operation on a PrimitiveArray and a scalar value.
• neq_utf8Deprecated
  Perform left != right operation on StringArray / LargeStringArray.
• neq_utf8_scalarDeprecated
  Perform left != right operation on StringArray / LargeStringArray and a scalar.
• nilike
  Perform SQL left NOT ILIKE right
• nlike
  Perform SQL left NOT LIKE right
• not
  Performs unary NOT operation on an arrays. If value is null then the result is also null.
• nullif
  Copies original array, setting validity bit to false if a secondary comparison boolean array is set to true
• num_days_from_mondayDeprecated
  Extracts the day of week of a given temporal primitive array as an array of integers.
• num_days_from_monday_dynDeprecated
  Extracts the day of week of a given temporal array as an array of integers.
• num_days_from_sundayDeprecated
  Extracts the day of week of a given temporal primitive array as an array of integers, starting at Sunday.
• num_days_from_sunday_dynDeprecated
  Extracts the day of week of a given temporal array as an array of integers, starting at Sunday.
• or
  Performs OR operation on two arrays. If either left or right value is null then the result is also null.
• or_kleene
  Logical ‘or’ boolean values with Kleene logic
• partial_sort
  It’s unstable_sort, may not preserve the order of equal elements
• partition
  Given a list of lexicographically sorted columns, computes the Partitions, where a partition consists of the set of consecutive rows with equal values
• prep_null_mask_filter
  Remove null values by do a bitmask AND operation with null bits and the boolean bits.
• quarterDeprecated
  Extracts the quarter of a given temporal primitive array as an array of integers within the range of [1, 4].
• quarter_dynDeprecated
  Extracts the quarter of a given temporal array as an array of integersa within the range of [1, 4]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• rank
  Assigns a rank to each value in array based on its position in the sorted order
• regexp_is_match_utf8
  Perform SQL array ~ regex_array operation on StringArray / LargeStringArray. If regex_array element has an empty value, the corresponding result value is always true.
• regexp_is_match_utf8_scalar
  Perform SQL array ~ regex_array operation on StringArray / LargeStringArray and a scalar.
• regexp_match
  Extract all groups matched by a regular expression for a given String array.
• secondDeprecated
  Extracts the seconds of a given temporal primitive array as an array of integers
• second_dynDeprecated
  Extracts the seconds of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• shift
  Shifts array by defined number of items (to left or right) A positive value for offset shifts the array to the right a negative value shifts the array to the left.
• sort
  Sort the ArrayRef using SortOptions.
• sort_limit
  Sort the ArrayRef partially.
• sort_to_indices
  Sort elements from ArrayRef into an unsigned integer (UInt32Array) of indices. Floats are sorted using IEEE 754 totalOrder. limit is an option for partial_sort.
• starts_with
  Perform SQL STARTSWITH(left, right)
• sum
  Returns the sum of values in the primitive array.
• sum_array
  Returns the sum of values in the array.
• sum_array_checked
  Returns the sum of values in the array.
• sum_checked
  Returns the sum of values in the primitive array.
• take
  Take elements by index from Array, creating a new Array from those indexes.
• take_record_batch
  Take rows by index from RecordBatch and returns a new RecordBatch from those indexes.
• try_binary
  Applies the provided fallible binary operation across a and b, returning any error, and collecting the results into a PrimitiveArray. If any index is null in either a or b, the corresponding index in the result will also be null
• try_binary_mut
  Applies the provided fallible binary operation across a and b by mutating the mutable PrimitiveArray a with the results, returning any error. If any index is null in either a or b, the corresponding index in the result will also be null
• try_unary
  See PrimitiveArray::try_unary
• try_unary_dynDeprecated
  Applies a fallible unary function to an array with primitive values.
• try_unary_mut
  See PrimitiveArray::try_unary_mut
• unary
  See PrimitiveArray::unary
• unary_cmpDeprecated
  Applies an unary and infallible comparison function to a primitive array.
• unary_dynDeprecated
  Applies an infallible unary function to an array with primitive values.
• unary_mut
  See PrimitiveArray::unary_mut
• using_chrono_tz_and_utc_naive_date_timeDeprecated
  Parse the given string into a string representing fixed-offset that is correct as of the given UTC NaiveDateTime. Note that the offset is function of time and can vary depending on whether daylight savings is in effect or not. e.g. Australia/Sydney is +10:00 or +11:00 depending on DST.
• weekDeprecated
  Extracts the week of a given temporal primitive array as an array of integers
• week_dynDeprecated
  Extracts the week of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.
• yearDeprecated
  Extracts the years of a given temporal primitive array as an array of integers
• year_dynDeprecated
  Extracts the years of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

Type Aliases

• FilterDeprecated
  Function that can filter arbitrary arrays