Module arrow::compute

Computation kernels on Arrow Arrays

Re-exports

pub use self::kernels::comparison::*;

pub use self::kernels::limit::*;

Modules

kernels

Computation kernels on Arrow Arrays

Structs

CastOptions

CastOptions provides a way to override the default cast behaviors

FilterBuilder

A builder to construct FilterPredicate

FilterPredicate

A filtering predicate that can be applied to an Array

LexicographicalComparator

A lexicographical comparator that wraps given array data (columns) and can lexicographically compare data at given two indices. The lifetime is the same at the data wrapped.

SlicesIterator

An iterator of (usize, usize) each representing an interval [start, end) whose slots of a bitmap Buffer are true. Each interval corresponds to a contiguous region of memory to be “taken” from an array to be filtered.

SortColumn

One column to be used in lexicographical sort

SortOptions

Options that define the sort order of a given column

TakeOptions

Options that define how take should behave

Constants

DEFAULT_CAST_OPTIONS

Functions

add

Perform left + right operation on two arrays. If either left or right value is null then the result is also null.

add_checked

Perform left + right operation on two arrays. If either left or right value is null then the result is also null.

add_dyn

Perform left + right operation on two arrays. If either left or right value is null then the result is also null.

add_dyn_checked

Perform left + right operation on two arrays. If either left or right value is null then the result is also null.

add_scalar

Add every value in an array by a scalar. If any value in the array is null then the result is also null.

add_scalar_checked

Add every value in an array by a scalar. If any value in the array is null then the result is also null.

add_scalar_checked_dyn

Add every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

add_scalar_dyn

Add every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

and

Performs AND operation on two arrays. If either left or right value is null then the result is also null.

and_kleene

Logical ‘and’ boolean values with Kleene logic

binary

Given two arrays of length len, calls op(a[i], b[i]) for i in 0..len, collecting the results in a PrimitiveArray. If any index is null in either a or b, the corresponding index in the result will also be null

binary_mut

Given two arrays of length len, calls op(a[i], b[i]) for i in 0..len, mutating the mutable PrimitiveArray a. If any index is null in either a or b, the corresponding index in the result will also be null.

build_filterDeprecated

Returns a prepared function optimized to filter multiple arrays. Creating this function requires time, but using it is faster than filter when the same filter needs to be applied to multiple arrays (e.g. a multi-column RecordBatch). WARNING: the nulls of filter are ignored and the value on its slot is considered. Therefore, it is considered undefined behavior to pass filter with null values.

can_cast_types

Return true if a value of type from_type can be cast into a value of to_type. Note that such as cast may be lossy.

cast

Cast array to the provided data type and return a new Array with type to_type, if possible.

cast_with_options

Cast array to the provided data type and return a new Array with type to_type, if possible. It accepts CastOptions to allow consumers to configure cast behavior.

concat

Concatenate multiple Array of the same type into a single ArrayRef.

concat_batches

Concatenates batches together into a single record batch.

day

Extracts the day of a given temporal primitive array as an array of integers

day_dyn

Extracts the day of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

divide

Perform left / right operation on two arrays without checking for division by zero or overflow.

divide_checked

Perform left / right operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will be Err(ArrowError::DivideByZero).

divide_dyn

Perform left / right operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will be Err(ArrowError::DivideByZero).

divide_dyn_checked

Perform left / right operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will be Err(ArrowError::DivideByZero).

divide_dyn_opt

Perform left / right operation on two arrays. If either left or right value is null then the result is also null.

divide_opt

Perform left / right operation on two arrays. If either left or right value is null then the result is also null.

divide_scalar

Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be Err(ArrowError::DivideByZero).

divide_scalar_checked_dyn

Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be Err(ArrowError::DivideByZero). The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

divide_scalar_dyn

Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be Err(ArrowError::DivideByZero). The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

divide_scalar_opt_dyn

Divide every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

doy

Extracts the day of year of a given temporal primitive array as an array of integers The day of year that ranges from 1 to 366

doy_dyn

Extracts the day of year of a given temporal array as an array of integers The day of year that ranges from 1 to 366. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

filter

Filters an Array, returning elements matching the filter (i.e. where the values are true).

filter_record_batch

Returns a new RecordBatch with arrays containing only values matching the filter.

hour

Extracts the hours of a given temporal primitive array as an array of integers within the range of [0, 23].

hour_dyn

Extracts the hours of a given array as an array of integers within the range of [0, 23]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

interleave

Takes elements by index from a list of Array, creating a new Array from those values.

is_not_null

Returns a non-null BooleanArray with whether each value of the array is not null.

is_null

Returns a non-null BooleanArray with whether each value of the array is null.

lexicographical_partition_ranges

Given a list of already sorted columns, find partition ranges that would partition lexicographically equal values across columns.

lexsort

Sort a list of ArrayRef using SortOptions provided for each array.

lexsort_to_indices

Sort elements lexicographically from a list of ArrayRef into an unsigned integer (UInt32Array) of indices.

math_op

Helper function to perform math lambda function on values from two arrays. If either left or right value is null then the output value is also null, so 1 + null is null.

max

Returns the maximum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value

max_array

Returns the max of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.

max_binary

Returns the maximum value in the binary array, according to the natural order.

max_boolean

Returns the maximum value in the boolean array

max_string

Returns the maximum value in the string array, according to the natural order.

min

Returns the minimum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value

min_array

Returns the min of values in the array of ArrowNumericType type, or dictionary array with value of ArrowNumericType type.

min_binary

Returns the minimum value in the binary array, according to the natural order.

min_boolean

Returns the minimum value in the boolean array.

min_string

Returns the minimum value in the string array, according to the natural order.

minute

Extracts the minutes of a given temporal primitive array as an array of integers

minute_dyn

Extracts the minutes of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

modulus

Perform left % right operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will be Err(ArrowError::DivideByZero).

modulus_scalar

Modulus every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be Err(ArrowError::DivideByZero).

month

Extracts the month of a given temporal primitive array as an array of integers within the range of [1, 12].

month_dyn

Extracts the month of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

multiply

Perform left * right operation on two arrays. If either left or right value is null then the result is also null.

multiply_checked

Perform left * right operation on two arrays. If either left or right value is null then the result is also null.

multiply_dyn

Perform left * right operation on two arrays. If either left or right value is null then the result is also null.

multiply_dyn_checked

Perform left * right operation on two arrays. If either left or right value is null then the result is also null.

multiply_scalar

Multiply every value in an array by a scalar. If any value in the array is null then the result is also null.

multiply_scalar_checked

Multiply every value in an array by a scalar. If any value in the array is null then the result is also null.

multiply_scalar_checked_dyn

Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

multiply_scalar_dyn

Multiply every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

nanosecond

Extracts the nanoseconds of a given temporal primitive array as an array of integers

nanosecond_dyn

Extracts the nanoseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

negate

Perform - operation on an array. If value is null then the result is also null.

negate_checked

Perform - operation on an array. If value is null then the result is also null.

not

Performs unary NOT operation on an arrays. If value is null then the result is also null.

num_days_from_monday

Extracts the day of week of a given temporal primitive array as an array of integers.

num_days_from_monday_dyn

Extracts the day of week of a given temporal array as an array of integers.

num_days_from_sunday

Extracts the day of week of a given temporal primitive array as an array of integers, starting at Sunday.

num_days_from_sunday_dyn

Extracts the day of week of a given temporal array as an array of integers, starting at Sunday.

or

Performs OR operation on two arrays. If either left or right value is null then the result is also null.

or_kleene

Logical ‘or’ boolean values with Kleene logic

partial_sort

It’s unstable_sort, may not preserve the order of equal elements

powf_scalar

Raise array with floating point values to the power of a scalar.

prep_null_mask_filter

Remove null values by do a bitmask AND operation with null bits and the boolean bits.

quarter

Extracts the quarter of a given temporal primitive array as an array of integers within the range of [1, 4].

quarter_dyn

Extracts the quarter of a given temporal array as an array of integersa within the range of [1, 4]. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

regexp_is_match_utf8

Perform SQL array ~ regex_array operation on StringArray / LargeStringArray. If regex_array element has an empty value, the corresponding result value is always true.

regexp_is_match_utf8_scalar

Perform SQL array ~ regex_array operation on StringArray / LargeStringArray and a scalar.

regexp_match

Extract all groups matched by a regular expression for a given String array.

second

Extracts the seconds of a given temporal primitive array as an array of integers

second_dyn

Extracts the seconds of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

shift

Shifts array by defined number of items (to left or right) A positive value for offset shifts the array to the right a negative value shifts the array to the left.

sort

Sort the ArrayRef using SortOptions.

sort_limit

Sort the ArrayRef partially.

sort_to_indices

Sort elements from ArrayRef into an unsigned integer (UInt32Array) of indices. For floating point arrays any NaN values are considered to be greater than any other non-null value limit is an option for partial_sort

subtract

Perform left - right operation on two arrays. If either left or right value is null then the result is also null.

subtract_checked

Perform left - right operation on two arrays. If either left or right value is null then the result is also null.

subtract_dyn

Perform left - right operation on two arrays. If either left or right value is null then the result is also null.

subtract_dyn_checked

Perform left - right operation on two arrays. If either left or right value is null then the result is also null.

subtract_scalar

Subtract every value in an array by a scalar. If any value in the array is null then the result is also null.

subtract_scalar_checked

Subtract every value in an array by a scalar. If any value in the array is null then the result is also null.

subtract_scalar_checked_dyn

Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

subtract_scalar_dyn

Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a PrimitiveArray of the type same as the scalar, or a DictionaryArray of the value type same as the scalar.

sum

Returns the sum of values in the primitive array.

sum_array

Returns the sum of values in the array.

sum_array_checked

Returns the sum of values in the array.

sum_checked

Returns the sum of values in the primitive array.

take

Take elements by index from Array, creating a new Array from those indexes.

try_binary

Applies the provided fallible binary operation across a and b, returning any error, and collecting the results into a PrimitiveArray. If any index is null in either a or b, the corresponding index in the result will also be null

try_binary_mut

Applies the provided fallible binary operation across a and b by mutating the mutable PrimitiveArray a with the results, returning any error. If any index is null in either a or b, the corresponding index in the result will also be null

try_unary

See PrimitiveArray::try_unary

try_unary_dyn

Applies a fallible unary function to an array with primitive values.

try_unary_mut

See PrimitiveArray::try_unary_mut

unary

See PrimitiveArray::unary

unary_dyn

Applies an infallible unary function to an array with primitive values.

unary_mut

See PrimitiveArray::unary_mut

using_chrono_tz_and_utc_naive_date_timeDeprecated

Parse the given string into a string representing fixed-offset that is correct as of the given UTC NaiveDateTime. Note that the offset is function of time and can vary depending on whether daylight savings is in effect or not. e.g. Australia/Sydney is +10:00 or +11:00 depending on DST.

week

Extracts the week of a given temporal primitive array as an array of integers

week_dyn

Extracts the week of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

year

Extracts the years of a given temporal primitive array as an array of integers

year_dyn

Extracts the years of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an Err will be returned.

Type Definitions

FilterDeprecated

Function that can filter arbitrary arrays