Expand description
Computation kernels on Arrow Arrays
Re-exports
pub use self::kernels::comparison::*;
pub use self::kernels::limit::*;
Modules
- Computation kernels on Arrow Arrays
Structs
- CastOptions provides a way to override the default cast behaviors
- A builder to construct
FilterPredicate
- A filtering predicate that can be applied to an
Array
- A lexicographical comparator that wraps given array data (columns) and can lexicographically compare data at given two indices. The lifetime is the same at the data wrapped.
- An iterator of
(usize, usize)
each representing an interval[start, end)
whose slots of a bitmap Buffer are true. Each interval corresponds to a contiguous region of memory to be “taken” from an array to be filtered. - One column to be used in lexicographical sort
- Options that define the sort order of a given column
- Options that define how
take
should behave
Functions
- Perform
left + right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left + right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left + right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left + right
operation on two arrays. If either left or right value is null then the result is also null. - Add every value in an array by a scalar. If any value in the array is null then the result is also null.
- Add every value in an array by a scalar. If any value in the array is null then the result is also null.
- Add every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Add every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Performs
AND
operation on two arrays. If either left or right value is null then the result is also null. - Logical ‘and’ boolean values with Kleene logic
- Given two arrays of length
len
, callsop(a[i], b[i])
fori
in0..len
, collecting the results in aPrimitiveArray
. If any index is null in eithera
orb
, the corresponding index in the result will also be null - Given two arrays of length
len
, callsop(a[i], b[i])
fori
in0..len
, mutating the mutablePrimitiveArray
a
. If any index is null in eithera
orb
, the corresponding index in the result will also be null. - Returns the bitwise and of all non-null input values.
- Returns the bitwise or of all non-null input values.
- Returns the bitwise xor of all non-null input values.
- Returns true if all non-null input values are true, otherwise false.
- Returns true if any non-null input value is true, otherwise false.
- build_filterDeprecatedReturns a prepared function optimized to filter multiple arrays. Creating this function requires time, but using it is faster than filter when the same filter needs to be applied to multiple arrays (e.g. a multi-column
RecordBatch
). WARNING: the nulls offilter
are ignored and the value on its slot is considered. Therefore, it is considered undefined behavior to passfilter
with null values. - Return true if a value of type
from_type
can be cast into a value ofto_type
. Note that such as cast may be lossy. - Cast
array
to the provided data type and return a new Array with typeto_type
, if possible. - Cast
array
to the provided data type and return a new Array with typeto_type
, if possible. It acceptsCastOptions
to allow consumers to configure cast behavior. - Concatenates
batches
together into a single record batch. - Extracts the day of a given temporal primitive array as an array of integers
- Extracts the day of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Perform
left / right
operation on two arrays without checking for division by zero or overflow. - Perform
left / right
operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will beErr(ArrowError::DivideByZero)
. - Perform
left / right
operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will beErr(ArrowError::DivideByZero)
. - Perform
left / right
operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will beErr(ArrowError::DivideByZero)
. - Perform
left / right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left / right
operation on two arrays. If either left or right value is null then the result is also null. - Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be
Err(ArrowError::DivideByZero)
. - Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be
Err(ArrowError::DivideByZero)
. The given array must be aPrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Divide every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be
Err(ArrowError::DivideByZero)
. The given array must be aPrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Divide every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Extracts the day of year of a given temporal primitive array as an array of integers The day of year that ranges from 1 to 366
- Extracts the day of year of a given temporal array as an array of integers The day of year that ranges from 1 to 366. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Filters an Array, returning elements matching the filter (i.e. where the values are true).
- Returns a new RecordBatch with arrays containing only values matching the filter.
- Extracts the hours of a given temporal primitive array as an array of integers within the range of [0, 23].
- Extracts the hours of a given array as an array of integers within the range of [0, 23]. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Returns a non-null BooleanArray with whether each value of the array is not null.
- Returns a non-null BooleanArray with whether each value of the array is null.
- Given a list of already sorted columns, find partition ranges that would partition lexicographically equal values across columns.
- Sort a list of
ArrayRef
usingSortOptions
provided for each array. - Sort elements lexicographically from a list of
ArrayRef
into an unsigned integer (UInt32Array
) of indices. - Helper function to perform math lambda function on values from two arrays. If either left or right value is null then the output value is also null, so
1 + null
isnull
. - Returns the maximum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value
- Returns the max of values in the array of
ArrowNumericType
type, or dictionary array with value ofArrowNumericType
type. - Returns the maximum value in the binary array, according to the natural order.
- Returns the maximum value in the boolean array
- Returns the maximum value in the string array, according to the natural order.
- Returns the minimum value in the array, according to the natural order. For floating point arrays any NaN values are considered to be greater than any other non-null value
- Returns the min of values in the array of
ArrowNumericType
type, or dictionary array with value ofArrowNumericType
type. - Returns the minimum value in the binary array, according to the natural order.
- Returns the minimum value in the boolean array.
- Returns the minimum value in the string array, according to the natural order.
- Extracts the minutes of a given temporal primitive array as an array of integers
- Extracts the minutes of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Perform
left % right
operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will beErr(ArrowError::DivideByZero)
. - Perform
left % right
operation on two arrays. If either left or right value is null then the result is also null. If any right hand value is zero then the result of this operation will beErr(ArrowError::DivideByZero)
. - Modulus every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be
Err(ArrowError::DivideByZero)
. - Modulus every value in an array by a scalar. If any value in the array is null then the result is also null. If the scalar is zero then the result of this operation will be
Err(ArrowError::DivideByZero)
. - Extracts the month of a given temporal primitive array as an array of integers within the range of [1, 12].
- Extracts the month of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Perform
left * right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two decimal arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two decimal arrays. If either left or right value is null then the result is also null. - Perform
left * right
operation on two decimal arrays. If either left or right value is null then the result is also null. - Multiply every value in an array by a scalar. If any value in the array is null then the result is also null.
- Multiply every value in an array by a scalar. If any value in the array is null then the result is also null.
- Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Multiply every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Extracts the nanoseconds of a given temporal primitive array as an array of integers
- Extracts the nanoseconds of a given temporal primitive array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Perform
-
operation on an array. If value is null then the result is also null. - Perform
-
operation on an array. If value is null then the result is also null. - Performs unary
NOT
operation on an arrays. If value is null then the result is also null. - Copies original array, setting validity bit to false if a secondary comparison boolean array is set to true
- Extracts the day of week of a given temporal primitive array as an array of integers.
- Extracts the day of week of a given temporal array as an array of integers.
- Extracts the day of week of a given temporal primitive array as an array of integers, starting at Sunday.
- Extracts the day of week of a given temporal array as an array of integers, starting at Sunday.
- Performs
OR
operation on two arrays. If either left or right value is null then the result is also null. - Logical ‘or’ boolean values with Kleene logic
- It’s unstable_sort, may not preserve the order of equal elements
- Remove null values by do a bitmask AND operation with null bits and the boolean bits.
- Extracts the quarter of a given temporal primitive array as an array of integers within the range of [1, 4].
- Extracts the quarter of a given temporal array as an array of integersa within the range of [1, 4]. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Perform SQL
array ~ regex_array
operation onStringArray
/LargeStringArray
. Ifregex_array
element has an empty value, the corresponding result value is always true. - Extract all groups matched by a regular expression for a given String array.
- Extracts the seconds of a given temporal primitive array as an array of integers
- Extracts the seconds of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Shifts array by defined number of items (to left or right) A positive value for
offset
shifts the array to the right a negative value shifts the array to the left. - Sort the
ArrayRef
usingSortOptions
. - Sort the
ArrayRef
partially. - Sort elements from
ArrayRef
into an unsigned integer (UInt32Array
) of indices. For floating point arrays any NaN values are considered to be greater than any other non-null value.limit
is an option for partial_sort. - Perform
left - right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left - right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left - right
operation on two arrays. If either left or right value is null then the result is also null. - Perform
left - right
operation on two arrays. If either left or right value is null then the result is also null. - Subtract every value in an array by a scalar. If any value in the array is null then the result is also null.
- Subtract every value in an array by a scalar. If any value in the array is null then the result is also null.
- Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Subtract every value in an array by a scalar. If any value in the array is null then the result is also null. The given array must be a
PrimitiveArray
of the type same as the scalar, or aDictionaryArray
of the value type same as the scalar. - Returns the sum of values in the primitive array.
- Returns the sum of values in the array.
- Returns the sum of values in the array.
- Returns the sum of values in the primitive array.
- Applies the provided fallible binary operation across
a
andb
, returning any error, and collecting the results into aPrimitiveArray
. If any index is null in eithera
orb
, the corresponding index in the result will also be null - Applies the provided fallible binary operation across
a
andb
by mutating the mutablePrimitiveArray
a
with the results, returning any error. If any index is null in eithera
orb
, the corresponding index in the result will also be null - Applies a fallible unary function to an array with primitive values.
- Applies an infallible unary function to an array with primitive values.
- using_chrono_tz_and_utc_naive_date_timeDeprecatedParse the given string into a string representing fixed-offset that is correct as of the given UTC NaiveDateTime. Note that the offset is function of time and can vary depending on whether daylight savings is in effect or not. e.g. Australia/Sydney is +10:00 or +11:00 depending on DST.
- Extracts the week of a given temporal primitive array as an array of integers
- Extracts the week of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned. - Extracts the years of a given temporal primitive array as an array of integers
- Extracts the years of a given temporal array as an array of integers. If the given array isn’t temporal primitive or dictionary array, an
Err
will be returned.
Type Definitions
- FilterDeprecatedFunction that can filter arbitrary arrays