Module compute

Compute kernels on top of Vortex Arrays.

We aim to provide a basic set of compute kernels that can be used to efficiently index, slice, and filter Vortex Arrays in their encoded forms.

Every array encoding has the ability to implement their own efficient implementations of these operators, else we will decode, and perform the equivalent operator from Arrow.

Structs

BetweenKernelAdapter

BetweenKernelRef

BetweenOptions

BinaryArgs

For binary compute functions, it’s useful to just have this short-cut.

BooleanKernelAdapter

BooleanKernelRef

CastKernelAdapter

CastKernelRef

CompareKernelAdapter

CompareKernelRef

ComputeFn

An instance of a compute function holding the implementation vtable and a set of registered compute kernels.

FillNullKernelAdapter

FillNullKernelRef

FilterKernelAdapter

Adapter to convert a FilterKernel into a Kernel.

FilterKernelRef

A kernel that implements the filter function.

InvertKernelAdapter

InvertKernelRef

InvocationArgs

Arguments to a compute function invocation.

IsConstantKernelAdapter

IsConstantKernelRef

IsConstantOpts

Configuration for is_constant_opts operations.

IsSortedKernelAdapter

IsSortedKernelRef

LikeKernelAdapter

LikeKernelRef

LikeOptions

Options for SQL LIKE function

MaskKernelAdapter

MaskKernelRef

MinMax

MinMaxKernelAdapter

MinMaxKernelRef

MinMaxResult

NaNCountKernelAdapter

NaNCountKernelRef

NumericKernelAdapter

NumericKernelRef

SumKernelAdapter

SumKernelRef

Take

TakeFrom

TakeFromKernelAdapter

TakeFromKernelRef

TakeKernelAdapter

TakeKernelRef

A kernel that implements the filter function.

UnaryArgs

For unary compute functions, it’s useful to just have this short-cut.

Enums

BooleanOperator

Cost

When calling is_constant the children are all checked for constantness. This enum decide at each precision/cost level the constant check should run as. The cost increase as we move down the list.

Input

Input to a compute function.

Operator

Output

Output from a compute function.

StrictComparison

Statics

BETWEEN_FN

BOOLEAN_FN

CAST_FN

COMPARE_FN

FILL_NULL_FN

FILTER_FN

The filter ComputeFn.

INVERT_FN

IS_CONSTANT_FN

IS_SORTED_FN

LIKE_FN

MASK_FN

MIN_MAX_FN

NAN_COUNT_FN

NUMERIC_FN

SUM_FN

TAKE_FN

TAKE_FROM_FN

Traits

BetweenKernel

BooleanKernel

CastKernel

CompareKernel

ComputeFnVTable

VTable for the implementation of a compute function.

FillNullKernel

FilterKernel

InvertKernel

IsConstantKernel

IsSortedIteratorExt

Helper methods to check sortedness with strictness

IsSortedKernel

Kernel

Compute functions can ask arrays for compute kernels for a given invocation.

LikeKernel

MaskKernel

MinMaxKernel

Computes the min and max of an array, returning the (min, max) values If the array is empty or has only nulls, the result is None.

NaNCountKernel

Computes the min and max of an array, returning the (min, max) values

NumericKernel

Options

Options for a compute function invocation.

SumKernel

TakeFromKernel

TakeKernel

Functions

add

Point-wise add two numeric arrays.

add_scalar

Point-wise add a scalar value to this array on the right-hand-side.

and

Point-wise logical and between two Boolean arrays.

and_kleene

Point-wise Kleene logical and between two Boolean arrays.

arrow_filter_fn

between

Compute between (a <= x <= b), this can be implemented using compare and boolean and but this will likely have a lower runtime.

boolean

Point-wise logical operator between two Boolean arrays.

cast

Attempt to cast an array to a desired DType.

compare

Compares two arrays and returns a new boolean array with the result of the comparison. Or, returns None if comparison is not supported for these arrays.

compare_lengths_to_empty

Helper function to compare empty values with arrays that have external value length information like VarBin.

div

Point-wise divide two numeric arrays.

div_scalar

Point-wise divide a scalar value into this array on the right-hand-side.

fill_null

filter

Keep only the elements for which the corresponding mask value is true.

invert

Logically invert a boolean array, preserving its validity.

is_constant

Computes whether an array has constant values. If the array’s encoding doesn’t implement the relevant VTable, it’ll try and canonicalize in order to make a determination.

is_constant_opts

Computes whether an array has constant values. Configurable by IsConstantOpts.

is_sorted

is_sorted_opts

is_strict_sorted

like

Perform SQL left LIKE right

list_contains

Compute a Bool-typed array the same length as array where elements is true if the list item contains the value, false otherwise.

list_elem_len

Returns a new array of u64 representing the length of each list element.

mask

Replace values with null where the mask is true.

min_max

Computes the min & max of an array, returning the (min, max) values The return values are (min, max) scalars, where None indicates that the value is non-existent (e.g. for an empty array). The return value dtype is the non-nullable version of the array dtype.

mul

Point-wise multiply two numeric arrays.

mul_scalar

Point-wise multiply a scalar value into this array on the right-hand-side.

nan_count

Computes the number of NaN values in the array.

numeric

Point-wise numeric operation between two arrays of the same type and length.

or

Point-wise logical or between two Boolean arrays.

or_kleene

Point-wise Kleene logical or between two Boolean arrays.

scalar_cmp

sub

Point-wise subtract two numeric arrays.

sub_scalar

Point-wise subtract a scalar value from this array on the right-hand-side.

sum

Sum an array.

sum_impl

Sum an array.

take