Module compute

Compute kernels on top of Vortex Arrays.

We aim to provide a basic set of compute kernels that can be used to efficiently index, slice, and filter Vortex Arrays in their encoded forms.

Every array encoding has the ability to implement their own efficient implementations of these operators, else we will decode, and perform the equivalent operator from Arrow.

Structs

BetweenKernelAdapter

BetweenKernelRef

BetweenOptions

BooleanKernelAdapter

BooleanKernelRef

CastKernelAdapter

CastKernelRef

CompareKernelAdapter

CompareKernelRef

ComputeFn

An instance of a compute function holding the implementation vtable and a set of registered compute kernels.

FilterKernelAdapter

Adapter to convert a FilterKernel into a Kernel.

FilterKernelRef

A kernel that implements the filter function.

InvertKernelAdapter

InvertKernelRef

InvocationArgs

Arguments to a compute function invocation.

IsConstantOpts

Configuration for is_constant_opts operations.

LikeOptions

Options for SQL LIKE function

MaskKernelAdapter

MaskKernelRef

MinMaxResult

NumericKernelAdapter

NumericKernelRef

SumKernelAdapter

SumKernelRef

Enums

BooleanOperator

Input

Input to a compute function.

Operator

Output

Output from a compute function.

SearchResult

Result of performing search_sorted on an Array

SearchSortedSide

StrictComparison

Statics

BETWEEN_FN

BOOLEAN_FN

CAST_FN

COMPARE_FN

FILTER_FN

The filter ComputeFn.

INVERT_FN

MASK_FN

NUMERIC_FN

SUM_FN

Traits

BetweenKernel

BooleanKernel

CastKernel

CompareKernel

ComputeFnVTable

VTable for the implementation of a compute function.

FillNullFn

Implementation of fill_null for an encoding.

FilterKernel

IndexOrd

InvertKernel

IsConstantFn

IsSortedFn

IsSortedIteratorExt

Helper methods to check sortedness with strictness

Kernel

Compute functions can ask arrays for compute kernels for a given invocation.

LikeFn

MaskKernel

MinMaxFn

Computes the min and max of an array, returning the (min, max) values If the array is empty or has only nulls, the result is None.

NumericKernel

OptimizeFn

Options

Options for a compute function invocation.

ScalarAtFn

Implementation of scalar_at for an encoding.

SearchSorted

SearchSortedFn

Searches for value assuming the array is sorted.

SearchSortedUsizeFn

SliceFn

Limit array to start…stop range

SumKernel

TakeFn

TakeFromFn

ToArrowFn

Trait for Arrow conversion compute function.

UncompressedSizeFn

Functions

add

Point-wise add two numeric arrays.

add_scalar

Point-wise add a scalar value to this array on the right-hand-side.

and

Point-wise logical and between two Boolean arrays.

and_kleene

Point-wise Kleene logical and between two Boolean arrays.

between

Compute between (a <= x <= b), this can be implemented using compare and boolean and but this will likely have a lower runtime.

boolean

Point-wise logical operator between two Boolean arrays.

cast

Attempt to cast an array to a desired DType.

compare

Compares two arrays and returns a new boolean array with the result of the comparison. Or, returns None if comparison is not supported for these arrays.

compare_lengths_to_empty

Helper function to compare empty values with arrays that have external value length information like VarBin.

div

Point-wise divide two numeric arrays.

div_scalar

Point-wise divide a scalar value into this array on the right-hand-side.

fill_null

filter

Keep only the elements for which the corresponding mask value is true.

invert

Logically invert a boolean array.

is_constant

Computes whether an array has constant values. If the array’s encoding doesn’t implement the relevant VTable, it’ll try and canonicalize in order to make a determination. An array is constant IFF at least one of the following conditions apply:

is_constant_opts

Computes whether an array has constant values. Configurable by IsConstantOpts.

is_sorted

is_strict_sorted

like

Perform SQL left LIKE right

mask

Replace values with null where the mask is true.

min_max

Computes the min & max of an array, returning the (min, max) values The return values are (min, max) scalars, where None indicates that the value is non-existent (e.g. for an empty array). The return value dtype is the non-nullable version of the array dtype.

mul

Point-wise multiply two numeric arrays.

mul_scalar

Point-wise multiply a scalar value into this array on the right-hand-side.

numeric

Point-wise numeric operation between two arrays of the same type and length.

optimize

or

Point-wise logical or between two Boolean arrays.

or_kleene

Point-wise Kleene logical or between two Boolean arrays.

preferred_arrow_data_type

Return the preferred Arrow DataType of the array.

scalar_at

scalar_cmp

search_sorted

search_sorted_many

Search for many elements in the array.

search_sorted_usize

search_sorted_usize_many

slice

Return a zero-copy slice of an array, between start (inclusive) and end (exclusive).

sub

Point-wise subtract two numeric arrays.

sub_scalar

Point-wise subtract a scalar value from this array on the right-hand-side.

sum

Sum an array.

sum_impl

Sum an array.

take

take_into

to_arrow

Convert the array to an Arrow array of the given type.

to_arrow_preferred

uncompressed_size

Computes the uncompressed size of the array, in bytes.