Module utils 

Utility Functions - SIMD Processing and Memory Management Utilities

Core utilities supporting SIMD kernel implementations with efficient memory handling, bitmask operations, and performance-critical helper functions.

Functions

bitmask_to_simd_mask

Extracts a core::SIMD Mask<M, N> for a batch of N lanes from a Minarrow Bitmask.

confirm_capacity

Validates that actual capacity matches expected capacity for kernel operations.

confirm_equal_len

Validates that two lengths are equal for binary kernel operations.

confirm_mask_capacity

Checks the mask capacity is large enough Used so we can avoid bounds checks in the hot loop

estimate_categorical_cardinality

Estimate cardinality ratio on a sample from a CategoricalArray. Used to quickly figure out the optimal strategy when comparing StringArray and CategoricalArrays.

estimate_string_cardinality

Estimate cardinality ratio on a sample from a StringArray. Used to quickly figure out the optimal strategy when comparing StringArray and CategoricalArrays.

format_finite

Strips ‘.0’ from concatenated decimal values so ‘Hello1.0’ becomes ‘Hello1’.

has_nulls

Determines whether nulls are present given an optional null count and mask reference. Avoids computing mask cardinality to preserve performance guarantees.

is_simd_aligned

SIMD Alignment check. Returns true if the slice is properly 64-byte aligned for SIMD operations, false otherwise.

merge_bitmasks_to_new

Merge two optional Bitmasks into a new output mask, computing per-row AND. Returns None if both inputs are None (output is dense).

simd_mask

Creates a SIMD mask from a bitmask window for vectorised conditional operations.

simd_mask_to_bitmask

Converts a SIMD Mask<M, N> to a Minarrow Bitmask for the given logical length. Used at the end of a block operation within SIMD-accelerated kernel functions.

write_global_bitmask_block

Bulk-ORs a local bitmask block (from a SIMD mask or similar) into the global Minarrow bitmask at the correct byte offset. The block (block_mask) is expected to contain at least ceil(n_lanes/8) bytes, with the bit-packed validity bits starting from position 0.