Enum Mask

pub enum Mask {
    AllTrue(usize),
    AllFalse(usize),
    Values(Arc<MaskValues>),
}

Represents a set of sorted unique positive integers.

A Mask can be constructed from various representations, and converted to various others. Internally, these are cached.

Variants

AllTrue(usize)

All values are included.

AllFalse(usize)

No values are included.

Values(Arc<MaskValues>)

Some values are included, represented as a BooleanBuffer.

Implementations

impl Mask

pub fn intersect_by_rank(&self, mask: &Mask) -> Mask

Take the intersection of the mask with the set of true values in self.

We are more interested in low selectivity self (as indices) with a boolean buffer mask, so we don’t optimize for other cases, yet.

Note: we might be able to accelerate this function on x86 with BMI, see: https://www.microsoft.com/en-us/research/uploads/prod/2023/06/parquet-select-sigmod23.pdf

Examples

Keep the third and fifth set values from mask m1:

use vortex_mask::Mask;

let m1 = Mask::from_iter([true, false, false, true, true, true, false, true]);
let m2 = Mask::from_iter([false, false, true, false, true]);
assert_eq!(
    m1.intersect_by_rank(&m2),
    Mask::from_iter([false, false, false, false, true, false, false, true])
);

impl Mask

pub fn iter_bools<F, T>(&self, f: F) -> T

where F: FnMut(&mut dyn Iterator<Item = bool>) -> T,

Provides a closure with an iterator over the boolean values of the mask.

This allows us to provide different implementations of the iterator based on the underlying representation of the mask, while avoiding a heap allocation to return a boxed iterator.

Note that bool iteration might not be the fastest way to achieve whatever is it you’re trying to do!

impl Mask

pub fn new_true(length: usize) -> Self

Create a new Mask where all values are set.

pub fn new_false(length: usize) -> Self

Create a new Mask where no values are set.

pub fn from_buffer(buffer: BooleanBuffer) -> Self

Create a new Mask from a BooleanBuffer.

pub fn from_indices(len: usize, indices: Vec<usize>) -> Self

Create a new Mask from a Vec<usize>.

pub fn from_excluded_indices( len: usize, indices: impl IntoIterator<Item = usize>, ) -> Self

Create a new Mask from an [IntoIterator<Item = usize>] of indices to be excluded.

pub fn from_slices(len: usize, vec: Vec<(usize, usize)>) -> Self

Create a new Mask from a [Vec<(usize, usize)>] where each range represents a contiguous range of true values.

pub fn from_intersection_indices( len: usize, lhs: impl Iterator<Item = usize>, rhs: impl Iterator<Item = usize>, ) -> Self

Create a new Mask from the intersection of two indices slices.

pub fn len(&self) -> usize

Returns the length of the mask (not the number of true values).

pub fn true_count(&self) -> usize

Get the true count of the mask.

pub fn false_count(&self) -> usize

Get the false count of the mask.

pub fn all_true(&self) -> bool

Returns true if all values in the mask are true.

pub fn all_false(&self) -> bool

Returns true if all values in the mask are false.

pub fn density(&self) -> f64

Return the density of the full mask.

pub fn value(&self, idx: usize) -> bool

Returns the boolean value at a given index.

Panics

Panics if the index is out of bounds.

pub fn first(&self) -> Option<usize>

Returns the first true index in the mask.

pub fn slice(&self, offset: usize, length: usize) -> Self

Slice the mask.

pub fn boolean_buffer(&self) -> AllOr<&BooleanBuffer>

Return the boolean buffer representation of the mask.

pub fn to_boolean_buffer(&self) -> BooleanBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn to_null_buffer(&self) -> Option<NullBuffer>

Returns an Arrow null buffer representation of the mask.

pub fn indices(&self) -> AllOr<&[usize]>

Return the indices representation of the mask.

pub fn slices(&self) -> AllOr<&[(usize, usize)]>

Return the slices representation of the mask.

pub fn threshold_iter(&self, threshold: f64) -> AllOr<MaskIter<'_>>

Return an iterator over either indices or slices of the mask based on a density threshold.

pub fn values(&self) -> Option<&MaskValues>

Return MaskValues if the mask is not all true or all false.

Trait Implementations

impl BitAnd for &Mask

type Output = Mask

The resulting type after applying the & operator.

fn bitand(self, rhs: Self) -> Self::Output

Performs the & operation.

impl Clone for Mask

fn clone(&self) -> Mask

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Mask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<BooleanBuffer> for Mask

fn from(value: BooleanBuffer) -> Self

Converts to this type from the input type.

impl FromIterator<Mask> for Mask

fn from_iter<T: IntoIterator<Item = Mask>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<bool> for Mask

fn from_iter<T: IntoIterator<Item = bool>>(iter: T) -> Self

Creates a value from an iterator.

impl Not for &Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl PartialEq for Mask

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Mask