Enum Mask 

pub enum Mask {
    AllTrue(usize),
    AllFalse(usize),
    Values(Arc<MaskValues>),
}

Represents a set of sorted unique positive integers.

A Mask can be constructed from various representations, and converted to various others. Internally, these are cached.

Variants

AllTrue(usize)

All values are included.

AllFalse(usize)

No values are included.

Values(Arc<MaskValues>)

Some values are included, represented as a BitBuffer.

Implementations

impl Mask

pub fn intersect_by_rank(&self, mask: &Mask) -> Mask

Take the intersection of the mask with the set of true values in self.

We are more interested in low selectivity self (as indices) with a boolean buffer mask, so we don’t optimize for other cases, yet.

Note: we might be able to accelerate this function on x86 with BMI, see: https://www.microsoft.com/en-us/research/uploads/prod/2023/06/parquet-select-sigmod23.pdf

Examples

Keep the third and fifth set values from mask m1:

use vortex_mask::Mask;

let m1 = Mask::from_iter([true, false, false, true, true, true, false, true]);
let m2 = Mask::from_iter([false, false, true, false, true]);
assert_eq!(
    m1.intersect_by_rank(&m2),
    Mask::from_iter([false, false, false, false, true, false, false, true])
);

impl Mask

pub fn iter_bools<F, T>(&self, f: F) -> T

where F: FnMut(&mut dyn Iterator<Item = bool>) -> T,

Provides a closure with an iterator over the boolean values of the mask.

This allows us to provide different implementations of the iterator based on the underlying representation of the mask, while avoiding a heap allocation to return a boxed iterator.

Note that bool iteration might not be the fastest way to achieve whatever is it you’re trying to do!

impl Mask

pub fn try_into_mut(self) -> Result<MaskMut, Mask>

Attempts to convert an immutable mask into a mutable one, returning an error of Self if the underlying BitBuffer data if there are any other references.

pub fn into_mut(self) -> MaskMut

Convert an immutable mask into a mutable one, cloning the underlying BitBuffer data if there are any other references.

impl Mask

pub fn new(length: usize, value: bool) -> Mask

Create a new Mask with the given length.

pub fn new_true(length: usize) -> Mask

Create a new Mask where all values are set.

pub fn new_false(length: usize) -> Mask

Create a new Mask where no values are set.

pub fn from_buffer(buffer: BitBuffer) -> Mask

Create a new Mask from a BitBuffer.

pub fn from_indices(len: usize, indices: Vec<usize>) -> Mask

Create a new Mask from a Vec<usize>.

pub fn from_excluded_indices( len: usize, indices: impl IntoIterator<Item = usize>, ) -> Mask

Create a new Mask from an IntoIterator<Item = usize> of indices to be excluded.

pub fn from_slices(len: usize, vec: Vec<(usize, usize)>) -> Mask

Create a new Mask from a Vec<(usize, usize)> where each range represents a contiguous range of true values.

pub fn from_intersection_indices( len: usize, lhs: impl Iterator<Item = usize>, rhs: impl Iterator<Item = usize>, ) -> Mask

Create a new Mask from the intersection of two indices slices.

pub fn clear(&mut self)

Clears the mask of all data. Drops any allocated capacity.

pub fn len(&self) -> usize

Returns the length of the mask (not the number of true values).

pub fn is_empty(&self) -> bool

Returns true if the mask is empty i.e., it’s length is 0.

pub fn true_count(&self) -> usize

Get the true count of the mask.

pub fn false_count(&self) -> usize

Get the false count of the mask.

pub fn all_true(&self) -> bool

Returns true if all values in the mask are true.

pub fn all_false(&self) -> bool

Returns true if all values in the mask are false.

pub fn density(&self) -> f64

Return the density of the full mask.

pub fn value(&self, idx: usize) -> bool

Returns the boolean value at a given index.

Panics

Panics if the index is out of bounds.

pub fn first(&self) -> Option<usize>

Returns the first true index in the mask.

pub fn rank(&self, n: usize) -> usize

Returns the position in the mask of the nth true value.

pub fn slice(&self, range: impl RangeBounds<usize>) -> Mask

Slice the mask.

pub fn bit_buffer(&self) -> AllOr<&BitBuffer>

Return the boolean buffer representation of the mask.

pub fn to_bit_buffer(&self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn into_bit_buffer(self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn indices(&self) -> AllOr<&[usize]>

Return the indices representation of the mask.

pub fn slices(&self) -> AllOr<&[(usize, usize)]>

Return the slices representation of the mask.

pub fn threshold_iter(&self, threshold: f64) -> AllOr<MaskIter<'_>>

Return an iterator over either indices or slices of the mask based on a density threshold.

pub fn values(&self) -> Option<&MaskValues>

Return MaskValues if the mask is not all true or all false.

pub fn valid_counts_for_indices(&self, indices: &[usize]) -> Vec<usize>

Given monotonically increasing indices in [0, n_rows], returns the count of valid elements up to each index.

This is O(n_rows).

pub fn limit(self, limit: usize) -> Mask

Limit the mask to the first limit true values

pub fn concat<'a>( masks: impl Iterator<Item = &'a Mask>, ) -> Result<Mask, VortexError>

Concatenate multiple masks together into a single mask.

Trait Implementations

impl ArrayEq for Mask

fn array_eq(&self, other: &Mask, precision: Precision) -> bool

impl ArrayHash for Mask

fn array_hash<H>(&self, state: &mut H, precision: Precision)

where H: Hasher,

impl BitAnd for &Mask

type Output = Mask

The resulting type after applying the & operator.

fn bitand(self, rhs: &Mask) -> <&Mask as BitAnd>::Output

Performs the & operation.

impl BitOr for &Mask

type Output = Mask

The resulting type after applying the | operator.

fn bitor(self, rhs: &Mask) -> <&Mask as BitOr>::Output

Performs the | operation.

impl Clone for Mask

fn clone(&self) -> Mask

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Mask

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for Mask

fn default() -> Mask

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Mask

fn deserialize<__D>( __deserializer: __D, ) -> Result<Mask, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<const NB: usize> Filter<BitView<'_, NB>> for &Mask

type Output = Mask

The result type after performing the operation.

fn filter( self, selection: &BitView<'_, NB>, ) -> <&Mask as Filter<BitView<'_, NB>>>::Output

Filters an object using the provided mask, returning a new value.

impl<T> Filter<Mask> for &[T]

where T: Copy,

type Output = Buffer<T>

The result type after performing the operation.

fn filter(self, selection_mask: &Mask) -> Buffer<T>

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &BitBuffer

type Output = BitBuffer

The result type after performing the operation.

fn filter(self, selection_mask: &Mask) -> BitBuffer

Filters an object using the provided mask, returning a new value.

impl<T> Filter<Mask> for &Buffer<T>

where T: Copy,

type Output = Buffer<T>

The result type after performing the operation.

fn filter(self, selection_mask: &Mask) -> Buffer<T>

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &Mask

type Output = Mask

The result type after performing the operation.

fn filter(self, selection_mask: &Mask) -> Mask

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &NullVector

type Output = NullVector

The result type after performing the operation.

fn filter(self, selection: &Mask) -> <&NullVector as Filter<Mask>>::Output

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &Vector

type Output = Vector

The result type after performing the operation.

fn filter(self, selection: &Mask) -> <&Vector as Filter<Mask>>::Output

Filters an object using the provided mask, returning a new value.

impl<T> Filter<Mask> for &mut [T]

where T: Copy,

type Output = &mut [T]

The result type after performing the operation.

fn filter(self, selection: &Mask) -> <&mut [T] as Filter<Mask>>::Output

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &mut BitBufferMut

type Output = ()

The result type after performing the operation.

fn filter(self, selection_mask: &Mask)

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &mut MaskMut

type Output = ()

The result type after performing the operation.

fn filter(self, selection_mask: &Mask)

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &mut NullVectorMut

type Output = ()

The result type after performing the operation.

fn filter( self, selection: &Mask, ) -> <&mut NullVectorMut as Filter<Mask>>::Output

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for &mut VectorMut

type Output = ()

The result type after performing the operation.

fn filter(self, selection: &Mask) -> <&mut VectorMut as Filter<Mask>>::Output

Filters an object using the provided mask, returning a new value.

impl Filter<Mask> for NullVector

type Output = NullVector

The result type after performing the operation.

fn filter(self, selection: &Mask) -> NullVector

Filters an object using the provided mask, returning a new value.

impl<'a> From<&'a Mask> for Input<'a>

fn from(value: &'a Mask) -> Input<'a>

Converts to this type from the input type.

impl From<BitBuffer> for Mask

fn from(value: BitBuffer) -> Mask

Converts to this type from the input type.

impl From<Mask> for Option<NullBuffer>

fn from(value: Mask) -> Option<NullBuffer>

Converts to this type from the input type.

impl FromIterator<Mask> for Mask

fn from_iter<T>(iter: T) -> Mask

where T: IntoIterator<Item = Mask>,

Creates a value from an iterator.

impl FromIterator<Mask> for Validity

fn from_iter<T>(iter: T) -> Validity

where T: IntoIterator<Item = Mask>,

Creates a value from an iterator.

impl FromIterator<bool> for Mask

fn from_iter<T>(iter: T) -> Mask

where T: IntoIterator<Item = bool>,

Creates a value from an iterator.

impl IntoArray for Mask

fn into_array(self) -> Arc<dyn Array>

impl Not for &Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> <&Mask as Not>::Output

Performs the unary ! operation.

impl Not for Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> <Mask as Not>::Output

Performs the unary ! operation.

impl PartialEq for Mask

fn eq(&self, other: &Mask) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Mask

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<I> Take<[I]> for &Mask

where I: UnsignedPType,

type Output = Mask

The result type after performing the operation.

fn take(self, indices: &[I]) -> Mask

Creates a new object using the elements from the input indexed by indices.

impl<I> Take<PVector<I>> for &Mask

where I: UnsignedPType,

fn take(self, indices: &PVector<I>) -> Mask

Implementation of take on Mask that is null-aware.

If an index is specified as null by the PVector, then the taken mask value is set to false.

This is useful for many of the take implementations for vectors.

type Output = Mask

The result type after performing the operation.

impl Eq for Mask