Enum Mask 

pub enum Mask {
    AllTrue(usize),
    AllFalse(usize),
    Values(Arc<MaskValues>),
}

Represents a set of sorted unique positive integers.

A Mask can be constructed from various representations, and converted to various others. Internally, these are cached.

Variants

AllTrue(usize)

All values are included.

AllFalse(usize)

No values are included.

Values(Arc<MaskValues>)

Some values are included, represented as a BitBuffer.

Implementations

impl Mask

pub fn intersect_by_rank(&self, mask: &Mask) -> Mask

Take the intersection of the mask with the set of true values in self.

We are more interested in low selectivity self (as indices) with a boolean buffer mask, so we don’t optimize for other cases, yet.

Note: we might be able to accelerate this function on x86 with BMI, see: https://www.microsoft.com/en-us/research/uploads/prod/2023/06/parquet-select-sigmod23.pdf

Examples

Keep the third and fifth set values from mask m1:

use vortex_mask::Mask;

let m1 = Mask::from_iter([true, false, false, true, true, true, false, true]);
let m2 = Mask::from_iter([false, false, true, false, true]);
assert_eq!(
    m1.intersect_by_rank(&m2),
    Mask::from_iter([false, false, false, false, true, false, false, true])
);

impl Mask

pub fn iter_bools<F, T>(&self, f: F) -> T

where F: FnMut(&mut dyn Iterator<Item = bool>) -> T,

Provides a closure with an iterator over the boolean values of the mask.

This allows us to provide different implementations of the iterator based on the underlying representation of the mask, while avoiding a heap allocation to return a boxed iterator.

Note that bool iteration might not be the fastest way to achieve whatever is it you’re trying to do!

impl Mask

pub fn try_into_mut(self) -> Result<MaskMut, Self>

Attempts to convert an immutable mask into a mutable one, returning an error of Self if the underlying BitBuffer data if there are any other references.

pub fn into_mut(self) -> MaskMut

Convert an immutable mask into a mutable one, cloning the underlying BitBuffer data if there are any other references.

impl Mask

pub fn new(length: usize, value: bool) -> Self

Create a new Mask with the given length.

pub fn new_true(length: usize) -> Self

Create a new Mask where all values are set.

pub fn new_false(length: usize) -> Self

Create a new Mask where no values are set.

pub fn from_buffer(buffer: BitBuffer) -> Self

Create a new Mask from a BitBuffer.

pub fn from_indices(len: usize, indices: Vec<usize>) -> Self

Create a new Mask from a Vec<usize>.

pub fn from_excluded_indices( len: usize, indices: impl IntoIterator<Item = usize>, ) -> Self

Create a new Mask from an IntoIterator<Item = usize> of indices to be excluded.

pub fn from_slices(len: usize, vec: Vec<(usize, usize)>) -> Self

Create a new Mask from a Vec<(usize, usize)> where each range represents a contiguous range of true values.

pub fn from_intersection_indices( len: usize, lhs: impl Iterator<Item = usize>, rhs: impl Iterator<Item = usize>, ) -> Self

Create a new Mask from the intersection of two indices slices.

pub fn clear(&mut self)

Clears the mask of all data. Drops any allocated capacity.

pub fn len(&self) -> usize

Returns the length of the mask (not the number of true values).

pub fn is_empty(&self) -> bool

Returns true if the mask is empty i.e., it’s length is 0.

pub fn true_count(&self) -> usize

Get the true count of the mask.

pub fn false_count(&self) -> usize

Get the false count of the mask.

pub fn all_true(&self) -> bool

Returns true if all values in the mask are true.

pub fn all_false(&self) -> bool

Returns true if all values in the mask are false.

pub fn density(&self) -> f64

Return the density of the full mask.

pub fn value(&self, idx: usize) -> bool

Returns the boolean value at a given index.

Panics

Panics if the index is out of bounds.

pub fn first(&self) -> Option<usize>

Returns the first true index in the mask.

pub fn rank(&self, n: usize) -> usize

Returns the position in the mask of the nth true value.

pub fn slice(&self, range: impl RangeBounds<usize>) -> Self

Slice the mask.

pub fn bit_buffer(&self) -> AllOr<&BitBuffer>

Return the boolean buffer representation of the mask.

pub fn to_bit_buffer(&self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn into_bit_buffer(self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn indices(&self) -> AllOr<&[usize]>

Return the indices representation of the mask.

pub fn slices(&self) -> AllOr<&[(usize, usize)]>

Return the slices representation of the mask.

pub fn threshold_iter(&self, threshold: f64) -> AllOr<MaskIter<'_>>

Return an iterator over either indices or slices of the mask based on a density threshold.

pub fn values(&self) -> Option<&MaskValues>

Return MaskValues if the mask is not all true or all false.

pub fn valid_counts_for_indices(&self, indices: &[usize]) -> Vec<usize>

Given monotonically increasing indices in [0, n_rows], returns the count of valid elements up to each index.

This is O(n_rows).

pub fn limit(self, limit: usize) -> Self

Limit the mask to the first limit true values

pub fn concat<'a>(masks: impl Iterator<Item = &'a Self>) -> VortexResult<Self>

Concatenate multiple masks together into a single mask.

Trait Implementations

impl BitAnd for &Mask

type Output = Mask

The resulting type after applying the & operator.

fn bitand(self, rhs: Self) -> Self::Output

Performs the & operation.

impl BitOr for &Mask

type Output = Mask

The resulting type after applying the | operator.

fn bitor(self, rhs: Self) -> Self::Output

Performs the | operation.

impl Clone for Mask

fn clone(&self) -> Mask

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Mask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Mask

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Mask

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<BitBuffer> for Mask

fn from(value: BitBuffer) -> Self

Converts to this type from the input type.

impl From<Mask> for Option<NullBuffer>

fn from(value: Mask) -> Self

Converts to this type from the input type.

impl FromIterator<Mask> for Mask

fn from_iter<T: IntoIterator<Item = Mask>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<bool> for Mask

fn from_iter<T: IntoIterator<Item = bool>>(iter: T) -> Self

Creates a value from an iterator.

impl Not for &Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl Not for Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl PartialEq for Mask

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Mask

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for Mask