Enum Mask 

pub enum Mask {
    AllTrue(usize),
    AllFalse(usize),
    Values(Arc<MaskValues>),
}

Represents a set of sorted unique positive integers. If a value is included in a Mask, it’s valid.

A Mask can be constructed from various representations, and converted to various others. Internally, these are cached.

Variants

AllTrue(usize)

All values are included.

AllFalse(usize)

No values are included.

Values(Arc<MaskValues>)

Some values are included, represented as a BitBuffer.

Implementations

impl Mask

pub fn bitand_not(self, rhs: &Mask) -> Mask

Computes self & !rhs (AND NOT), equivalent to set difference.

impl Mask

pub fn intersect_by_rank(&self, mask: &Mask) -> Mask

Take the intersection of the mask with the set of true values in self.

We are more interested in low selectivity self (as indices) with a boolean buffer mask, so we don’t optimize for other cases, yet.

Note: we might be able to accelerate this function on x86 with BMI, see: https://www.microsoft.com/en-us/research/uploads/prod/2023/06/parquet-select-sigmod23.pdf

Examples

Keep the third and fifth set values from mask m1:

use vortex_mask::Mask;

let m1 = Mask::from_iter([true, false, false, true, true, true, false, true]);
let m2 = Mask::from_iter([false, false, true, false, true]);
assert_eq!(
    m1.intersect_by_rank(&m2),
    Mask::from_iter([false, false, false, false, true, false, false, true])
);

impl Mask

pub fn new(length: usize, value: bool) -> Self

Create a new Mask with the given length.

pub fn new_true(length: usize) -> Self

Create a new Mask where all values are set.

pub fn new_false(length: usize) -> Self

Create a new Mask where no values are set.

pub fn from_buffer(buffer: BitBuffer) -> Self

Create a new Mask from a BitBuffer.

pub fn from_indices(len: usize, indices: Vec<usize>) -> Self

Create a new Mask from a Vec<usize>.

pub fn from_excluded_indices( len: usize, indices: impl IntoIterator<Item = usize>, ) -> Self

Create a new Mask from an IntoIterator<Item = usize> of indices to be excluded.

pub fn from_slices(len: usize, vec: Vec<(usize, usize)>) -> Self

Create a new Mask from a Vec<(usize, usize)> where each range represents a contiguous range of true values.

pub fn from_intersection_indices( len: usize, lhs: impl Iterator<Item = usize>, rhs: impl Iterator<Item = usize>, ) -> Self

Create a new Mask from the intersection of two indices slices.

pub fn clear(&mut self)

Clears the mask of all data. Drops any allocated capacity.

pub fn len(&self) -> usize

Returns the length of the mask (not the number of true values).

pub fn is_empty(&self) -> bool

Returns true if the mask is empty i.e., it’s length is 0.

pub fn true_count(&self) -> usize

Get the true count of the mask.

pub fn false_count(&self) -> usize

Get the false count of the mask.

pub fn all_true(&self) -> bool

Returns true if all values in the mask are true.

pub fn all_false(&self) -> bool

Returns true if all values in the mask are false.

pub fn density(&self) -> f64

Return the density of the full mask.

pub fn value(&self, idx: usize) -> bool

Returns the boolean value at a given index.

Panics

Panics if the index is out of bounds.

pub fn first(&self) -> Option<usize>

Returns the first true index in the mask.

pub fn last(&self) -> Option<usize>

Returns the last true index in the mask.

pub fn rank(&self, n: usize) -> usize

Returns the position in the mask of the nth true value.

pub fn slice(&self, range: impl RangeBounds<usize>) -> Self

Slice the mask.

pub fn bit_buffer(&self) -> AllOr<&BitBuffer>

Return the boolean buffer representation of the mask.

pub fn to_bit_buffer(&self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn into_bit_buffer(self) -> BitBuffer

Return a boolean buffer representation of the mask, allocating new buffers for all-true and all-false variants.

pub fn indices(&self) -> AllOr<&[usize]>

Return the indices representation of the mask.

pub fn slices(&self) -> AllOr<&[(usize, usize)]>

Return the slices representation of the mask.

pub fn threshold_iter(&self, threshold: f64) -> AllOr<MaskIter<'_>>

Return an iterator over either indices or slices of the mask based on a density threshold.

pub fn values(&self) -> Option<&MaskValues>

Return MaskValues if the mask is not all true or all false.

pub fn valid_counts_for_indices(&self, indices: &[usize]) -> Vec<usize>

Given monotonically increasing indices in [0, n_rows], returns the count of valid elements up to each index.

This is O(n_rows).

pub fn limit(self, limit: usize) -> Self

Limit the mask to the first limit true values

pub fn concat<'a>(masks: impl Iterator<Item = &'a Self>) -> VortexResult<Self>

Concatenate multiple masks together into a single mask.

Trait Implementations

impl BitAnd for &Mask

type Output = Mask

The resulting type after applying the & operator.

fn bitand(self, rhs: Self) -> Self::Output

Performs the & operation.

impl BitOr for &Mask

type Output = Mask

The resulting type after applying the | operator.

fn bitor(self, rhs: Self) -> Self::Output

Performs the | operation.

impl Clone for Mask

fn clone(&self) -> Mask

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Mask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Mask

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Mask

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<BitBuffer> for Mask

fn from(value: BitBuffer) -> Self

Converts to this type from the input type.

impl FromIterator<Mask> for Mask

fn from_iter<T: IntoIterator<Item = Mask>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<bool> for Mask

fn from_iter<T: IntoIterator<Item = bool>>(iter: T) -> Self

Creates a value from an iterator.

impl Not for &Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl Not for Mask

type Output = Mask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.

impl PartialEq for Mask

fn eq(&self, other: &Self) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for Mask

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for Mask