Struct lance_core::utils::mask::RowIdMask

pub struct RowIdMask {
    pub allow_list: Option<RowIdTreeMap>,
    pub block_list: Option<RowIdTreeMap>,
}

A row id mask to select or deselect particular row ids

If both the allow_list and the block_list are Some then the only selected row ids are those that are in the allow_list but not in the block_list (the block_list takes precedence)

If both the allow_list and the block_list are None (the default) then all row ids are selected

Fields

allow_list: Option<RowIdTreeMap>

If Some then only these row ids are selected

block_list: Option<RowIdTreeMap>

If Some then these row ids are not selected.

Implementations

impl RowIdMask

pub fn all_rows() -> Self

pub fn allow_nothing() -> Self

pub fn from_allowed(allow_list: RowIdTreeMap) -> Self

pub fn from_block(block_list: RowIdTreeMap) -> Self

pub fn selected(&self, row_id: u64) -> bool

True if the row_id is selected by the mask, false otherwise

pub fn selected_indices(&self, row_ids: &[u64]) -> Vec<u64>

Return the indices of the input row ids that were valid

pub fn also_block(self, block_list: RowIdTreeMap) -> Self

Also block the given ids

pub fn also_allow(self, allow_list: RowIdTreeMap) -> Self

Also allow the given ids

pub fn into_arrow(&self) -> Result<BinaryArray>

Converst a mask into an arrow array

A row id mask is not very arrow-compatible. We can’t make it a batch with two columns because the block list and allow list will have different lengths. Also, there is no Arrow type for compressed bitmaps.

However, we need to shove it into some kind of Arrow container to pass it along the datafusion stream. Perhaps, in the future, we can add row id masks as first class types in datafusion, and this can be passed along as a mask / selection vector.

We serialize this as a variable length binary array with two items. The first item is the block list and the second item is the allow list.

pub fn from_arrow(array: &GenericBinaryArray<i32>) -> Result<Self>

Deserialize a row id mask from Arrow

Trait Implementations

impl BitAnd for RowIdMask

type Output = RowIdMask

The resulting type after applying the & operator.

fn bitand(self, rhs: Self) -> Self::Output

Performs the & operation.

impl BitOr for RowIdMask

type Output = RowIdMask

The resulting type after applying the | operator.

fn bitor(self, rhs: Self) -> Self::Output

Performs the | operation.

impl Clone for RowIdMask

fn clone(&self) -> RowIdMask

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RowIdMask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RowIdMask

fn default() -> RowIdMask

Returns the “default value” for a type.

impl Not for RowIdMask

type Output = RowIdMask

The resulting type after applying the ! operator.

fn not(self) -> Self::Output

Performs the unary ! operation.