Enum Validity

Source

pub enum Validity {
    AllValid,
    AllInvalid,
    Mask(Bitmap),
}

Expand description

Which values in a vector are valid.

Variants§

§

AllValid

Nothing is null. No mask is stored and the kernels that read this take the fast path.

§

AllInvalid

Everything is null. Most operators can answer without looking at the data at all.

§

Mask(Bitmap)

Some of each, one bit per value, set meaning valid.

Implementations§

Source §

impl Validity

Source

pub fn is_valid(&self, index: usize) -> bool

Whether the value at index is not null.

Out of range reads report invalid rather than panicking, because this is called from kernels that are allowed to read past the end of a partially filled vector.

Source

pub fn has_nulls(&self, len: usize) -> bool

Whether any value in the first len is null.

Source

pub fn count_valid(&self, len: usize) -> usize

How many of the first len values are not null.

Source

pub fn normalize(self, len: usize) -> Self

Collapses a mask that turned out to be uniform back to one of the flag cases.

Worth doing at the end of any operation that builds a mask, because every kernel downstream then gets to take the branch it wants rather than walking a bitmap to find out what it already could have been told.

Source

pub fn with_null(self, index: usize, len: usize) -> Self

The validity of a vector where index has just been made null.

Takes and returns by value because setting a null on an AllValid vector has to materialize a mask, and hiding that behind &mut self hides an allocation.

Source

pub fn from_iter(len: usize, valid: impl Fn(usize) -> bool) -> Self

Validity built from a per-value predicate, normalized.

Source

pub fn from_run(valid: &[bool]) -> Self

Validity packed from one byte a row, which is what a kernel that accumulated its answer in a Vec<bool> is holding when it finishes.

The difference from Self::from_iter is the shape rather than the answer. from_iter calls a closure and then a read modify write on a byte of the bitmap, once per row, and the read modify write is a dependency on the row before it. This reads sixty four bytes and writes one word, which has no dependency in it at all and is what the compiler needs to see before it will use a vector instruction. On a thousand row vector that is the difference between two nanoseconds a row and something too small to measure. The bits past the end of the last word are set rather than clear, which looks like a detail and is not. Bitmap does not carry a length, so its equality is over whole words, and Bitmap::all_valid leaves those bits set. A constructor that left them clear would build a validity that says exactly the same thing about every row that exists and still compares unequal to the one Self::from_iter builds, which is a test failure with no wrong answer in it and an afternoon to work out.

Source