Trait StructArrayExt

Source

pub trait StructArrayExt {
    // Required methods
    fn normalize_slicing(&self) -> Result<Self, ArrowError>
       where Self: Sized;
    fn pushdown_nulls(&self) -> Result<Self, ArrowError>
       where Self: Sized;
}

Required Methods§

Source

fn normalize_slicing(&self) -> Result<Self, ArrowError>
where Self: Sized,

Removes the offset / length of the struct array by pushing it into the children

In arrow-rs when slice is called it recursively slices the children. In arrow-cpp when slice is called it just sets the offset/length of the struct array and leaves the children as-is

Both are legal approaches (╥﹏╥)

This method helps reduce complexity by folding into the arrow-rs approach

Source

fn pushdown_nulls(&self) -> Result<Self, ArrowError>
where Self: Sized,

Structs are allowed to mask valid items. For example, a struct array might be:

[ {“items”: [1, 2, 3]}, NULL, {“items”: NULL}]

However, the underlying items array might be: [[1, 2, 3], [4, 5], NULL]

The [4, 5] list is masked out because the struct array is null.

The struct validity would be [true, false, true] and the list validity would be [true, true, false]

This method pushes nulls down into all children. In the above example the list validity would become [true, false, false].

This method is not recursive. If a child is a struct array it will not push that child’s nulls down.

This method does not remove garbage lists. It only updates the validity so a future call to crate::list::ListArrayExt::filter_garbage_nulls will remove the garbage lists (without this pushdown it would not)