pub trait StructArrayExt {
// Required methods
fn normalize_slicing(&self) -> Result<Self, ArrowError>
where Self: Sized;
fn pushdown_nulls(&self) -> Result<Self, ArrowError>
where Self: Sized;
}Required Methods§
Sourcefn normalize_slicing(&self) -> Result<Self, ArrowError>where
Self: Sized,
fn normalize_slicing(&self) -> Result<Self, ArrowError>where
Self: Sized,
Removes the offset / length of the struct array by pushing it into the children
In arrow-rs when slice is called it recursively slices the children. In arrow-cpp when slice is called it just sets the offset/length of the struct array and leaves the children as-is
Both are legal approaches (╥﹏╥)
This method helps reduce complexity by folding into the arrow-rs approach
Sourcefn pushdown_nulls(&self) -> Result<Self, ArrowError>where
Self: Sized,
fn pushdown_nulls(&self) -> Result<Self, ArrowError>where
Self: Sized,
Structs are allowed to mask valid items. For example, a struct array might be:
[ {“items”: [1, 2, 3]}, NULL, {“items”: NULL}]
However, the underlying items array might be: [[1, 2, 3], [4, 5], NULL]
The [4, 5] list is masked out because the struct array is null.
The struct validity would be [true, false, true] and the list validity would be [true, true, false]
This method pushes nulls down into all children. In the above example the list validity would become [true, false, false].
This method is not recursive. If a child is a struct array it will not push that child’s nulls down.
This method does not remove garbage lists. It only updates the validity so a future call to crate::list::ListArrayExt::filter_garbage_nulls will remove the garbage lists (without this pushdown it would not)