Skip to main content

fix_nested_null_masks

Function fix_nested_null_masks 

Source
pub fn fix_nested_null_masks(batch: StructArray) -> StructArray
Available on crate feature arrow-expression and crate feature internal-api and (crate features arrow-conversion or declarative-plans or default-engine-base) only.
Expand description

Use this function to recursively compute properly unioned null masks for all nested columns of a record batch, making it safe to project out and consume nested columns.

Arrow does not guarantee that the null masks associated with nested columns are accurate – instead, the reader must consult the union of logical null masks the column and all ancestors. The parquet reader stopped doing this automatically as of arrow-53.3, for example.