Skip to main content

top_level_columns

Function top_level_columns 

Source
pub fn top_level_columns(leaves: &[String]) -> Vec<String>
Expand description

The top-level column names in a list of Parquet leaf paths.

A footer names every leaf, so a struct or a list arrives as inputs.list.element. address and its wrappers are an encoding choice: the same column written by parquet-mr and by Arrow gives different leaves. Comparing those would make a dataset whose writer changed look like two tables, so the comparison is over the columns a reader sees. Order is kept and duplicates dropped, since many leaves share one root.