pub fn directory_format(dir: &Path) -> DirectoryFormatExpand description
What DirectoryFormat the data files directly in dir amount to.
One level only, and no file is opened: this reads names, exactly as the rest of
this module does. A directory of two hundred thousand files costs one listing
and nothing per file beyond it — the entry’s own type comes back with the name, so
there is no stat to bound.
Deliberately not bounded by MAX_ENTRIES_PER_DIR, unlike every listing in this
module. The files this returns are not a menu to show, they are the table to read:
stopping at five thousand of a directory’s six thousand CSVs would open it with a row
count, a schema union and every aggregate quietly computed over a subset, and the
take running before the sort would drop whichever file the directory read
happened to return last. The Parquet route this mirrors hands the directory to a scan
that enumerates it, with no cap either.