Skip to main content

column_schema_of

Function column_schema_of 

Source
pub fn column_schema_of(
    path: &Path,
    format: FileFormat,
    as_read: &ReadAs,
) -> Option<Vec<(String, DataType)>>
Expand description

The column names one data file holds, read as cheaply as its format allows.

The evidence is_nested wants, for the formats that have no footer. Parquet’s answer comes from its footer and has its own path; this is for the rest, where the names are at the front of the file — a CSV header line, an NDJSON object’s keys — and Polars’ own schema inference is what reads them, so the names are the ones the open will use, separator and all.

None for a format whose schema cannot be had without reading the whole file, and for a file that would not parse. Both mean “no evidence”, which every caller here treats as it treats an unreadable footer: the directory keeps the kind its names suggested, and the read that follows is lenient enough to survive being wrong.

Deliberately not JSON: a .json file is one document, and its keys are only known once it has been parsed. Sampling three of those in a listing pass is a read of three whole files for a label.