pub fn csv_fields(paths: &[String], given: Given) -> Result<Vec<Field>>Expand description
The columns a read_csv of paths produces, sniffed out of the front of every one of them.
Every file and not only the first, which is the one place this differs from Parquet and is
DuckDB’s rule rather than a choice made here. It was measured at two, three, four and six files:
four files where only the fourth holds a decimal answer DOUBLE, and six where only the sixth
holds text answer VARCHAR. A Parquet file states its schema in its footer, so there is a first
file’s word to take. A CSV file states nothing, so there is not, and a directory of daily exports
where one day happens to hold whole numbers in an otherwise decimal column would come out BIGINT
or DOUBLE depending on which day sorted first. So all of them are sniffed and the answers are
combined by rudb_csv::across.
That is an open and one sample read per file at bind time. It is what the binary does, it is the only way the answer can be right, and it is a sample against a scan that is about to read all of those files anyway.
§Errors
Everything open_csv reports, and a file that is missing a column the first one has.