rudb_csv/lib.rs
1//! The CSV reader and writer, including dialect sniffing and type inference.
2//!
3//! Rank 5 in the layer rule. See `xtask/layers.toml` and `spec/18-package-layout.md`.
4//!
5//! A CSV file says nothing about itself. The delimiter, the quote, whether the first line names the
6//! columns and what type each column holds are all conventions, and a reader that has to be told
7//! them is a reader every loader script in the world has to be rewritten for. So this sniffs, the
8//! way DuckDB does, and every rule in it was read off duckdb v1.4.1 rather than reasoned about.
9//! Where the two differ, the difference is written down next to the code, because a sniffer that
10//! quietly disagrees produces a column of the wrong type and that is a wrong answer rather than a
11//! slow query.
12//!
13//! Three pieces. [`scan`] turns bytes into fields, [`dialect`] works out the punctuation by running
14//! the scanner under each candidate and seeing which one is consistent, and [`infer`] walks a ladder
15//! of types to find the first one every value in a column fits. [`Reader`] is the three of them over
16//! a file, handing back chunks.
17//!
18//! There is no writer yet. `COPY t TO 'out.csv'` is the statement that wants one and it is not
19//! bound, so a writer here would be a writer nothing calls.
20
21#![forbid(unsafe_code)]
22
23pub mod dialect;
24pub mod infer;
25pub mod reader;
26pub mod scan;
27
28pub use dialect::Dialect;
29pub use reader::Reader;
30
31/// The crate this rank belongs to, so that the layer check has something to read.
32pub const RANK: u8 = 5;