Expand description
The readers: what datui does with a file of each format.
A format’s descriptor (crate::FileFormat::descriptor, in datui-cli) says what is
true of it without a file. Its Reader holds the code: signature bytes, the scan,
conversion to files of its own, listed tables, the home preview, Copy as Python’s
call, the default export format, and Info tab facts (from the scan, or Reader::facts
reading a footer). Readers live beside their parsers (crate::formats::sqlite::READER); those
Polars reads are in polars. of maps every format exhaustively, so a missing
reader does not compile. Adding a format: variant and descriptor in datui-cli, parser
and reader in a module, a line in of.
Outside those, a format is named only where it changes app behavior:
- Parquet: hive partitions and footers (directories and globs as one scan, part files by directory name, footer row counts, object-store directories read as Parquet only).
- Arrow IPC: streams are converted to a file first (
Scan::Streams,Conversion::Streams); Hugging Face caches and DatasetDicts, a split at a time. - JSON: Hugging Face metadata and model configs are not data left out.
- SafeTensors and GGUF: a directory of weights is the model; remote models read by
headers (
crate::cloud::remote_model). - CSV: the reader delimited specs read through.
- Text, and unnamed CSV, TSV, JSON and NDJSON: told apart by
crate::formats::lines::guess, notsniff; a text name still has its bytes asked (FileFormat::TEXT); followed lines are counted by the watcher. - Audio: a full quality run checks the signal (
crate::formats::audio::recording).
Modules§
- hive
- Hive-partitioned Parquet on disk: a directory of
key=valuedirectories read as one scan, with the keys as columns.
Structs§
- Format
Facts - What a
FactsFnread: the format’s tab of the Info panel, and the footer the Schema tab’s Compression column is drawn from. - Typing
- The columns a read gave a type, the frame before it did, and its notes.
Constants§
- HEAD
- Bytes read from the start of a file to tell its format by.