Skip to main content

Module readers

Module readers 

Source
Expand description

The readers: what datui does with a file of each format.

A format’s descriptor (crate::FileFormat::descriptor, in datui-cli) says what is true of it without a file. Its Reader holds the code: signature bytes, the scan, conversion to files of its own, listed tables, the home preview, Copy as Python’s call, the default export format, and Info tab facts (from the scan, or Reader::facts reading a footer). Readers live beside their parsers (crate::formats::sqlite::READER); those Polars reads are in polars. of maps every format exhaustively, so a missing reader does not compile. Adding a format: variant and descriptor in datui-cli, parser and reader in a module, a line in of.

Outside those, a format is named only where it changes app behavior:

  • Parquet: hive partitions and footers (directories and globs as one scan, part files by directory name, footer row counts, object-store directories read as Parquet only).
  • Arrow IPC: streams are converted to a file first (Scan::Streams, Conversion::Streams); Hugging Face caches and DatasetDicts, a split at a time.
  • JSON: Hugging Face metadata and model configs are not data left out.
  • SafeTensors and GGUF: a directory of weights is the model; remote models read by headers (crate::cloud::remote_model).
  • CSV: the reader delimited specs read through.
  • Text, and unnamed CSV, TSV, JSON and NDJSON: told apart by crate::formats::lines::guess, not sniff; a text name still has its bytes asked (FileFormat::TEXT); followed lines are counted by the watcher.
  • Audio: a full quality run checks the signal (crate::formats::audio::recording).

Modules§

hive
Hive-partitioned Parquet on disk: a directory of key=value directories read as one scan, with the keys as columns.

Structs§

FormatFacts
What a FactsFn read: the format’s tab of the Info panel, and the footer the Schema tab’s Compression column is drawn from.
Typing
The columns a read gave a type, the frame before it did, and its notes.

Constants§

HEAD
Bytes read from the start of a file to tell its format by.