Enum FileFormat

Source

pub enum FileFormat {
Show 27 variants    Parquet,
    Csv,
    Tsv,
    Psv,
    Json,
    Jsonl,
    Arrow,
    Avro,
    Orc,
    Excel,
    Safetensors,
    Gguf,
    Nmea,
    Gpx,
    Audio,
    Midi,
    Sqlite,
    Vcd,
    Fix,
    Sdf,
    Numpy,
    Elf,
    Ulog,
    Dataflash,
    Candump,
    Text,
    Journal,
}

Expand description

A format datui reads, as --format names it.

Variants§

§

Journal

Implementations§

Source §

impl FileFormat

Source

pub const ALL: [Self; 27]

Every format, for the places that have to consider all of them, in the order --format’s help lists them.

Written out, and so able to fall behind the enum. every_format_is_listed matches a variant exhaustively, so adding one stops that test compiling. What a format missing here would cost is bounded: from_name answers None for it, and every caller reads None as “not Parquet”, which leaves counts off a directory rather than giving it another format’s.

Source

pub const TEXT: Self = Self::Text

What text with nothing else to say is read as: a pipe, a followed file, a compressed file whose format is not named, when nothing in it says more.

Source

pub const fn descriptor(self) -> &'static Descriptor

The format’s descriptor.

Source

pub fn from_path(path: &Path) -> Option<Self>

Detect file format from path extension. Returns None when extension is missing or unknown.

A name ending a descriptor lists (model.safetensors.index.json) says its format before the extension does.

Source

pub fn from_name_ending(path: &Path) -> Option<Self>

The format a whole-name ending says (model.safetensors.index.json), before the extension does.

Source

pub fn name(self) -> &'static str

The format’s name, as a row on the home screen says it: 12 parquet, 3 csv.

The dataset cache stores it (Holds), so renaming one makes the records already on disk unreadable.

Source

pub fn title(self) -> &'static str

What the format is called in a sentence.

Source

pub fn from_name(name: &str) -> Option<Self>

The format a FileFormat::name names, for a name that was stored rather than carried. The inverse of that method, and the only way back: a name is not an extension, so from_extension cannot read one.

Source

pub fn reads_many_files(self) -> bool

Whether many files of this format can be read as one table.

Asked before a directory is offered as a dataset, so the home screen cannot promise an open the reader has no route for.

Source

pub fn holds_tables(self) -> bool

Whether a file of this format can hold several tables, each with a path inside it (shop.db/orders, run.npz/weights) that the home screen lists like a directory’s files and --table names.

Source

pub fn opens_one_table(self) -> bool

Whether a file of this format holding several tables opens one when none is named. See Tables::opens.

Source

pub fn summary_tab(self) -> Option<&'static str>

The name of the format’s tab of the Info panel, when it has one.

Source

pub fn takes_table(self) -> bool

Whether --table picks one of the tables of a file of this format.

Source

pub fn reads_into(self) -> bool

Whether a file of this format is read into files of its own before it is scanned: a GPS log, VCD dump, FIX log or SDF file.

Source

pub fn read_mode(self, stored: Stored) -> Option<ReadMode>

How a file of this format is read when it is opened, as stored on disk.

The one answer to “does this read only what it shows”: the format table in docs/formats/index.md, the home screen’s marker and the Info panel’s Read: line all ask here. None when a file stored that way does not open: a compressed Parquet file, or an IPC stream of anything but Arrow.

Source

pub fn bucket_object(self, stored: Stored) -> RemoteRead

How one object of this format in a bucket (S3, GCS, Azure), as stored, is read: in place with ranged reads, or downloaded first and then read as Self::read_mode says. Only a plain file is read in place.

Source

pub fn http_file(self) -> RemoteRead

How one HTTP(S) file of this format is read. A server that sends no ranges gets the download question.

Source

pub fn bucket_prefix(self, stored: Stored) -> Option<RemoteRead>

How a bucket prefix or glob of files of this format, as stored, is read as one table: in place, downloaded first, or None when it is not (browsed into and opened an object at a time). A prefix of streams is converted as it downloads.

Source

pub fn reads_bucket_prefix(self) -> bool

Whether a bucket prefix or glob of this format is scanned in place as one table. See Self::bucket_prefix.

Source

pub fn separator(self) -> Option<u8>

The column separator a delimited format is read with when --delimiter is not given. None for the formats that are not delimited text.

Source

pub fn follows(self) -> bool

Whether --follow reads a file of this format as it grows: text read a line at a time.

Source

pub fn is_lines(self) -> bool

Whether a file of this format is read a line a row, as it stands.

Source

pub fn decompressed_once(self) -> bool

Whether a compressed file of this format is decompressed once, to a file or into memory, before it is read: delimited text and lines.

Source

pub fn conversion(self) -> Conversion

What the open says while a file of this format is read into files of its own. A format with nothing of its own to say says it converts.

Source

pub fn from_extension(ext: &str) -> Option<Self>

Parse format from extension string (e.g. “parquet”, “csv”).

The one place an extension becomes a format. Everything that asks whether a name is data — the home screen, the search, ~ path input, the CLI and the cloud listings — asks here, so no route can offer a file another route cannot open.

.txt and .log say text, read a line a row. A directory ranks text below every other format, so a README beside Parquet files does not decide what the directory is. A tabular .txt opens with --format csv.