Struct SkippedFiles 

pub struct SkippedFiles {
    pub bookkeeping: usize,
    pub not_parquet: usize,
    pub empty: usize,
}

What a dataset’s listing walked past: files under the directory that are not read.

Split by whether anyone could have meant them as data, which is a question about where a file is rather than what it is called.

A file beside the data — a .csv in a directory that also holds Parquet — is one somebody may have expected in the table. That is worth saying.

A file somewhere else is infrastructure. Delta keeps its log in _delta_log/, Hudi in .hoodie/, Iceberg in a plain metadata/ beside the data; a bucket made through a console is full of zero-byte folder markers. Naming those conventions one by one is a game with no end — the test that holds for all of them is that a directory with no Parquet in it is nobody’s table, whatever it is called.

Not a complete accounting of the directory: a subtree that cannot be read, or one below the depth the walk stops at, is neither listed nor counted. What the note says is how many files were passed over among those it saw.

Fields

bookkeeping: usize

Files a writer leaves beside the data: a name beginning _ or ..

not_parquet: usize

Everything else that is not a Parquet file.

empty: usize

Objects with nothing in them, whose name says they are data. Not a file anyone left on purpose: a write that stopped, which is the skip most worth saying.

A store’s listing knows each size; a directory knows it from the stat past one wave, or from the footer that would not read within one.

Implementations

impl SkippedFiles

pub fn count(&mut self, bookkeeping: bool)

Count one file the listing passed over.

bookkeeping is the caller’s, because only the caller knows where the file was. A .json is a mistake beside the data and a record of the table inside _delta_log/ or metadata/, and its own name cannot say which.

Trait Implementations

impl Clone for SkippedFiles

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SkippedFiles

impl Debug for SkippedFiles

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SkippedFiles

fn default() -> Self

Returns the “default value” for a type.

impl Eq for SkippedFiles

impl PartialEq for SkippedFiles

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for SkippedFiles