Struct CommonFileOptions

pub struct CommonFileOptions {
    pub path_glob_filter: Option<String>,
    pub recursive_file_lookup: Option<bool>,
    pub modified_before: Option<String>,
    pub modified_after: Option<String>,
    pub ignore_corrupt_files: Option<bool>,
    pub ignore_missing_files: Option<bool>,
}

Common file options that are shared across multiple file formats (e.g., CSV, JSON, ORC, Parquet, Text).

These options allow for file filtering and error handling during file discovery and processing.

Fields

• path_glob_filter: Optional glob pattern to filter files by path. This can be used to select specific files within a directory.

• recursive_file_lookup: Whether to recursively search for files in subdirectories. If set to true, the reader will search all subdirectories for matching files.

• modified_before: An optional string specifying a cutoff date for filtering files. Files modified after this date will not be included.

• modified_after: An optional string specifying a start date for filtering files. Only files modified after this date will be included.

• ignore_corrupt_files: If set to true, the reader will skip corrupt files instead of throwing an error. This is useful in scenarios where some files may be malformed.

• ignore_missing_files: If set to true, missing files (e.g., those filtered out by the glob pattern) will be ignored instead of causing an error. This is useful when running in environments where files may be missing intermittently.

Fields

path_glob_filter: Option<String>

recursive_file_lookup: Option<bool>

modified_before: Option<String>

modified_after: Option<String>

ignore_corrupt_files: Option<bool>

ignore_missing_files: Option<bool>

Trait Implementations

impl Clone for CommonFileOptions

fn clone(&self) -> CommonFileOptions

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ConfigOpts for CommonFileOptions

fn to_options(&self) -> HashMap<String, String>

impl Debug for CommonFileOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CommonFileOptions

fn default() -> Self

Returns the “default value” for a type.