Struct datafusion::datasource::file_format::options::CsvReadOptions
source · pub struct CsvReadOptions<'a> {
pub has_header: bool,
pub delimiter: u8,
pub schema: Option<&'a Schema>,
pub schema_infer_max_records: usize,
pub file_extension: &'a str,
pub table_partition_cols: Vec<(String, DataType)>,
pub file_compression_type: FileCompressionType,
pub infinite: bool,
}
Expand description
Options that control the reading of CSV files.
Note this structure is supplied when a datasource is created and
can not not vary from statement to statement. For settings that
can vary statement to statement see
ConfigOptions
.
Fields§
§has_header: bool
Does the CSV file have a header?
If schema inference is run on a file with no headers, default column names are created.
delimiter: u8
An optional column delimiter. Defaults to b','
.
schema: Option<&'a Schema>
An optional schema representing the CSV files. If None, CSV reader will try to infer it based on data in file.
schema_infer_max_records: usize
Max number of rows to read from CSV files for schema inference if needed. Defaults to DEFAULT_SCHEMA_INFER_MAX_RECORD
.
file_extension: &'a str
File extension; only files with this extension are selected for data input.
Defaults to FileType::CSV.get_ext().as_str()
.
table_partition_cols: Vec<(String, DataType)>
Partition Columns
file_compression_type: FileCompressionType
File compression type
infinite: bool
Flag indicating whether this file may be unbounded (as in a FIFO file).
Implementations§
source§impl<'a> CsvReadOptions<'a>
impl<'a> CsvReadOptions<'a>
sourcepub fn has_header(self, has_header: bool) -> Self
pub fn has_header(self, has_header: bool) -> Self
Configure has_header setting
sourcepub fn mark_infinite(self, infinite: bool) -> Self
pub fn mark_infinite(self, infinite: bool) -> Self
Configure mark_infinite setting
sourcepub fn file_extension(self, file_extension: &'a str) -> Self
pub fn file_extension(self, file_extension: &'a str) -> Self
Specify the file extension for CSV file selection
sourcepub fn delimiter_option(self, delimiter: Option<u8>) -> Self
pub fn delimiter_option(self, delimiter: Option<u8>) -> Self
Configure delimiter setting with Option, None value will be ignored
sourcepub fn table_partition_cols(
self,
table_partition_cols: Vec<(String, DataType)>
) -> Self
pub fn table_partition_cols( self, table_partition_cols: Vec<(String, DataType)> ) -> Self
Specify table_partition_cols for partition pruning
sourcepub fn schema_infer_max_records(self, max_records: usize) -> Self
pub fn schema_infer_max_records(self, max_records: usize) -> Self
Configure number of max records to read for schema inference
sourcepub fn file_compression_type(
self,
file_compression_type: FileCompressionType
) -> Self
pub fn file_compression_type( self, file_compression_type: FileCompressionType ) -> Self
Configure file compression type
Trait Implementations§
source§impl<'a> Clone for CsvReadOptions<'a>
impl<'a> Clone for CsvReadOptions<'a>
source§fn clone(&self) -> CsvReadOptions<'a>
fn clone(&self) -> CsvReadOptions<'a>
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moresource§impl<'a> Default for CsvReadOptions<'a>
impl<'a> Default for CsvReadOptions<'a>
source§impl ReadOptions<'_> for CsvReadOptions<'_>
impl ReadOptions<'_> for CsvReadOptions<'_>
source§fn to_listing_options(&self, config: &SessionConfig) -> ListingOptions
fn to_listing_options(&self, config: &SessionConfig) -> ListingOptions
ListingTable
options