Struct datafusion::datasource::file_format::options::AvroReadOptions
source · pub struct AvroReadOptions<'a> {
pub schema: Option<&'a Schema>,
pub file_extension: &'a str,
pub table_partition_cols: Vec<(String, DataType)>,
pub infinite: bool,
}
Expand description
Options that control the reading of AVRO files.
Note this structure is supplied when a datasource is created and
can not not vary from statement to statement. For settings that
can vary statement to statement see
ConfigOptions
.
Fields§
§schema: Option<&'a Schema>
The data source schema.
file_extension: &'a str
File extension; only files with this extension are selected for data input.
Defaults to FileType::AVRO.get_ext().as_str()
.
table_partition_cols: Vec<(String, DataType)>
Partition Columns
infinite: bool
Flag indicating whether this file may be unbounded (as in a FIFO file).
Implementations§
source§impl<'a> AvroReadOptions<'a>
impl<'a> AvroReadOptions<'a>
sourcepub fn table_partition_cols(
self,
table_partition_cols: Vec<(String, DataType)>
) -> Self
pub fn table_partition_cols( self, table_partition_cols: Vec<(String, DataType)> ) -> Self
Specify table_partition_cols for partition pruning
sourcepub fn mark_infinite(self, infinite: bool) -> Self
pub fn mark_infinite(self, infinite: bool) -> Self
Configure mark_infinite setting
Trait Implementations§
source§impl<'a> Clone for AvroReadOptions<'a>
impl<'a> Clone for AvroReadOptions<'a>
source§fn clone(&self) -> AvroReadOptions<'a>
fn clone(&self) -> AvroReadOptions<'a>
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl<'a> Default for AvroReadOptions<'a>
impl<'a> Default for AvroReadOptions<'a>
source§impl ReadOptions<'_> for AvroReadOptions<'_>
impl ReadOptions<'_> for AvroReadOptions<'_>
source§fn to_listing_options(&self, config: &SessionConfig) -> ListingOptions
fn to_listing_options(&self, config: &SessionConfig) -> ListingOptions
Helper to convert these user facing options to
ListingTable
optionssource§fn get_resolved_schema<'life0, 'life1, 'async_trait>(
&'life0 self,
config: &'life1 SessionConfig,
state: SessionState,
table_path: ListingTableUrl
) -> Pin<Box<dyn Future<Output = Result<SchemaRef>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn get_resolved_schema<'life0, 'life1, 'async_trait>( &'life0 self, config: &'life1 SessionConfig, state: SessionState, table_path: ListingTableUrl ) -> Pin<Box<dyn Future<Output = Result<SchemaRef>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,
Infer and resolve the schema from the files/sources provided.
source§fn _get_resolved_schema<'life0, 'async_trait>(
&'a self,
config: &'life0 SessionConfig,
state: SessionState,
table_path: ListingTableUrl,
schema: Option<&'a Schema>,
infinite: bool
) -> Pin<Box<dyn Future<Output = Result<SchemaRef>> + Send + 'async_trait>>where
Self: Sync + 'async_trait,
'a: 'async_trait,
'life0: 'async_trait,
fn _get_resolved_schema<'life0, 'async_trait>( &'a self, config: &'life0 SessionConfig, state: SessionState, table_path: ListingTableUrl, schema: Option<&'a Schema>, infinite: bool ) -> Pin<Box<dyn Future<Output = Result<SchemaRef>> + Send + 'async_trait>>where Self: Sync + 'async_trait, 'a: 'async_trait, 'life0: 'async_trait,
helper function to reduce repetitive code. Infers the schema from sources if not provided. Infinite data sources not supported through this function.
Auto Trait Implementations§
impl<'a> RefUnwindSafe for AvroReadOptions<'a>
impl<'a> Send for AvroReadOptions<'a>
impl<'a> Sync for AvroReadOptions<'a>
impl<'a> Unpin for AvroReadOptions<'a>
impl<'a> UnwindSafe for AvroReadOptions<'a>
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more