Struct deltalake::datafusion::datasource::physical_plan::CsvExec
source · pub struct CsvExec {
pub file_compression_type: FileCompressionType,
/* private fields */
}
Expand description
Execution plan for scanning a CSV file
Fields§
§file_compression_type: FileCompressionType
Compression type of the file associated with CsvExec
Implementations§
source§impl CsvExec
impl CsvExec
sourcepub fn new(
base_config: FileScanConfig,
has_header: bool,
delimiter: u8,
quote: u8,
escape: Option<u8>,
file_compression_type: FileCompressionType
) -> CsvExec
pub fn new( base_config: FileScanConfig, has_header: bool, delimiter: u8, quote: u8, escape: Option<u8>, file_compression_type: FileCompressionType ) -> CsvExec
Create a new CSV reader execution plan provided base and specific configurations
sourcepub fn base_config(&self) -> &FileScanConfig
pub fn base_config(&self) -> &FileScanConfig
Ref to the base configs
sourcepub fn has_header(&self) -> bool
pub fn has_header(&self) -> bool
true if the first line of each file is a header
Trait Implementations§
source§impl ExecutionPlan for CsvExec
impl ExecutionPlan for CsvExec
source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
Return a reference to Any that can be used for downcasting
source§fn repartitioned(
&self,
target_partitions: usize,
config: &ConfigOptions
) -> Result<Option<Arc<dyn ExecutionPlan>>, DataFusionError>
fn repartitioned( &self, target_partitions: usize, config: &ConfigOptions ) -> Result<Option<Arc<dyn ExecutionPlan>>, DataFusionError>
Redistribute files across partitions according to their size
See comments on FileGroupPartitioner
for more detail.
Return None
if can’t get repartitioned(empty/compressed file).
source§fn properties(&self) -> &PlanProperties
fn properties(&self) -> &PlanProperties
Return properties of the output of the
ExecutionPlan
, such as output
ordering(s), partitioning information etc. Read moresource§fn children(&self) -> Vec<Arc<dyn ExecutionPlan>>
fn children(&self) -> Vec<Arc<dyn ExecutionPlan>>
Get a list of children
ExecutionPlan
s that act as inputs to this plan.
The returned list will be empty for leaf nodes such as scans, will contain
a single value for unary nodes, or two values for binary nodes (such as
joins).source§fn with_new_children(
self: Arc<CsvExec>,
_: Vec<Arc<dyn ExecutionPlan>>
) -> Result<Arc<dyn ExecutionPlan>, DataFusionError>
fn with_new_children( self: Arc<CsvExec>, _: Vec<Arc<dyn ExecutionPlan>> ) -> Result<Arc<dyn ExecutionPlan>, DataFusionError>
Returns a new
ExecutionPlan
where all existing children were replaced
by the children
, in ordersource§fn execute(
&self,
partition: usize,
context: Arc<TaskContext>
) -> Result<Pin<Box<dyn RecordBatchStream<Item = Result<RecordBatch, DataFusionError>> + Send>>, DataFusionError>
fn execute( &self, partition: usize, context: Arc<TaskContext> ) -> Result<Pin<Box<dyn RecordBatchStream<Item = Result<RecordBatch, DataFusionError>> + Send>>, DataFusionError>
source§fn statistics(&self) -> Result<Statistics, DataFusionError>
fn statistics(&self) -> Result<Statistics, DataFusionError>
Returns statistics for this
ExecutionPlan
node. If statistics are not
available, should return Statistics::new_unknown
(the default), not
an error.source§fn metrics(&self) -> Option<MetricsSet>
fn metrics(&self) -> Option<MetricsSet>
Return a snapshot of the set of
Metric
s for this
ExecutionPlan
. If no Metric
s are available, return None. Read moresource§fn required_input_distribution(&self) -> Vec<Distribution>
fn required_input_distribution(&self) -> Vec<Distribution>
Specifies the data distribution requirements for all the
children for this
ExecutionPlan
, By default it’s [Distribution::UnspecifiedDistribution] for each child,source§fn required_input_ordering(&self) -> Vec<Option<Vec<PhysicalSortRequirement>>>
fn required_input_ordering(&self) -> Vec<Option<Vec<PhysicalSortRequirement>>>
Specifies the ordering required for all of the children of this
ExecutionPlan
. Read moreAuto Trait Implementations§
impl Freeze for CsvExec
impl !RefUnwindSafe for CsvExec
impl Send for CsvExec
impl Sync for CsvExec
impl Unpin for CsvExec
impl !UnwindSafe for CsvExec
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreCreates a shared type from an unshared type.