Struct VortexSource 

pub struct VortexSource { /* private fields */ }

Execution plan for reading one or more Vortex files, intended to be consumed by DataSourceExec.

Implementations

impl VortexSource

pub fn with_expr_adapter_factory( &self, expr_adapter_factory: Arc<dyn PhysicalExprAdapterFactory>, ) -> Arc<dyn FileSource>

Sets a PhysicalExprAdapterFactory for the VortexSource. Currently, this must be provided in order to filter columns in files that have a different data type from the unified table schema.

This factory will take precedence when opening files over instances provided by the FileScanConfig.

Trait Implementations

impl Clone for VortexSource

fn clone(&self) -> VortexSource

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl FileSource for VortexSource

fn create_file_opener( &self, object_store: Arc<dyn ObjectStore>, base_config: &FileScanConfig, partition: usize, ) -> Arc<dyn FileOpener>

Creates a dyn FileOpener based on given parameters

fn as_any(&self) -> &dyn Any

Any

fn with_batch_size(&self, batch_size: usize) -> Arc<dyn FileSource>

Initialize new type with batch size configuration

fn with_schema(&self, schema: SchemaRef) -> Arc<dyn FileSource>

Initialize new instance with a new schema

fn with_projection(&self, _config: &FileScanConfig) -> Arc<dyn FileSource>

Initialize new instance with projection information

fn with_statistics(&self, statistics: Statistics) -> Arc<dyn FileSource>

Initialize new instance with projected statistics

fn metrics(&self) -> &ExecutionPlanMetricsSet

Return execution plan metrics

fn statistics(&self) -> DFResult<Statistics>

Return projected statistics

fn file_type(&self) -> &str

String representation of file source such as “csv”, “json”, “parquet”

fn fmt_extra(&self, t: DisplayFormatType, f: &mut Formatter<'_>) -> Result

Format FileType specific information

fn try_pushdown_filters( &self, filters: Vec<Arc<dyn PhysicalExpr>>, _config: &ConfigOptions, ) -> DFResult<FilterPushdownPropagation<Arc<dyn FileSource>>>

Try to push down filters into this FileSource. See ExecutionPlan::handle_child_pushdown_result for more details.

fn with_schema_adapter_factory( &self, factory: Arc<dyn SchemaAdapterFactory>, ) -> DFResult<Arc<dyn FileSource>>

Set optional schema adapter factory.

fn schema_adapter_factory(&self) -> Option<Arc<dyn SchemaAdapterFactory>>

Returns the current schema adapter factory if set

fn repartitioned( &self, target_partitions: usize, repartition_file_min_size: usize, output_ordering: Option<LexOrdering>, config: &FileScanConfig, ) -> Result<Option<FileScanConfig>, DataFusionError>

If supported by the FileSource, redistribute files across partitions according to their size. Allows custom file formats to implement their own repartitioning logic.