Struct VortexDataSource 

pub struct VortexDataSource { /* private fields */ }

DataFusion DataSource backed by a Vortex DataSourceRef.

VortexDataSource is the core execution adapter for the v2 integration. It presents DataFusion with a scanable Arrow data source while preserving the underlying Vortex source until execution time.

During planning, it reports the current output schema and column statistics. During execution, it builds the final Vortex ScanRequest from the current projection, pushed filters, ordering hints, and row limit.

This integration intentionally reports a single DataFusion output partition. Vortex then handles split-level concurrency internally by polling multiple split streams concurrently.

Use crate::VortexSource instead when DataFusion should discover and plan .vortex files on its own.

Implementations

impl VortexDataSource

pub fn builder( data_source: DataSourceRef, session: VortexSession, ) -> VortexDataSourceBuilder

Create a builder for a VortexDataSource.

Trait Implementations

impl Clone for VortexDataSource

fn clone(&self) -> VortexDataSource

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl DataSource for VortexDataSource

fn open( &self, partition: usize, _context: Arc<TaskContext>, ) -> DFResult<SendableRecordBatchStream>

fn as_any(&self) -> &dyn Any

fn fmt_as(&self, _t: DisplayFormatType, f: &mut Formatter<'_>) -> Result

Format this source for display in explain plans

fn repartitioned( &self, target_partitions: usize, _repartition_file_min_size: usize, output_ordering: Option<LexOrdering>, ) -> DFResult<Option<Arc<dyn DataSource>>>

Return a copy of this DataSource with a new partitioning scheme.

fn output_partitioning(&self) -> Partitioning

fn eq_properties(&self) -> EquivalenceProperties

fn partition_statistics( &self, _partition: Option<usize>, ) -> DFResult<Statistics>

Returns statistics for a specific partition, or aggregate statistics across all partitions if partition is None.

fn with_fetch(&self, limit: Option<usize>) -> Option<Arc<dyn DataSource>>

Return a copy of this DataSource with a new fetch limit

fn fetch(&self) -> Option<usize>

fn try_swapping_with_projection( &self, projection: &ProjectionExprs, ) -> DFResult<Option<Arc<dyn DataSource>>>

fn try_pushdown_filters( &self, filters: Vec<Arc<dyn PhysicalExpr>>, _config: &ConfigOptions, ) -> DFResult<FilterPushdownPropagation<Arc<dyn DataSource>>>

Try to push down filters into this DataSource.

fn scheduling_type(&self) -> SchedulingType

fn metrics(&self) -> ExecutionPlanMetricsSet

fn try_pushdown_sort( &self, _order: &[PhysicalSortExpr], ) -> Result<SortOrderPushdownResult<Arc<dyn DataSource>>, DataFusionError>

Try to create a new DataSource that produces data in the specified sort order.

fn with_preserve_order( &self, _preserve_order: bool, ) -> Option<Arc<dyn DataSource>>

Returns a variant of this DataSource that is aware of order-sensitivity.

impl Debug for VortexDataSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.