Struct deltalake::datafusion::datasource::file_format::arrow::ArrowFormat

pub struct ArrowFormat;

Arrow FileFormat implementation.

Trait Implementations

impl Debug for ArrowFormat

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for ArrowFormat

fn default() -> ArrowFormat

Returns the “default value” for a type.

impl FileFormat for ArrowFormat

fn as_any(&self) -> &(dyn Any + 'static)

Returns the table provider as Any so that it can be downcast to a specific implementation.

fn infer_schema<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, _state: &'life1 SessionState, store: &'life2 Arc<dyn ObjectStore>, objects: &'life3 [ObjectMeta] ) -> Pin<Box<dyn Future<Output = Result<Arc<Schema>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, ArrowFormat: 'async_trait,

Infer the common schema of the provided objects. The objects will usually be analysed up to a given number of records or files (as specified in the format config) then give the estimated common schema. This might fail if the files have schemas that cannot be merged.

fn infer_stats<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, _state: &'life1 SessionState, _store: &'life2 Arc<dyn ObjectStore>, table_schema: Arc<Schema>, _object: &'life3 ObjectMeta ) -> Pin<Box<dyn Future<Output = Result<Statistics, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, ArrowFormat: 'async_trait,

Infer the statistics for the provided object. The cost and accuracy of the estimated statistics might vary greatly between file formats.

fn create_physical_plan<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _state: &'life1 SessionState, conf: FileScanConfig, _filters: Option<&'life2 Arc<dyn PhysicalExpr>> ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn ExecutionPlan>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, ArrowFormat: 'async_trait,

Take a list of files and convert it to the appropriate executor according to this file format.

fn create_writer_physical_plan<'life0, 'life1, 'async_trait>( &'life0 self, input: Arc<dyn ExecutionPlan>, _state: &'life1 SessionState, conf: FileSinkConfig, order_requirements: Option<Vec<PhysicalSortRequirement>> ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn ExecutionPlan>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, ArrowFormat: 'async_trait,

Take a list of files and the configuration to convert it to the appropriate writer executor according to this file format.

fn file_type(&self) -> FileType

Returns the FileType corresponding to this FileFormat