Struct deltalake::datafusion::datasource::listing::PartitionedFile

pub struct PartitionedFile {
    pub object_meta: ObjectMeta,
    pub partition_values: Vec<ScalarValue>,
    pub range: Option<FileRange>,
    pub extensions: Option<Arc<dyn Any + Sync + Send>>,
}

A single file or part of a file that should be read, along with its schema, statistics and partition column values that need to be appended to each row.

Fields

object_meta: ObjectMeta

Path for the file (e.g. URL, filesystem path, etc)

partition_values: Vec<ScalarValue>

Values of partition columns to be appended to each row.

These MUST have the same count, order, and type than the table_partition_cols.

You may use wrap_partition_value_in_dict to wrap them if you have used wrap_partition_type_in_dict to wrap the column type.

range: Option<FileRange>

An optional file range for a more fine-grained parallel execution

extensions: Option<Arc<dyn Any + Sync + Send>>

An optional field for user defined per object metadata

Implementations

impl PartitionedFile

pub fn new(path: impl Into<String>, size: u64) -> PartitionedFile

Create a simple file without metadata or partition

pub fn new_with_range( path: String, size: u64, start: i64, end: i64 ) -> PartitionedFile

Create a file range without metadata or partition

pub fn from_path(path: String) -> Result<PartitionedFile, DataFusionError>

Return a file reference from the given path

pub fn path(&self) -> &Path

Return the path of this partitioned file

pub fn with_range(self, start: i64, end: i64) -> PartitionedFile

Update the file to only scan the specified range (in bytes)

Trait Implementations

impl Clone for PartitionedFile

fn clone(&self) -> PartitionedFile

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PartitionedFile

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<ObjectMeta> for PartitionedFile

fn from(object_meta: ObjectMeta) -> PartitionedFile

Converts to this type from the input type.

impl TryFrom<&PartitionedFile> for PartitionedFile

type Error = DataFusionError

The type returned in the event of a conversion error.

fn try_from( val: &PartitionedFile ) -> Result<PartitionedFile, <PartitionedFile as TryFrom<&PartitionedFile>>::Error>

Performs the conversion.