Struct datafusion::datasource::listing::PartitionedFile

pub struct PartitionedFile {
    pub object_meta: ObjectMeta,
    pub partition_values: Vec<ScalarValue>,
    pub range: Option<FileRange>,
    pub statistics: Option<Statistics>,
    pub extensions: Option<Arc<dyn Any + Send + Sync>>,
}

A single file or part of a file that should be read, along with its schema, statistics and partition column values that need to be appended to each row.

Fields

object_meta: ObjectMeta

Path for the file (e.g. URL, filesystem path, etc)

partition_values: Vec<ScalarValue>

Values of partition columns to be appended to each row.

These MUST have the same count, order, and type than the table_partition_cols.

You may use wrap_partition_value_in_dict to wrap them if you have used wrap_partition_type_in_dict to wrap the column type.

range: Option<FileRange>

An optional file range for a more fine-grained parallel execution

statistics: Option<Statistics>

Optional statistics that describe the data in this file if known.

DataFusion relies on these statistics for planning (in particular to sort file groups), so if they are incorrect, incorrect answers may result.

extensions: Option<Arc<dyn Any + Send + Sync>>

An optional field for user defined per object metadata

Implementations

impl PartitionedFile

pub fn new(path: impl Into<String>, size: u64) -> Self

Create a simple file without metadata or partition

pub fn new_with_range(path: String, size: u64, start: i64, end: i64) -> Self

Create a file range without metadata or partition

pub fn from_path(path: String) -> Result<Self>

Return a file reference from the given path

pub fn path(&self) -> &Path

Return the path of this partitioned file

pub fn with_range(self, start: i64, end: i64) -> Self

Update the file to only scan the specified range (in bytes)

Trait Implementations

impl Clone for PartitionedFile

fn clone(&self) -> PartitionedFile

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PartitionedFile

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<ObjectMeta> for PartitionedFile

fn from(object_meta: ObjectMeta) -> Self

Converts to this type from the input type.