Struct FileSinkConfig

pub struct FileSinkConfig {
    pub original_url: String,
    pub object_store_url: ObjectStoreUrl,
    pub file_group: FileGroup,
    pub table_paths: Vec<ListingTableUrl>,
    pub output_schema: SchemaRef,
    pub table_partition_cols: Vec<(String, DataType)>,
    pub insert_op: InsertOp,
    pub keep_partition_by_columns: bool,
    pub file_extension: String,
}

The base configurations to provide when creating a physical plan for writing to any given file format.

Fields

original_url: String

The unresolved URL specified by the user

object_store_url: ObjectStoreUrl

Object store URL, used to get an ObjectStore instance

file_group: FileGroup

A collection of files organized into groups. Each FileGroup contains one or more PartitionedFile objects.

table_paths: Vec<ListingTableUrl>

Vector of partition paths

output_schema: SchemaRef

The schema of the output file

table_partition_cols: Vec<(String, DataType)>

A vector of column names and their corresponding data types, representing the partitioning columns for the file

insert_op: InsertOp

Controls how new data should be written to the file, determining whether to append to, overwrite, or replace records in existing files.

keep_partition_by_columns: bool

Controls whether partition columns are kept for the file

file_extension: String

File extension without a dot(.)

Implementations

impl FileSinkConfig

pub fn output_schema(&self) -> &SchemaRef

Get output schema

Trait Implementations

impl Clone for FileSinkConfig

fn clone(&self) -> FileSinkConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileSinkConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.