Struct ListingTableConfig 

pub struct ListingTableConfig {
    pub table_paths: Vec<ListingTableUrl>,
    pub file_schema: Option<SchemaRef>,
    pub options: Option<ListingOptions>,
    /* private fields */
}

Configuration for creating a crate::ListingTable

Schema Evolution Support

This configuration supports schema evolution through the optional PhysicalExprAdapterFactory. You might want to override the default factory when you need:

• Type coercion requirements: When you need custom logic for converting between different Arrow data types (e.g., Int32 ↔ Int64, Utf8 ↔ LargeUtf8)
• Column mapping: You need to map columns with a legacy name to a new name
• Custom handling of missing columns: By default they are filled in with nulls, but you may e.g. want to fill them in with 0 or "".

Fields

table_paths: Vec<ListingTableUrl>

Paths on the ObjectStore for creating crate::ListingTable. They should share the same schema and object store.

file_schema: Option<SchemaRef>

Optional SchemaRef for the to be created crate::ListingTable.

See details on ListingTableConfig::with_schema

options: Option<ListingOptions>

Optional ListingOptions for the to be created crate::ListingTable.

See details on ListingTableConfig::with_listing_options

Implementations

impl ListingTableConfig

pub fn new(table_path: ListingTableUrl) -> Self

Creates new ListingTableConfig for reading the specified URL

pub fn new_with_multi_paths(table_paths: Vec<ListingTableUrl>) -> Self

Creates new ListingTableConfig with multiple table paths.

See ListingTableConfigExt::infer_options for details on what happens with multiple paths

pub fn schema_source(&self) -> SchemaSource

Returns the source of the schema for this configuration

pub fn with_schema(self, schema: SchemaRef) -> Self

Set the schema for the overall crate::ListingTable

crate::ListingTable will automatically coerce, when possible, the schema for individual files to match this schema.

If a schema is not provided, it is inferred using Self::infer_schema.

If the schema is provided, it must contain only the fields in the file without the table partitioning columns.

Example: Specifying Table Schema

let schema = Arc::new(Schema::new(vec![
    Field::new("id", DataType::Int64, false),
    Field::new("name", DataType::Utf8, true),
]));

let config = ListingTableConfig::new(table_paths)
    .with_listing_options(listing_options)  // Set options first
    .with_schema(schema);                    // Then set schema

pub fn with_listing_options(self, listing_options: ListingOptions) -> Self

Add listing_options to ListingTableConfig

If not provided, format and other options are inferred via ListingTableConfigExt::infer_options.

Example: Configuring Parquet Files with Custom Options

let options = ListingOptions::new(Arc::new(ParquetFormat::default()))
    .with_file_extension(".parquet")
    .with_collect_stat(true);

let config = ListingTableConfig::new(table_paths).with_listing_options(options);
// Configure file format and options

pub fn infer_file_extension_and_compression_type( path: &str, ) -> Result<(String, Option<String>)>

Returns a tuple of (file_extension, optional compression_extension)

For example a path ending with blah.test.csv.gz returns ("csv", Some("gz")) For example a path ending with blah.test.csv returns ("csv", None)

pub async fn infer_schema(self, state: &dyn Session) -> Result<Self>

Infer the SchemaRef based on table_paths.

This method infers the table schema using the first table_path. See ListingOptions::infer_schema for more details

Errors

• if self.options is not set. See Self::with_listing_options

pub async fn infer_partitions_from_path( self, state: &dyn Session, ) -> Result<Self>

Infer the partition columns from table_paths.

Errors

• if self.options is not set. See Self::with_listing_options

pub fn with_expr_adapter_factory( self, expr_adapter_factory: Arc<dyn PhysicalExprAdapterFactory>, ) -> Self

Set the PhysicalExprAdapterFactory for the crate::ListingTable

The expression adapter factory is used to create physical expression adapters that can handle schema evolution and type conversions when evaluating expressions with different schemas than the table schema.

pub fn with_schema_adapter_factory( self, _schema_adapter_factory: Arc<dyn SchemaAdapterFactory>, ) -> Self

👎Deprecated since 52.0.0: SchemaAdapterFactory has been removed. Use with_expr_adapter_factory and PhysicalExprAdapterFactory instead. See upgrading.md for more details.

Deprecated: Set the SchemaAdapterFactory for the crate::ListingTable

SchemaAdapterFactory has been removed. Use Self::with_expr_adapter_factory and PhysicalExprAdapterFactory instead. See upgrading.md for more details.

This method is a no-op and returns self unchanged.

Trait Implementations

impl Clone for ListingTableConfig

fn clone(&self) -> ListingTableConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ListingTableConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ListingTableConfig

fn default() -> ListingTableConfig

Returns the “default value” for a type.