Module file_scan_config

FileScanConfig to configure scanning of possibly partitioned file sources.

Structs

FileScanConfig

The base configurations for a DataSourceExec, the a physical plan for any given file format.

FileScanConfigBuilder

A builder for FileScanConfig’s.

PartitionColumnProjector

A helper that projects partition columns into the file record batches.

Functions

wrap_partition_type_in_dict

Convert type to a type suitable for use as a ListingTable partition column. Returns Dictionary(UInt16, val_type), which is a reasonable trade off between a reasonable number of partition values and space efficiency.

wrap_partition_value_in_dict

Convert a ScalarValue of partition columns to a type, as described in the documentation of wrap_partition_type_in_dict, which can wrap the types.