Expand description
Runtime configuration, via ConfigOptions
Structs§
- Catalog
Options - Options related to catalog and directory scanning
- Column
Decryption Properties - Column
Encryption Properties - Config
Entry - A key value pair, with a corresponding description
- Config
File Decryption Properties - Config
File Encryption Properties - Config
MinTwo Usize - A
usizeconfiguration value that rejects 0 and 1 when set from strings. - Config
NonZero Usize - A
usizeconfiguration value that rejects zero when set from strings. - Config
Options - Configuration options struct, able to store both built-in configuration and custom options
- CsvOptions
- Options controlling CSV format
- Dialect
Info - Metadata for a SQL dialect supported by DataFusion configuration.
- Encryption
Factory Options - Holds implementation-specific options for an encryption factory
- Execution
Options - Options related to query execution
- Explain
Options - Options controlling explain output
- Extensions
- A type-safe container for
ConfigExtension - Format
Options - Options controlling the format of output when printing record batches
Copies
arrow::util::display::FormatOptions - Json
Options - Options controlling JSON format
- MaxRow
Group Bytes - Target maximum size of a Parquet row group in bytes.
- Optimizer
Options - Options related to query optimization
- Parquet
CdcOptions - Options for content-defined chunking (CDC) when writing parquet files.
Mirrors
parquet::file::properties::CdcOptions. - Parquet
Column Options - Options controlling parquet format for individual columns.
- Parquet
Encryption Options - Options for configuring Parquet Modular Encryption
- Parquet
Options - Options for reading and writing parquet files
- Spark
Options - Options controlling DataFusion’s Spark-compatibility layer (functions
under
datafusion/spark). Keys here mirror theirspark.sql.*equivalents in Apache Spark. - SqlParser
Options - Options related to SQL parser
- Table
Options - Represents the configuration options available for handling different table formats within a data processing application. This struct encompasses options for various file formats including CSV, Parquet, and JSON, allowing for flexible configuration of parsing and writing behaviors specific to each format. Additionally, it supports extending functionality through custom extensions.
- Table
Parquet Options - Options that control how Parquet files are read, including global options that apply to all columns and optional column-specific overrides
Enums§
- Config
File Type - These file types have special built in behavior for configuration. Use TableOptions::Extensions for configuring other file types.
- Dialect
- This is the SQL dialect used by DataFusion’s parser.
This mirrors sqlparser::dialect::Dialect
trait in order to offer an easier API and avoid adding the
sqlparserdependency - MapKey
Dedup Policy - Policy for handling duplicate keys in Spark-compatible map-construction
functions (
map_from_arrays,map_from_entries,str_to_map). Mirrors Spark’sspark.sql.mapKeyDedupPolicy. - Output
Format - Spill
Compression
Constants§
- DATAFUSION_
FFI_ CONFIG_ NAMESPACE - This namespace is reserved for interacting with Foreign Function Interface (FFI) based configuration extensions.
Traits§
- Config
Extension ConfigExtensionprovides a mechanism to store third-party configuration within DataFusionConfigOptions- Config
Field - A trait implemented by
config_namespaceand for field types that provides the ability to walk and mutate the configuration tree - Extension
Options - An object-safe API for storing arbitrary configuration.
- Output
Format Ext - Visit
- An implementation trait used to recursively walk configuration
Functions§
- default_
config_ transform - Default transformation to parse a
ConfigFieldfor a string.