Module datafusion::datasource::file_format

Module containing helper methods for the various file formats See write.rs for write related helper methods

Modules

• arrow
  ArrowFormat: Apache Arrow FileFormat abstractions
• avro
  AvroFormat Apache Avro FileFormat abstractions
• csv
  CsvFormat, Comma Separated Value (CSV) FileFormat abstractions
• file_compression_type
  File Compression type abstraction
• json
  JsonFormat: Line delimited JSON FileFormat abstractions
• options
  User facing options for the file formats readers
• parquet
  ParquetFormat: Parquet FileFormat abstractions
• write
  Module containing helper methods/traits related to enabling write support for the various file formats

Structs

• DefaultFileType
  A container of FileFormatFactory which also implements FileType. This enables converting a dyn FileFormat to a dyn FileType. The former trait is a superset of the latter trait, which includes execution time relevant methods. FileType is only used in logical planning and only implements the subset of methods required during logical planning.

Constants

• DEFAULT_SCHEMA_INFER_MAX_RECORD
  Default max records to scan to infer the schema

Traits

• FileFormat
  This trait abstracts all the file format specific implementations from the TableProvider. This helps code re-utilization across providers that support the same file formats.
• FileFormatFactory
  Factory for creating FileFormat instances based on session and command level options

Functions

• file_type_to_format
  Converts a FileType to a FileFormatFactory. Returns an error if the FileType cannot be downcasted to a DefaultFileType.
• format_as_file_type
  Converts a FileFormatFactory to a FileType
• transform_schema_to_view
  Transform a schema to use view types for Utf8 and Binary