Expand description
DataFrameReader & DataFrameWriter representations
Structs§
- Common
File Options - Common file options that are shared across multiple file formats (e.g., CSV, JSON, ORC, Parquet, Text).
- CsvOptions
- A struct that represents options for configuring CSV file parsing.
- Data
Frame Reader - DataFrameReader represents the entrypoint to create a DataFrame from a specific file format.
- Data
Frame Writer - DataFrameWriter provides the ability to output a DataFrame to a specific file format supported by Spark
- Data
Frame Writer V2 - Json
Options - A struct that represents options for configuring JSON file parsing.
- OrcOptions
- A struct that represents options for configuring ORC file parsing.
- Parquet
Options - A struct that represents options for configuring Parquet file parsing.
- Text
Options - A struct that represents options for configuring text file parsing.
Traits§
- Config
Opts - A trait used to convert to the expected Spark Options for readwriters
- ToSchema
- A trait used to a create a DDL string or JSON string