Module write

Module containing helper methods/traits related to enabling write support for the various file formats

Modules

demux

Module containing helper methods/traits related to enabling dividing input stream into multiple output files at execution time

orchestration

Module containing helper methods/traits related to orchestrating file serialization, streaming to object store, parallelization, and abort handling

Structs

ObjectWriterBuilder

A builder for an AsyncWrite that writes to an object store location.

SharedBuffer

A buffer with interior mutability shared by the SerializedFileWriter and ObjectStore writer

Traits

BatchSerializer

A trait that defines the methods required for a RecordBatch serializer.

Functions

create_writerDeprecated

Returns an AsyncWrite which writes to the given object store location with the specified compression.

get_writer_schema

Converts table schema to writer schema, which may differ in the case of hive style partitioning where some columns are removed from the underlying files.