Struct parquet::arrow::arrow_writer::ArrowWriter

pub struct ArrowWriter<W: ParquetWriter> { /* fields omitted */ }

Arrow writer

Writes Arrow RecordBatches to a Parquet writer

Implementations

impl<W: 'static + ParquetWriter> ArrowWriter<W>

pub fn try_new(
    writer: W,
    arrow_schema: SchemaRef,
    props: Option<WriterProperties>
) -> Result<Self>

Try to create a new Arrow writer

The writer will fail if:

• a SerializedFileWriter cannot be created from the ParquetWriter
• the Arrow schema contains unsupported datatypes such as Unions

pub fn write(&mut self, batch: &RecordBatch) -> Result<()>

Write a RecordBatch to writer

The writer will slice the batch into max_row_group_size, but if a batch has left-over rows less than the row group size, the last row group will have fewer records. This is currently a limitation because we close the row group instead of keeping it open for the next batch.

pub fn close(&mut self) -> Result<FileMetaData>

Close and finalize the underlying Parquet writer