Struct parquet::arrow::arrow_writer::ArrowWriter
source ·
[−]pub struct ArrowWriter<W: ParquetWriter> { /* fields omitted */ }
Expand description
Arrow writer
Writes Arrow RecordBatch
es to a Parquet writer
Implementations
Try to create a new Arrow writer
The writer will fail if:
- a
SerializedFileWriter
cannot be created from the ParquetWriter - the Arrow schema contains unsupported datatypes such as Unions
Write a RecordBatch to writer
The writer will slice the batch
into max_row_group_size
,
but if a batch has left-over rows less than the row group size,
the last row group will have fewer records.
This is currently a limitation because we close the row group
instead of keeping it open for the next batch.
Close and finalize the underlying Parquet writer