Struct parquet::column::writer::GenericColumnWriter

pub struct GenericColumnWriter<'a, E: ColumnValueEncoder> { /* private fields */ }

Implementations

impl<'a, E: ColumnValueEncoder> GenericColumnWriter<'a, E>

pub fn new( descr: ColumnDescPtr, props: WriterPropertiesPtr, page_writer: Box<dyn PageWriter + 'a> ) -> Self

pub fn write_batch( &mut self, values: &E::Values, def_levels: Option<&[i16]>, rep_levels: Option<&[i16]> ) -> Result<usize>

Writes batch of values, definition levels and repetition levels. Returns number of values processed (written).

If definition and repetition levels are provided, we write fully those levels and select how many values to write (this number will be returned), since number of actual written values may be smaller than provided values.

If only values are provided, then all values are written and the length of of the values buffer is returned.

Definition and/or repetition levels can be omitted, if values are non-nullable and/or non-repeated.

pub fn write_batch_with_statistics( &mut self, values: &E::Values, def_levels: Option<&[i16]>, rep_levels: Option<&[i16]>, min: Option<&E::T>, max: Option<&E::T>, distinct_count: Option<u64> ) -> Result<usize>

Writer may optionally provide pre-calculated statistics for use when computing chunk-level statistics

NB: WriterProperties::statistics_enabled must be set to EnabledStatistics::Chunk for these statistics to take effect. If EnabledStatistics::None they will be ignored, and if EnabledStatistics::Page the chunk statistics will instead be computed from the computed page statistics

pub fn get_total_bytes_written(&self) -> u64

Returns total number of bytes written by this column writer so far. This value is also returned when column writer is closed.

Note: this value does not include any buffered data that has not yet been flushed to a page.

pub fn get_total_rows_written(&self) -> u64

Returns total number of rows written by this column writer so far. This value is also returned when column writer is closed.

pub fn get_descriptor(&self) -> &ColumnDescPtr

Returns a reference to a ColumnDescPtr

pub fn close(self) -> Result<ColumnCloseResult>

Finalizes writes and closes the column writer. Returns total bytes written, total rows written and column chunk metadata.