Struct parquet::column::writer::ColumnWriterImpl

pub struct ColumnWriterImpl<T: DataType> { /* fields omitted */ }

Typed column writer for a primitive column.

Implementations

impl<T: DataType> ColumnWriterImpl<T>

pub fn new(
    descr: ColumnDescPtr,
    props: WriterPropertiesPtr,
    page_writer: Box<dyn PageWriter>
) -> Self

pub fn write_batch(
    &mut self,
    values: &[T::T],
    def_levels: Option<&[i16]>,
    rep_levels: Option<&[i16]>
) -> Result<usize>

Writes batch of values, definition levels and repetition levels. Returns number of values processed (written).

If definition and repetition levels are provided, we write fully those levels and select how many values to write (this number will be returned), since number of actual written values may be smaller than provided values.

If only values are provided, then all values are written and the length of of the values buffer is returned.

Definition and/or repetition levels can be omitted, if values are non-nullable and/or non-repeated.

pub fn write_batch_with_statistics(
    &mut self,
    values: &[T::T],
    def_levels: Option<&[i16]>,
    rep_levels: Option<&[i16]>,
    min: &Option<T::T>,
    max: &Option<T::T>,
    nulls_count: Option<u64>,
    distinct_count: Option<u64>
) -> Result<usize>

Writer may optionally provide pre-calculated statistics for this batch, in which case we do not calculate page level statistics as this will defeat the purpose of speeding up the write process with pre-calculated statistics.

pub fn get_total_bytes_written(&self) -> u64

Returns total number of bytes written by this column writer so far. This value is also returned when column writer is closed.

pub fn get_total_rows_written(&self) -> u64

Returns total number of rows written by this column writer so far. This value is also returned when column writer is closed.

pub fn close(self) -> Result<(u64, u64, ColumnChunkMetaData)>

Finalises writes and closes the column writer. Returns total bytes written, total rows written and column chunk metadata.

pub fn make_typed_statistics(&self, level: Level) -> Statistics