Struct parquet::column::reader::ColumnReaderImpl

pub struct ColumnReaderImpl<T: DataType> { /* fields omitted */ }

Typed value reader for a particular primitive column.

Implementations

impl<T: DataType> ColumnReaderImpl<T>

pub fn new(descr: ColumnDescPtr, page_reader: Box<dyn PageReader>) -> Self

Creates new column reader based on column descriptor and page reader.

pub fn read_batch(
    &mut self,
    batch_size: usize,
    def_levels: Option<&mut [i16]>,
    rep_levels: Option<&mut [i16]>,
    values: &mut [T::T]
) -> Result<(usize, usize)>

Reads a batch of values of at most batch_size.

This will try to read from the row group, and fills up at most batch_size values for def_levels, rep_levels and values. It will stop either when the row group is depleted or batch_size values has been read, or there is no space in the input slices (values/definition levels/repetition levels).

Note that in case the field being read is not required, values could contain less values than def_levels. Also note that this will skip reading def / rep levels if the field is required / not repeated, respectively.

If def_levels or rep_levels is None, this will also skip reading the respective levels. This is useful when the caller of this function knows in advance that the field is required and non-repeated, therefore can avoid allocating memory for the levels data. Note that if field has definition levels, but caller provides None, there might be inconsistency between levels/values (see comments below).

Returns a tuple where the first element is the actual number of values read, and the second element is the actual number of levels read.