Struct parquet::encoding::DeltaBitPackEncoder

pub struct DeltaBitPackEncoder<T: DataType> { /* fields omitted */ }

Delta bit packed encoder. Consists of a header followed by blocks of delta encoded values binary packed.

Delta-binary-packing:

  [page-header] [block 1], [block 2], ... [block N]

Each page header consists of:

  [block size] [number of miniblocks in a block] [total value count] [first value]

Each block consists of:

  [min delta] [list of bitwidths of miniblocks] [miniblocks]

Current implementation writes values in put method, multiple calls to put to existing block or start new block if block size is exceeded. Calling flush_buffer writes out all data and resets internal state, including page header.

Supports only INT32 and INT64.

Implementations

impl<T: DataType> DeltaBitPackEncoder<T>

pub fn new() -> Self

Creates new delta bit packed encoder.

Trait Implementations

impl<T: DataType> Encoder<T> for DeltaBitPackEncoder<T>

fn put(&mut self, values: &[T::T]) -> Result<()>

Encodes data from values.

fn encoding(&self) -> Encoding

Returns the encoding type of this encoder.

fn estimated_data_encoded_size(&self) -> usize

Returns an estimate of the encoded data, in bytes. Method call must be O(1).

fn flush_buffer(&mut self) -> Result<ByteBufferPtr>

Flushes the underlying byte buffer that’s being processed by this encoder, and return the immutable copy of it. This will also reset the internal state.

fn put_spaced(&mut self, values: &[T::T], valid_bits: &[u8]) -> Result<usize>

Encodes data from values, which contains spaces for null values, that is identified by valid_bits.