Trait streamvbyte64::Coder

source ·

pub trait Coder: Sized + Copy + Clone {
    type Elem: PrimInt + WrappingAdd + WrappingSub + Debug + Sized + Copy + Clone;

    // Required methods
    fn new() -> Self;
    fn encode(
        &self,
        values: &[Self::Elem],
        tags: &mut [u8],
        data: &mut [u8]
    ) -> usize;
    fn encode_deltas(
        &self,
        initial: Self::Elem,
        values: &[Self::Elem],
        tags: &mut [u8],
        data: &mut [u8]
    ) -> usize;
    fn decode(
        &self,
        tags: &[u8],
        data: &[u8],
        values: &mut [Self::Elem]
    ) -> usize;
    fn decode_deltas(
        &self,
        initial: Self::Elem,
        tags: &[u8],
        data: &[u8],
        values: &mut [Self::Elem]
    ) -> usize;
    fn data_len(&self, tags: &[u8]) -> usize;
    fn skip_deltas(&self, tags: &[u8], data: &[u8]) -> (usize, Self::Elem);

    // Provided method
    fn max_compressed_bytes(len: usize) -> (usize, usize) { ... }
}

Expand description

Coder compresses and decompresses integers in a byte-aligned format compose of two streams.

Groups of 4 integers are coded into two separate streams: a tag stream where each byte describes a group, and a data stream containing values as described by the tag. The coder does not record the number of entries in the stream, instead assuming a multiple of 4.

Different coder implementations support different integer widths (32 or 64 bit) as well as different byte length distributions to better compress some data sets.

Use max_compressed_bytes() to compute the number of tag and data bytes that must be allocated to safely encode a slice of input values.

Required Associated Types§

source

type Elem: PrimInt + WrappingAdd + WrappingSub + Debug + Sized + Copy + Clone

The input/output element type for this coder, typically u32 or u64.

Required Methods§

source

fn new() -> Self

Create a new Coder, selecting the fastest implementation available.

These objects should be relatively cheap to create and require no heap allocation.

source

fn encode( &self, values: &[Self::Elem], tags: &mut [u8], data: &mut [u8] ) -> usize

Encodes a slice of values, writing tags and data to separate streams.

For every 4 input values one tag byte and up to std::mem::size_of::<Elem>() * 4 data bytes may be written to output.

Returns the number of bytes written to the data stream.

Panics

If values.len() % 4 != 0
If tags or data are too small to fit all of the output data.

source

fn encode_deltas( &self, initial: Self::Elem, values: &[Self::Elem], tags: &mut [u8], data: &mut [u8] ) -> usize

Encodes a slice of values, writing tags and data to separate streams.

Values are interpreted as deltas starting from initial which produces a more compact output that is also more expensive to encode and decode.

For every 4 input values one tag byte and up to std::mem::size_of::<Elem>() * 4 data bytes may be written to output.

Returns the number of bytes written to the data stream.

Panics

If values.len() % 4 != 0
If tags or data are too small to fit all of the output data.

source

fn decode(&self, tags: &[u8], data: &[u8], values: &mut [Self::Elem]) -> usize

Decodes input tags and data streams to an output slice.

Consumes all tag values in the input stream to produce tags.len() * 4 values. May consume up to std::mem::size_of<Elem>() * tags.len() * 4 bytes from data.

Returns the number of bytes consumed from the data stream.

Panics

If values.len() % 4 != 0.
If tags.len() < values.len() / 4.
If decoding would consume bytes past the end of data.

source

fn decode_deltas( &self, initial: Self::Elem, tags: &[u8], data: &[u8], values: &mut [Self::Elem] ) -> usize

Decodes input tags and data streams to an output slice.

Values are interepreted as deltas starting from initial.

Consumes all tag values in the input stream to produce tags.len() * 4 values. May consume up to std::mem::size_of<Elem>() * tags.len() * 4 bytes from data.

Returns the number of bytes consumed from the data stream.