Struct compressed_vec::vector::VectorAppender

pub struct VectorAppender<T, W> where
    T: VectBase + Clone + PartialOrd,
    W: FixedSectionWriter<T>,  { /* fields omitted */ }

A builder for a BinaryVector holding encoded/compressed integral/floating values as 256-element FixedSections. Buffers elements to be written and writes them in 256-element sections at a time. This builder owns its own write buffer memory, expanding it as needed. finish() wraps up the vector, cloning a copy, and reset() can be called to reuse this appender. The write buffer stays with this builder to minimize allocations. NOTE: the vector state (elements, num bytes etc) are only updated when a section is updated. So readers who read just the vector itself will not get the updates in write_buf. This appender must be consulted for querying write_buf values.

The easiest way to encode a vector is to create an appender, then use encode_all():

     let my_vec = vec![0.5, 1.0, 1.5];
     let mut appender = VectorF32XorAppender::try_new(2048).unwrap();
     let bytes = appender.encode_all(my_vec).unwrap();

Implementations

impl<T, W> VectorAppender<T, W> where
    T: VectBase + Clone + PartialOrd + BaseSubtypeMapping,
    W: FixedSectionWriter<T>, 

pub fn try_new(initial_capacity: usize) -> Result<Self, CodingError>

Creates a new VectorAppender. Initializes the vect_buf with a valid section header. Initial capacity is the initial size of the write buffer, which can grow.

pub fn encode_all<C>(&mut self, collection: C) -> Result<Vec<u8>, CodingError> where
    C: IntoIterator<Item = T>, 

Convenience method to append all values from a collection and finish a vector, returning the encoded bytes. Appender is reset and ready to use, so this can be called repeatedly for successive vectors.

pub fn num_elements(&self) -> usize

Total number of elements including encoded sections and write buffer

pub fn reset(&mut self) -> Result<(), CodingError>

Resets the internal state for appending a new vector.

pub fn append(&mut self, value: T) -> Result<(), CodingError>

Appends a single value to this vector. When a section fills up, will encode all values in write buffer into the vector.

pub fn append_nulls(&mut self, num_nulls: usize) -> Result<(), CodingError>

Appends a number of nulls at once to the vector. Super useful and fast for sparse data. Nulls are equivalent to zero value for type T.

pub fn finish(&mut self, total_num_rows: usize) -> Result<Vec<u8>, CodingError>

Call this method to wrap up a vector and any unfinished sections, and clone out resulting vector. We have no more values, and need to fill up the appender with nulls/0's until it is the right length. This is because most query engines expect all vectors to be of the same number of elements. The number passed in will be stored as the actual number of elements for iteration purposes, however since this is a fixed size section vector, the number will be rounded up to the next FIXED_LEN so that an entire section is written. NOTE: TooFewRows is returned if total_num_rows is below the total number of elements written so far.

pub fn reader(&self) -> VectorReader<T>

Obtains a reader for reading from the bytes of this appender. NOTE: reader will only read what has been written so far, and due to Rust borrowing rules, one should not attempt to read and append at the same time; the returned reader is not safe across threads.