Struct SymbolTable

Source

pub struct SymbolTable { /* private fields */ }

Expand description

A trained symbol table, and everything needed to compress and decompress against it.

Implementations§

Source §

impl SymbolTable

Source

pub fn empty() -> Self

A table with no symbols, which escapes everything and doubles its input. The starting point of training, and what a column of nothing but unique bytes ends up with.

Source

pub fn train(samples: &[&[u8]]) -> Self

Trains a table on a sample.

The caller picks the sample. Section 6.3 says a systematic sample across the chunk rather than the first N rows, because column data is frequently clustered, and that decision belongs to whoever knows what the chunk is rather than to this function.

Source

pub fn len(&self) -> usize

How many symbols are in the table.

Source

pub fn is_empty(&self) -> bool

Whether the table has no symbols, in which case every byte of every string escapes.

Source

pub fn serialized_len(&self) -> usize

How many bytes serialize writes. At most 2049 for a full table, and that is the number section 6.4 is weighing when it says a shared symbol table is cheaper than a shared dictionary.

Source

pub fn serialize(&self, out: &mut Vec<u8>)

Writes the table itself, which has to travel with the data it compressed.

Source

pub fn deserialize(bytes: &[u8]) -> Result<(Self, usize)>

Reads back what serialize wrote, and says how many bytes it consumed.

§Errors

If the bytes are truncated or describe a symbol of zero or more than eight bytes.

Source

pub fn compress(&self, input: &[u8], out: &mut Vec<u8>)

Compresses one string, appending to out.

Strings are compressed one at a time against a shared table rather than as one stream, because that is what keeps random access, which is the first of the two reasons this encoding was chosen at all.

Source