Crate fsst

fsst-rs

A pure-Rust, zero-dependency implementation of the FSST string compression algorithm.

FSST is a string compression algorithm meant for use in database systems. It was designed by Peter Boncz, Thomas Neumann, and Viktor Leis. It provides 1-3GB/sec compression and decompression of strings at compression rates competitive with or better than LZ4.

This implementation is somewhat inspired by the MIT-licensed implementation from the paper authors, written in C++, but it is mostly written from a careful reading of the paper.

NOTE: This current implementation is still in-progress and is not production ready, please use at your own risk.

NOTE: This crate only works on little-endian architectures currently. There are no current plans to support big-endian targets.

Structs

Compressor

A compressor that uses a symbol table to greedily compress strings.

CompressorBuilder

Entrypoint for building a new Compressor.

Decompressor

Decompressor uses a symbol table to take a stream of 8-bit codes into a string.

Symbol

Symbols are small (up to 8-byte) segments of strings, stored in a Compressor and identified by an 8-bit code.

Constants

ESCAPE_CODE

Code used to indicate bytes that are not in the symbol table.

FSST_CODE_BASE

First code in the symbol table that corresponds to a non-escape symbol.

FSST_CODE_BITS

Number of bits in the ExtendedCode that are used to dictate a code value.

FSST_CODE_MASK

Maximum value for the extended code range.

FSST_CODE_MAX

Maximum code value in the extended code range.

FSST_LEN_BITS

First bit of the “length” portion of an extended code.