rudb_encoding/lib.rs
1//! Every encoding, the cascade machinery, the cost model and multi-column detection.
2//!
3//! Rank 2 in the layer rule. See `xtask/layers.toml` and `spec/18-package-layout.md`.
4//!
5//! [`bitpack`] is the bottom of every integer encoding in `spec/06-compression.md` section 6.2 and
6//! the thing FOR, DELTA and DICT all end in. [`integer`] is those encodings and the cascade over
7//! them from section 6.3, which is where the ratios actually are. [`fsst`] is one string against
8//! one symbol table and [`string`] is a column of them, which is where most of ClickBench `hits`
9//! lives. [`sketch`] is how a write path answers a question about a column it cannot hold in
10//! memory, which is where every decision in sections 6.4 and 6.5 starts. [`chooser`] is the search
11//! over all of that, held apart from the encodings themselves so that how long the writer is willing
12//! to spend deciding is a knob rather than a property of the format.
13
14#![forbid(unsafe_code)]
15
16pub mod bitpack;
17pub mod chooser;
18pub mod integer;
19mod lz;
20pub mod multi;
21mod reader;
22pub mod sequence;
23pub mod sketch;
24pub mod string;
25pub mod tally;
26
27/// The symbol table and the code, which live a layer down now that a vector can be in FSST form.
28///
29/// They were written here, because this is where compression is. They moved to `rudb-vector` when
30/// the vector gained the form, because a vector that holds FSST codes has to be able to read one and
31/// this crate is above it in the layer rule. What stayed here is everything that decides to use it:
32/// [`string`] trains a table on a column and picks between this and the other string encodings, and
33/// [`multi`] looks for one table that suits several columns.
34///
35/// The re-export is so that a caller that had `rudb_encoding::fsst::SymbolTable` still has it. There
36/// is one implementation and it is over there.
37pub use rudb_vector::fsst;