rudb_encoding/lib.rs
1//! Every encoding, the cascade machinery, the cost model and multi-column detection.
2//!
3//! Rank 2 in the layer rule. See `xtask/layers.toml` and `spec/18-package-layout.md`.
4//!
5//! [`bitpack`] is the bottom of every integer encoding in `spec/06-compression.md` section 6.2 and
6//! the thing FOR, DELTA and DICT all end in. [`integer`] is those encodings and the cascade over
7//! them from section 6.3, which is where the ratios actually are. [`fsst`] is one string against
8//! one symbol table and [`string`] is a column of them, which is where most of ClickBench `hits`
9//! lives. [`sketch`] is how a write path answers a question about a column it cannot hold in
10//! memory, which is where every decision in sections 6.4 and 6.5 starts.
11
12#![forbid(unsafe_code)]
13
14pub mod bitpack;
15pub mod integer;
16pub mod multi;
17mod reader;
18pub mod sketch;
19pub mod string;
20
21/// The symbol table and the code, which live a layer down now that a vector can be in FSST form.
22///
23/// They were written here, because this is where compression is. They moved to `rudb-vector` when
24/// the vector gained the form, because a vector that holds FSST codes has to be able to read one and
25/// this crate is above it in the layer rule. What stayed here is everything that decides to use it:
26/// [`string`] trains a table on a column and picks between this and the other string encodings, and
27/// [`multi`] looks for one table that suits several columns.
28///
29/// The re-export is so that a caller that had `rudb_encoding::fsst::SymbolTable` still has it. There
30/// is one implementation and it is over there.
31pub use rudb_vector::fsst;