rudb_encoding/lib.rs
1//! Every encoding, the cascade machinery, the cost model and multi-column detection.
2//!
3//! Rank 2 in the layer rule. See `xtask/layers.toml` and `spec/18-package-layout.md`.
4//!
5//! [`bitpack`] is the bottom of every integer encoding in `spec/06-compression.md` section 6.2 and
6//! the thing FOR, DELTA and DICT all end in. [`integer`] is those encodings and the cascade over
7//! them from section 6.3, which is where the ratios actually are. [`fsst`] is one string against
8//! one symbol table and [`string`] is a column of them, which is where most of ClickBench `hits`
9//! lives. [`sketch`] is how a write path answers a question about a column it cannot hold in
10//! memory, which is where every decision in sections 6.4 and 6.5 starts. [`chooser`] is the search
11//! over all of that, held apart from the encodings themselves so that how long the writer is willing
12//! to spend deciding is a knob rather than a property of the format.
13
14#![forbid(unsafe_code)]
15
16pub mod bitpack;
17pub mod chooser;
18pub mod integer;
19mod lz;
20pub mod multi;
21mod reader;
22pub mod sketch;
23pub mod string;
24
25/// The symbol table and the code, which live a layer down now that a vector can be in FSST form.
26///
27/// They were written here, because this is where compression is. They moved to `rudb-vector` when
28/// the vector gained the form, because a vector that holds FSST codes has to be able to read one and
29/// this crate is above it in the layer rule. What stayed here is everything that decides to use it:
30/// [`string`] trains a table on a column and picks between this and the other string encodings, and
31/// [`multi`] looks for one table that suits several columns.
32///
33/// The re-export is so that a caller that had `rudb_encoding::fsst::SymbolTable` still has it. There
34/// is one implementation and it is over there.
35pub use rudb_vector::fsst;