Expand description
Every encoding, the cascade machinery, the cost model and multi-column detection.
Rank 2 in the layer rule. See xtask/layers.toml and spec/18-package-layout.md.
bitpack is the bottom of every integer encoding in spec/06-compression.md section 6.2 and
the thing FOR, DELTA and DICT all end in. integer is those encodings and the cascade over
them from section 6.3, which is where the ratios actually are. fsst is one string against
one symbol table and string is a column of them, which is where most of ClickBench hits
lives. sketch is how a write path answers a question about a column it cannot hold in
memory, which is where every decision in sections 6.4 and 6.5 starts.
Modulesยง
- bitpack
- Bit packing in the FastLanes unified transposed layout.
- fsst
- FSST, the string encoding.
- integer
- The single column integer encodings and the cascade over them.
- multi
- Columns encoded together instead of one at a time.
- sketch
- Counting distinct values, and deciding whether two columns are related, without holding either column in memory.
- string
- The string column, which is offsets, bytes, and the choice between compressing the bytes and not storing most of them at all.