Skip to main content

fsst_shape

Function fsst_shape 

Source
pub fn fsst_shape(values: &[&[u8]]) -> Settled
Expand description

A shape that is FSST alone, against one table trained on a sample of values.

For a writer that compresses page after page of one column with FSST and nothing else. Training is most of what FSST costs on a page of a thousand short values, and a page of l_comment trained a table of its own, which on a TPC-H lineitem load from CSV was six percent of every cycle. The table a page trains is much the same as the one the page before it trained, so the writer trains one here, hands it to encode_fsst for the pages after, and trains again when it stops paying.

The time is counted as choosing, the way the rest of the time spent deciding is.