pub struct Sampled { /* private fields */ }Expand description
Encode every candidate on a sample, then encode only the winner on the whole chunk.
The bet is that a chunk of 122,880 values and a sample of 8,192 drawn from it agree about which encoding suits them, which is a bet about the data and is what the ablation settles. Where it is wrong the cost is size and never correctness, because the winner still has to apply to the whole chunk and is still encoded over all of it.
The sample is windows of consecutive values rather than values picked one at a time, because
three of the candidates are about what a value has in common with the value before it. A sample
of scattered singletons would show FRONT and RLE nothing to find and would rule them out on
every column, which is the wrong answer arrived at quickly.
There are two guards on whether to sample at all and both of them are there because a measurement said so. A chunk with fewer values than the sample is not sampled, because encoding every candidate on something the size of the chunk and then encoding the winner on the chunk is more work than the exhaustive chooser for the same answer. A chunk holding less than a page of bytes is not sampled either, because the cost of the search scales with the bytes in the chunk and not with how many values they are spread over, so on a narrow column there is nothing to save and a sample that misses the structure gives up real size for it.