#[non_exhaustive]pub enum Form {
Flat,
Constant,
Sequence,
Dictionary,
BitPacked,
Rle,
}Expand description
Which physical form a vector is in.
An operator asks this once per vector and then takes the path it wants, which is the one branch per vector that the whole design is willing to spend.
Not exhaustive, and that is a decision rather than an oversight. Encoded is the fifth form
and it arrives at layer three with the specialization contract. If this enum were exhaustive,
the day it lands is the day every kernel in the workspace stops compiling, and the pressure at
that moment would be to add an arm to each of them in a hurry rather than to think about what
each one should do with an encoded vector. A required fallback arm means each kernel already
has a correct answer for a form it has never seen, and specializing it is then a change that
can be made one kernel at a time with a benchmark next to it.
Variants (Non-exhaustive)§
This enum is marked as non-exhaustive
Flat
One value per position.
Constant
One value, repeated.
Sequence
A start and a step, computed rather than stored.
Dictionary
Codes into a smaller vector of distinct values.
BitPacked
Integers stored in as many bits as the range of the column needs, offset from a base.
The form a narrow integer column is in. A ClickBench ResolutionWidth is a SMALLINT whose
values live between 0 and 2560, which is twelve bits, so the column is three quarters of the
size it was and the pages behind it are three quarters of the reads. What it costs is a shift
and a mask per value, which is why this is worth it at storage and at rest and is not a form
anything should be building in the middle of a pipeline.
Rle
One value per run, with the row each run ends at.
The form a clustered column is in. hits is written in time order, so EventDate is a few
hundred runs over a hundred million rows, and a sum over it is a few hundred multiplications
rather than a hundred million additions. Dictionary says which distinct values there are and
this says where they stop, and a column can want either one without wanting the other.