Module vector 

The vector itself.

spec/07-execution.md section 7.1 calls this the widest interface in the system, says every operator depends on it, and says changing it after twenty operators exist is expensive. So it is written before the first operator rather than after the fifth.

A vector is a type, a length of at most VECTOR_SIZE, a physical form, a validity representation and some data. Four of the forms are the ones in spec/04-architecture.md section 4.3: flat, constant, sequence and dictionary. Run length, bit packed and string view come after them, one at a time with the kernels that read them rather than all at once ahead of anything that can use them.

Dictionary and run length are the pair worth understanding together, because they answer different questions about the same column. A dictionary says which distinct values there are, so it wins on low cardinality however the rows are ordered. Run length says where the values stop, so it wins on a clustered column however many distinct values it has. A column can want either one without wanting the other, and hits has columns of both kinds.

String view is the odd one out, because it is not about making a column smaller. It is about who owns the bytes: the views are the vector’s and the arena is shared, so cutting a chunk out of a page of strings moves sixteen bytes a row and copies none of the payload. Every other form here trades a little work per row for less memory, and that one trades nothing at all.

What is not here yet. Buffers are owned. Section 7.1 says a vector borrowed from a buffer managed page carries a pin, and there is no buffer manager until M2, so there is nothing to pin and pretending otherwise would be an interface built against an imaginary caller. Nested types are not stored yet either, for the same reason: a LIST(STRUCT(...)) is offsets plus child column chunks, and child column chunks are storage.

Structs

Coded

The codes of a compressed column and the table they are against.

Packed

The bits of a packed vector and what they mean, for a kernel that wants to stay in code space.

Vector

A type, a length, a validity representation and some data.

Enums

Data

The values of a flat vector, one Rust vector per physical type.

Form

Which physical form a vector is in.

Constants

FSST_PAYS_AT

How much smaller compressing has to be before it is worth a decompression on every read.

PACKED_WIDTH_MAX

The widest a packed code is allowed to be.

PACKING_PAYS_AT

How much smaller packing has to be before it is worth the shift and the mask on every read.

VECTOR_SIZE

How many values are in a full vector.