Module validity 

Per-block validity bitmaps (.csx v2, D-NULL).

Before v2 an absent field was encoded as a type-specific sentinel — 0, "" or false — and only an aggregate null_count survived in the block statistics. A stored zero and an absent field were therefore indistinguishable at read time, so IS NULL never matched, = 0 matched rows that had no value at all, and AVG/MIN/COUNT silently folded sentinels into their results.

v2 stores an Arrow-compatible validity bitmap next to each column block: LSB-first packed bits, one per row, set = the row holds a real value. Blocks with no nulls store no bitmap at all (validity_length == 0), so dense time-series data — the common case — pays nothing.

The bitmap is written uncompressed and unencrypted. It leaks only which rows have values, never what they are, and keeping it outside the compressed/encrypted payload lets the reader build an Arrow NullBuffer with a single copy.

Structs

ValidityBuilder

Incrementally builds a validity bitmap while a column block is encoded.

Functions

bitmap_len

Number of bytes needed to hold len validity bits.

null_buffer_from_bytes

Rebuild an Arrow NullBuffer from a stored bitmap.