Expand description
A worked Chunk: the columnar UpdatesTyped<U> trie, resident or paged.
Where vec backs a chunk with a flat Vec<((K,V),T,R)>, this
backs it with the column-oriented trie from crate::columnar::updates —
deduplicated keys, per-key val runs, per-val (time, diff) runs. It is a
retargeting of the columnar trace pile at the Chunk abstraction: the
storage (UpdatesTyped) and the trie-native merge (trie_merger) are reused
verbatim, and the four transducers delegate to them. The harness
(ChunkBatch, the straddle cursor, the batcher/builder/
spine aliases) is shared with vec.
This makes columnar trace merges trie-native (the old OrdValBatch-backed
trace ran them through ord_neu’s row-oriented merger).
§Resident vs paged
A ColChunk is either Resident (the trie in memory)
or Paged (resident bounds + a byte handle). Chunk::settle
is the spill point: it pages committed chunks out via
spill when a worker has installed a spiller. Reads
fetch a paged chunk’s trie back, caching it in a OnceCell so repeated
cursor access pays the fetch once; len and bounds
read the resident metadata and never fetch.
Chunk::advance is trie-native: it melds its input in place and rewrites
only the time column (keys/vals/diffs stay columnar refs), then re-consolidates
the advanced time runs (a global pass a future per-(key,val) streaming
consolidation could narrow).
Structs§
- Chunk
Meta - Resident bounds for a paged chunk: the first and last
(key, val, time)as single-element columnar containers (so [Chunk::bounds] returns refs without fetching), plus the record count. - ColChunk
Cursor - A cursor over a
ColChunk, tracking the current key and value as absolute indices into the trie’s flatkeys.values/vals.valuescolumns. - Paged
Chunk - A paged chunk: resident
meta, a byte handle to fetch the trie, and a cache populated on first read. Opaque — held insideColChunk::Paged.
Enums§
- ColChunk
- A sorted, consolidated columnar trie of
((key, val), time, diff): resident, or paged to backing storage.