Module columnar 

Columnar container infrastructure for differential dataflow.

Experimental. API and internals are still settling. Expect breaking changes; do not rely on stability across releases.

Known rough edges:

• ContainerBytes for UpdatesTyped is unimplemented!(). The wire-side container is RecordedUpdates, whose ContainerBytes is implemented; UpdatesTyped is the input-builder type and isn’t shipped over channels.
• leave_dynamic consolidates eagerly on each batch; the crate::dynamic counterpart defers consolidation. Same observable semantics, different work distribution.
• join_function is restricted to same-ColumnarUpdate input and output; it does not yet generalize to Key/Val/Diff-changing maps.
• Several public items (join_function, leave_dynamic, DynTime) have no in-tree callers yet and are not exercised by tests.

Files inside this module that touch both the local module path and the columnar crate should use columnar as col; to disambiguate.

Module layout (bottom-up):

• layout — ColumnarUpdate / ColumnarLayout / OrdContainer.
• updates — UpdatesTyped<U> trie, Consolidating, UpdatesBuilder.
• builder — ValColBuilder: the input-side ContainerBuilder.
• exchange — ValPact / ValDistributor: PACT for shuffling.
• arrangement — type aliases + Coltainer + TrieChunker + trie_merger + ValMirror (trace Builder).
• This file — RecordedUpdates<U> (the stream container), container-trait impls (Negate, Enter, Leave, ResultsIn), and top-level operators (join_function, leave_dynamic, as_recorded_updates).

Re-exports

pub use updates::UpdatesTyped;

pub use builder::ValBuilder as ValColBuilder;

pub use exchange::ValPact;

pub use arrangement::ValBatcher;

pub use arrangement::ValBuilder;

pub use arrangement::ValChunker;

pub use arrangement::ValSpine;

Modules

arrangement

Columnar arrangement plumbing.

batcher

A Batcher for columnar streams that merges sorted chains via the free functions in trie_merger.

builder

ValColBuilder: the ContainerBuilder that feeds the dataflow input side.

exchange

Exchange / parallelization contract for RecordedUpdates.

layout

Layout traits for columnar arrangements.

spill

Traits for paging chunks of merge-batcher state to and from backing storage.

updates

Trie-structured update storage.

Structs

RecordedUpdates

A thin wrapper around Updates that tracks the pre-consolidation record count for timely’s exchange accounting. This wrapper is the stream container type; the TrieChunker strips it, passing bare UpdatesTyped into the merge batcher.

Constants

LINK_TARGET

Target size for update batches, in number of updates.

Functions

as_recorded_updates

Extract a Collection<_, RecordedUpdates<U>> from a columnar Arranged.

join_function

A columnar flat_map: iterates RecordedUpdates, calls logic per (key, val, time, diff), joins output times with input times, multiplies output diffs with input diffs.

leave_dynamic

Leave a dynamic iterative scope, truncating PointStamp coordinates.

Type Aliases

DynTime

Timestamp shape of a dynamic iterative scope: an outer timestamp paired with a per-level PointStamp of loop counters.