Struct ChunkAssembler 

pub struct ChunkAssembler { /* private fields */ }

Bridges arbitrary-length row batches onto the fixed chunk partition.

A streaming row source (gam_sae::corpus) yields batches whose lengths are set by I/O policy (batch size, shard boundaries) — they do not align with the deterministic chunk partition the accumulation tree is keyed on. This assembler buffers incoming rows and submits exact chunks in order, so the resulting Gram is bit-identical to having sliced the partition directly: the batching of the producer can never leak into the bits.

Checkpointing is exposed at chunk granularity only: ChunkAssembler::checkpoint returns Some exactly when the internal buffer is empty (a chunk boundary), because buffered raw rows are not part of the accumulation state contract — a resumed pass re-reads its row stream from the checkpointed chunk cursor (StreamingBorderGram::chunk_rows of the frontier names the next row).

Implementations

impl ChunkAssembler

pub fn new( border_dim: usize, n_rows: usize, chunk_size: usize, ) -> Result<Self, String>

New assembler over the same partition parameters as StreamingBorderGram::new.

pub fn push_rows(&mut self, rows: ArrayView2<'_, f64>) -> Result<(), String>

Append a batch of rows (any length, including empty) in stream order, submitting every chunk the buffer completes.

pub fn checkpoint(&self) -> Option<BorderGramCheckpoint>

Serialize the accumulation state — only at a chunk boundary. None while rows are buffered mid-chunk (checkpoint after the next boundary, or size batches to the chunk size for checkpoint-every-batch).

pub fn resume(state: BorderGramCheckpoint) -> Result<Self, String>

Resume an assembler at the chunk boundary a checkpoint names. The caller re-positions its row stream at row checkpoint.frontier * checkpoint.chunk_size (the partition is pure, so that index is exact) and replays from there.

pub fn finish(self) -> Result<Array2<f64>, String>

Finish the pass. Errors if the stream ended mid-chunk or short of the declared row count — a truncated stream is rejected loudly, never folded as a silently shorter corpus.