Skip to main content

Module buffering

Module buffering 

Source
Expand description

Table buffering (sink + source) function model.

Lifecycle (keyed by execution_id):

  1. init phase TABLE_BUFFERING (sink) — mint execution_id, header-only.
  2. table_buffering_process (unary, per input batch) → state_id.
  3. table_buffering_combine (unary, once) → finalize_state_ids.
  4. init phase TABLE_BUFFERING_FINALIZE (source, per finalize_state_id) → a producer that drains the buffered state.
  5. table_buffering_destructor (unary) — cleanup.

State is held in the worker’s FunctionStorage backend (cross-process: the subprocess transport pools workers, so the sink and source phases can run in different PIDs).

Structs§

BufferingParams
Parameters for buffering process / combine / finalize.

Traits§

TableBufferingFunction
A table buffering (sink+source) function.