fv-streams-runtime 0.6.0

The dataflow runtime of FusionVault Streams: tasks over bounded in-memory edges carrying Arrow batches, watermarks and barriers in band; epochs; the cross-worker exchange; placement; the cluster handshake.
Documentation

fv-streams-runtime

The dataflow runtime of FusionVault Streams: tasks on OS threads over bounded in-memory edges carrying Arrow batches, watermarks and checkpoint barriers in band; epochs from a coordinator; a cross-worker exchange; placement over a member set; the cluster handshake. It runs the contract crate's sources, sinks and operators and knows nothing else — no connector, no operator algebra, not even the shuffle's hash function, which the planner supplies.

use fv_streams_runtime::dataflow::vnode_ranges;

// 64 vnodes over 4 tasks: contiguous, disjoint, covering
let ranges = vnode_ranges(64, 4);
assert_eq!(ranges.len(), 4);
assert_eq!(ranges[0], 0..16);
assert_eq!(ranges[3], 48..64);

License

Apache-2.0.