fv-streams-ops
The stateful operators behind FusionVault Streams: event-time windows with keyed incremental aggregation, session windows, stream-stream interval joins, and top-N / last-N ranking. Pure and synchronous: no I/O, no runtime, no Kafka. A host feeds events in, drains fired results out, and asks for a snapshot when it checkpoints.
A tumbling window
use ;
use Value;
// 10-second tumbling windows, a 2-second watermark delay, count + sum per `auction`.
let mut w = tumbling;
w.expect_partitions;
let row = ;
w.push; // partition 0, event time 1s → the [0, 10s) window
w.push;
// Nothing fires until the watermark (max event time − delay) passes a window's end:
// after 4s the watermark is 2s, so the window is still open.
assert!;
w.push; // 13s → watermark 11s ≥ 10s: the first window can fire
let fired = w.advance;
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
push returns false for an event that arrives after every window it belongs to has fired;
late_dropped() counts them. flush() fires everything still open at shutdown.
The operators
WindowAgg— tumbling or sliding windows over event time; one running accumulator per(window, group key), O(1) per event; fires when the watermark passes a window's end.SessionAgg— session windows: a gap closes a session, overlapping sessions merge.StreamJoin— a two-input interval join on a key within a time window, with a bounded buffer on each side and eviction by watermark.TopN/LastN— unbounded per-key rankings: the top n rows by a column, or a ring of the last n aggregated rows; emit as upserts.
All of them implement WindowOperator (push / advance / flush), carry a per-partition
watermark with a bounded-idleness timeout (with_idle_timeout) so a quiet partition cannot stall
firing, and support snapshot() / load_snapshot() — a versioned, config-fingerprinted encoding
of their dynamic state, which is what lets the engine checkpoint and restore them exactly.
Aggregations
AggOp: Count, Sum, Avg, Min, Max, CountDistinct, the variance and standard-deviation
family (VarPop, VarSamp, StddevPop, StddevSamp, Welford online), boolean folds (BoolAnd,
BoolOr) and bitwise folds. Build one with Agg::count("alias") or Agg::of(op, "column", "alias"); a pipeline names them as
in the aggregations reference.
Part of FusionVault Streams, whose engine drives these operators as tasks. Apache-2.0.