Expand description
Streaming flow runtime: continuously evaluates registered flow definitions over the change stream coming out of the transaction layer, applies the operator graph the planner produced, and writes the resulting deltas back into the catalog so downstream queries observe a derived view that updates in step with its inputs.
The runtime hosts both built-in operators and FFI-loaded ones from extensions, threading them through a shared deferred-work queue so backpressure from a slow consumer does not block fast ones. Connectors at the edges of the graph translate between the engine’s internal column shape and external sources/sinks.
Invariant: a flow’s output for a given input set is fully determined by its definition - replaying the same input deltas through the same flow definition produces the same output deltas. Operators that introduce hidden state (a clock, a random number, an external read that may differ between runs) break this guarantee and break replay.
Modules§
- builder
- connector
- Source and sink connector registry. Connectors that authoring code defines through
reifydb-sdkare registered here at boot, and the runtime instantiates them at flow start using the typed factory closures. Source factories produce inputs; sink factories accept outputs; both share the same registration and configuration shape. - engine
- Flow execution engine. Registers compiled flow definitions, evaluates each flow’s operator graph against incoming change deltas, and writes the resulting outputs back through the catalog. Process drives the per-tick work; eval is where individual operators run; register is the wiring step that turns a flow definition into an executable graph.
- error
- host
- operator
- subsystem
- testing
- transaction