Expand description
Runtime-checked graph, shape, port, and junction primitives.
This module contains Datum’s Akka-like graph layer: typed public ports, runtime-validated GraphDSL wiring, fused acyclic fast paths, and a queued fused interpreter for cyclic feedback topologies. Invalid wiring and illegal stage operations fail deterministically.
Structs§
- AnyInlet
- Type-erased
Inletcarrying the elementTypeIdand name. - AnyOutlet
- Type-erased
Outletcarrying the elementTypeIdand name. - Async
Boundary - A pass-through marker that splits the graph into separate fused segments
(one inlet, one outlet). Mirrors Akka’s
.asyncboundary. - Async
Boundary Execution Config - Execution settings for the current graph async-boundary benchmark path.
- Async
Callback - A cloneable, thread-safe handle that injects a callback into the stage’s execution context. The callback is serialized with element processing: callbacks are buffered in a lock-protected queue and drained by the fused executor between element deliveries — the callback itself runs on the executor thread, not the invoker’s thread.
- Balance
- Fan out: route each element to one of
outputsoutlets, round-robin by demand (one inlet, N outlets). - Bidi
Shape - Two inlets and two outlets arranged as two independent directions — the
shape of a bidirectional flow (top:
I1 → O1, bottom:I2 → O2). Explicit-only in the wiring DSL. - Broadcast
- Fan out: push each element to all
outputsoutlets (one inlet, N outlets). - Buffer
- A bounded in-stage buffer of
capacityelements, applyingstrategyon overflow. In the synchronous fused executor it drains one element per step so it never overflows (a pass-through on the typed cyclic path); the erased executor runs the full buffering logic. - Concat
- Fan in: fully drain each inlet in declared order before the next (N inlets, one outlet).
- FanIn
Shape - N inlets and one outlet — the shape of merge-style junctions (
Merge,Concat,Interleave,OrElse,MergeSorted, …).Outdefaults toIn;MergeLatestusesOut = Vec<In>. - FanOut
Shape - One inlet and N outlets — the shape of fan-out junctions (
Broadcast,Balance,Partition).Outdefaults toIn. - FanOut
Shape2 - One inlet and two heterogeneously-typed outlets — the shape of
UnzipandUnzipWith. Outlets are reached via.out0()/.out1(). - Flow
Shape - One inlet and one outlet — a graph that behaves as a
Flow. Re-exported asGraphFlowShapefrom the crate root. - Fused
Execution Config - Execution bound for the fused executor.
event_limitcaps the number of push/pull events a single run may take, so an unproductive cycle surfacesStreamError::EventLimitExceededinstead of hanging. Defaults to 100M. - Fused
Execution Report - Result of a collecting run: the
outputvector plus instrumentation (eventsprocessed,async_boundary_crossings). Returned by the*_with_input_reportmethods. - Fused
Segment - A maximal run of stages with no async boundary between them, executed
together by the fused executor. Boundaries (
AsyncBoundarystages) split a graph into consecutive segments. - Fused
Terminal Report - Result of a terminal (count/fold) run: the reduced
resultplus the same instrumentation asFusedExecutionReport. Returned by the*_count_*_report/*_fold_*_reportmethods. - Graph
Blueprint - An immutable, validated graph ready to run. Produced by
GraphDsl::create/try_create. Carries the externalShape, the stages, the wired edges, the precomputed fused segments, and graph-levelAttributes. Therun_*methods are defined in theexecutormodule; running one never mutates the blueprint, so it can be reused and run concurrently. - Graph
Builder - Mutable scratch state for assembling a graph inside a
GraphDslclosure. - Graph
Dsl - Entry points for building graphs.
createtakes a closure returning the shape directly,try_createa closure returningStreamResult<Shape>(so?works onconnect/try_wire), andpartialbuilds a reusablePartialGraphfragment. - Graph
Stage Logic - Per-materialization runtime state and handlers for one stage.
- Identity
- Pass elements through unchanged (one inlet, one outlet). Akka’s
identity. - Inlet
- A typed graph inlet: the downstream end of an edge that accepts
Telements. - Inlet
Cursor - Interleave
- Fan in: round-robin across
inputsinlets in chunks ofsegment_size(N inlets, one outlet).eager_closestops when any inlet completes. - MapStage
- Apply
fto each element, mappingIntoOut(one inlet, one outlet). - Merge
- Fan in: emit from whichever of
inputsinlets has an element available (N inlets, one outlet). Order across inlets is not guaranteed. - Merge
Latest - Fan in: once every inlet has emitted at least once, emit a
Vec<T>snapshot of the latest element per inlet on each push (N inlets, oneVec<T>outlet).eager_completeends the stream as soon as any inlet completes. - Merge
Preferred - Fan in: always drain the single preferred inlet ahead of the
secondary_portssecondaries (1 preferred + N secondary inlets, one outlet). - Merge
Preferred Shape - One preferred inlet, N secondary inlets, and one outlet — the shape of
MergePreferred. Explicit-only in the wiring DSL (use.preferred()/.secondary(i)). - Merge
Prioritized - Fan in: emit on a deterministic weighted schedule — inlet
iis drainedweights[i]times per cycle (N inlets, one outlet). Unlike Akka’sMergePrioritized, the schedule is fixed, not randomized. - Merge
Sequence - Fan in: reassemble elements in gap-free ascending order of the sequence
number
extract_sequencereturns (N inlets, one outlet). Fails on a missing or duplicate sequence number. - Merge
Sorted - Fan in: merge two already-sorted inlets into one sorted outlet (2 inlets, one
outlet). Requires
T: Ord. - OrElse
- Fan in: emit the primary inlet; fall back to the secondary only if the primary completes without emitting (2 inlets, one outlet).
- Outlet
- A typed graph outlet: the upstream end of an edge that emits
Telements. - Outlet
Cursor - Partial
Graph - A reusable graph fragment: a builder closure plus its
Shape, importable into multiple parent graphs viaGraphBuilder::import. Still a blueprint — the closure runs (allocating fresh ports) each time it is imported. Aliased asImportedGraph. - Partition
- Fan out: route each element to the outlet index
partitionerreturns (one inlet, N outlets).eager_cancelcancels the whole stage when any one outlet cancels; otherwise it runs while at least one outlet is live. - Port
Allocator - Hands a
GraphStagefresh, uniquely-identified ports when its shape is allocated. Eachinlet/outletcall draws a new globalPortId; the allocator itself is stateless. - PortId
- A process-global unique identifier for a port, drawn from a single atomic counter. Stages allocate contiguous id blocks; ids are not graph-local and do not start at zero.
- Sink
Shape - One inlet, no outlets — a graph that behaves as a
Sink. - Source
Shape - One outlet, no inlets — a graph that behaves as a
Source. - Stage
Spec - The blueprint description of a stage: its name, erased ports, the internal
kindthe executor switches on, async-boundary flag, and attributes. Construct one withStageSpec::opaquefor a custom stage that runs its ownGraphStageLogic; built-in stages use the crate-internal constructors. - Take
While - Emit elements while
predicateholds, then complete (one inlet, one outlet). - Unzip
- Fan out: split each
(A, B)pair, sendingAto out0 andBto out1 (one inlet, two outlets). Built onUnzipWithwith the identity split. - Unzip
With - Fan out: apply
splitto each element, sending the first component to out0 and the second to out1 (one inlet, two outlets). - Wire
Pair - Zip
- Fan in: pair one element from each inlet into
(Left, Right); completes when either inlet completes (2 inlets, one outlet). - ZipShape
- Two heterogeneously-typed inlets and one
(Left, Right)outlet — the shape ofZip. Inlets are reached via.in0()/.in1().
Enums§
- Port
Kind - Whether a port is an inlet (accepts elements) or an outlet (emits them).
Traits§
- Graph
- Anything exposing a graph
Shape— implemented byGraphBlueprint. - Graph
Stage - A stage blueprint: declares its
Shape, builds itsStageSpec, and (for opaque stages) creates the runtimeGraphStageLogic. Implement this to add a custom graph stage; the built-in junctions implement it too.add-ing a stage to aGraphBuilderallocates its ports and records the spec. - InHandler
- Reacts to events on one inlet. The default
on_upstream_failurefails the whole stage; override any method a stage needs. - OutHandler
- Reacts to events on one outlet. The default
on_downstream_finishcompletes the whole stage; override any method a stage needs. - PortRef
- Common read accessors over a typed port, independent of element type or
direction. Implemented by both
InletandOutlet; used by handler helpers (e.g.set_handler) that only need a port’s id, name, and kind. - Shape
- A bundle of typed external ports. Implementors expose their inlets/outlets
in a stable order;
inlets()/outlets()return the type-erased views the builder validates against. Cheap to clone (ports are id + name). - Timer
Handler - Reacts to a stage timer firing;
keyis the timer’s string key. Installed viaGraphStageLogic::set_timer_handler. - WireDsl
- Shape extension methods for method-based graph wiring.
- Wire
Spec - A graph wiring specification accepted by
GraphBuilder::wireandGraphBuilder::try_wire.