Module pipelined 

Pipelined compile + execute. Phase 7.5 B4.

PipelinedExecutor::realize runs compilation and execution on separate threads connected by a channel: a compiler thread walks the graph in topological order and emits work items; the executor (this thread) consumes them, allocates output Storage, calls the kernel, and stores the result in an internal cache. Both threads run concurrently so execution can begin while compilation is still resolving later nodes.

Today’s “compile” step is a single binding-table lookup — roughly nanoseconds per node. The threading delivers no measurable speedup in this regime. The pipelining infrastructure exists now — built on the compile_node / crate::compiled::execute_compiled interface from B5 — so future work that grows the compile step (residency-aware planning, transfer-cost minimization, kernel auto-tuning) plugs in without revisiting call sites.

Storage during the migration

fuel_graph::Graph::storage_map uses the legacy fuel_backend_contract::Storage (the Box<dyn DynBackendStorage> newtype). The pipelined executor uses the new fuel_memory::Storage (BackendStorage enum + dtype). During the migration the two coexist — neither is converted on the fly. The pipelined executor takes pre-realized inputs as a HashMap<NodeId, Arc<RwLock<fuel_memory::Storage>>> rather than reading from the graph’s storage_map. Phase D unifies the two paths once kernel migration completes.

Op coverage

B4 supports Op::Const (input-cache adoption — no kernel call) and Op::Add on f32 (mapped to OpKind::AddElementwise). Phase C adds the rest as more (op, dtype) bindings register.

Structs

PersistentOutputs

Persistent per-node output buffers for CUDA-graph capture (the “CapturedRun executor” build-out). Maps each compute node’s NodeId to a FIXED-ADDRESS output Storage Arc that survives across realizes, so a captured graph’s baked kernel-argument addresses stay valid on replay.

PipelinedExecutor

Pipelined executor: walks a graph, compiles each node in a dedicated thread, executes the compiled stream on the calling thread.

Type Aliases

StorageCache

Map from NodeId to a realized Storage Arc. Used both as the input cache (passed in by the caller for pre-realized leaves) and as the output cache (built up during execution).