Struct PipelinedExecutor 

pub struct PipelinedExecutor;

Pipelined executor: walks a graph, compiles each node in a dedicated thread, executes the compiled stream on the calling thread.

Architectural commitments (Phase 7.6 step 9c)

Fail-fast dispatch — no runtime CPU fallback

The binding-table lookup is fail-fast. If global_bindings().lookup(op_kind, dtypes, backend) returns None, realize / realize_many surface a typed Error::Msg at compile time. There is no runtime fallback to a CPU sibling when a registered kernel returns Err; we commit to the backend contract that registered kernels must succeed for shapes matching their declared coverage.

Rationale: per-decision-point alternatives (architecture v1.0 §04) already let multiple backends register at the same key. If a backend’s coverage is partial, the right answer is to register only the shapes it actually handles — not to silently fall back at runtime, which masks bugs and slows the hot path. The long-term home for “this op on this device” is graph-level dispatch insertion (Op::Copy edges to a backend that does support the shape), which makes the routing decision visible in the IR rather than hidden in the executor. Picker-arc step 4b landed exactly that: compile_plan’s off-device fallback (PlanOptions::fallback_placements_for) admits a candidate on another device when the pinned device lacks the implementation, and the bridge’s cross-device-copy pass stitches residency around the off-device winner. There is exactly ONE fallback owner — the plan-time picker; this executor still dispatches fail-fast, and ops with no implementation anywhere surface the plan-time NoBackendForOp error.

This is a deliberate departure from the legacy executor’s cpu_fallback(op, inputs, shape, dtype, cache) semantics.

Optimizer separation — callers compose

The pipelined executor does not run fuel_graph::opt or any RuleRegistry pass itself. Callers that want graph-level optimization compose it at the call site:

let optimized_targets = registry.optimize_to_fixpoint(&graph, &targets);
PipelinedExecutor::realize_many(graph, &optimized_targets, inputs)?;

Rationale: the legacy GraphExecutor<B>::with_optimization(bool) coupling reflected its per-typed-entry API surface (realize_f32, etc.) where the convenience of a single optimize+realize call was load-bearing. The pipelined path has a uniform realize_many that composes trivially — no need to re-couple them at the executor layer. This keeps the executor focused on execution and lets the optimizer evolve independently (different rule registries per call site, mid- pipeline opt+realize+opt+realize patterns, etc.).

Implementations

impl PipelinedExecutor

pub fn realize( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

Realize target and every transitive dependency. Compilation runs in a worker thread; execution runs on the calling thread.

Pre-conditions:

• Every reachable Op::Const node must have a corresponding entry in inputs (pre-realized Storage Arc).
• Every reachable non-Const node must have its target_backend set in the graph (Graph::set_target_backend).
• The op + dtype must be registered in global_bindings().

Returns the realized Storage Arc for target plus its resolved Layout. The Layout is contiguous for kernel outputs; for graphs whose target is a metadata-only view op (Op::Transpose, Op::Permute, Op::BroadcastTo), the returned Storage shares its bytes with an upstream node and the Layout encodes the view’s strides + offset.

Production-correct: errors on any unmet precondition rather than panicking.

pub fn realize_with_env( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

Env-carrying sibling of Self::realize: realize target with a per-pass SymEnv supplying the runtime bindings for any DynScalar op params (today: Op::WriteSlice’s dynamic start offset — Phase D symbolic extents). An empty env is byte-identical to Self::realize; the env is consulted only by ops that carry a DynScalar, so a graph with none ignores it. Uses the default (execution_plan) dispatch order.

This is the input-determined path for persistent decode (the per-token cached_len write offset). The OptimizedGraph/route entry points keep an empty env until the realize bridge threads the session env through them (Phase D step 1, fuel-core).

pub fn realize_with_env_producing( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout, SymEnv)>

Producing sibling of Self::realize_with_env: realize target and ALSO return the SymEnv of symbols BOUND BY PRODUCERS during the pass — data-determined dynamic shapes (Op::NonZeroIndices’s runtime count, etc.). The returned env is disjoint from the input sym_env (which is input-determined). Empty when the graph has no data-determined producer. This is the observation point for the increment-2a bind seam; a downstream pass consumes these bindings as dynamic extents.

pub fn realize_with_optimized( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

PR-A3b-1 entry: realize target driven from optimized’s in-place “plan IS the graph” form. The dispatch order is the OptimizedGraph’s extract_runs/lower_run lowering (computed after safety-copy insertion); per-node kernel resolution uses the binding-table-lookup path (no ExecutionPlan, so no route-picking — A3b-1 is branchless). The optimize-time generation drives the TopologyChanged chunk-boundary check.

Pre-conditions match Self::realize: every reachable kernel-bearing node must have target_backend set (the bridge’s stamp_plan_backends does this) and a registered binding.

pub fn realize_with_optimized_env( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

Env-carrying sibling of Self::realize_with_optimized: same optimized-graph dispatch, but with a per-pass SymEnv supplying the runtime bindings for DynScalar op params (Phase D symbolic extents). An empty env is byte-identical to Self::realize_with_optimized. The realize bridge threads its session env through here for persistent decode.

pub fn realize_with_optimized_route( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, route: &PickedRoute, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

PR-C1 entry: realize target driven from optimized’s in-place form following the runtime route picker’s chosen arms. The dispatch order is the route-aware lowering (fuel_graph::lower_picked_route) over route — the per-branch arm the picker (Picker 2) selected by live telemetry. A branch absent from route defaults to arm 0, so an empty route is byte-identical to Self::realize_with_optimized (the no-pressure / no-telemetry contract). A branchless graph has no branches ⇒ the route is empty ⇒ this is exactly the arm-0 path.

pub fn realize_with_optimized_route_env( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, route: &PickedRoute, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

Env-carrying sibling of Self::realize_with_optimized_route — the route-aware lowering with a per-pass SymEnv. An empty env is byte-identical to Self::realize_with_optimized_route.

pub fn realize_with_optimized_picking_env( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, selector: Option<Arc<dyn RuntimeSelector>>, lookup: Option<BackendRuntimeLookup>, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

Cleanup Step C/D entry: the executor OWNS Op::Branch arm-selection. Given the runtime selector + the live device lookup (built bridge-side from the realize Device + Judge oracle and passed in, exactly like the residency input_residency provider), pick one arm per branch HERE — at dispatch — then lower the chosen route. The per-arm candidates are re-enumerated from the runtime binding registry + the graph (Step D: no threaded ExecutionPlan).

No selector (the runtime_selector_disabled() opt-out) or a branchless graph yields the arm-0 lowering, byte-identical to Phase B.

Step E Phase C, PR C1 — STREAMING. A branched graph WITH a selector no longer resolves the whole route up front: it routes to the OrderSource::Streaming walk, where the compiler thread resolves each branch lazily as the frontier reaches it (via resolve_branch). C1 keeps the SAME VRAM-only selector chain — so the streamed route equals the one-shot pick_route route byte-for-byte — and only changes when a branch resolves; C2 makes that moment read live device load.

pub fn realize_with_optimized_picking_cached_env( graph: Arc<RwLock<Graph>>, target: NodeId, inputs: StorageCache, optimized: &OptimizedGraph, order_cache: &OnceLock<Vec<NodeId>>, selector: Option<Arc<dyn RuntimeSelector>>, lookup: Option<BackendRuntimeLookup>, sym_env: SymEnv, ) -> Result<(Arc<RwLock<Storage>>, Layout)>

order_for fix track (docs/design/incremental-consumer-index.md §5) — cached sibling of Self::realize_with_optimized_picking_env for the plan-once persistent-decode caller, which holds order_cache across repeated calls over the SAME graph/roots/optimized (one per DecodeSession, initially empty). On a branchless graph with no selector pressure (the decode shape this is for), dispatches via [OrderSource::Cached] — computed once, reused thereafter, see that variant’s doc for the safety argument. A branched graph WITH a resolved selector pick falls back to the uncached [OrderSource::Streaming] path unchanged (caching doesn’t apply there; this entry point exists for the decode case, not to cover every caller).

pub fn realize_many( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Realize multiple targets in one walk. Each target’s transitive dependency set is collapsed into a single execution plan (via execution_plan); shared subgraphs are evaluated once. Returns a Vec<(Storage, Layout)> whose order matches targets.

This is Phase A of Phase 7.6 step 9c — feature-parity with the legacy GraphExecutor::realize_many_f32. Multi-session migration target ([memory: project_phase_7_6_step_9c_parity_audit.md]).

Pre-conditions: every reachable Op::Const must be in inputs; every reachable non-Const must have its target_backend set; the op + dtype must be registered in global_bindings(). Same as single-target Self::realize.

pub fn realize_many_with_optimized( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, optimized: &OptimizedGraph, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Multi-target PR-A3b-1 entry — the realize_many sibling of Self::realize_with_optimized. Drives the executor from the OptimizedGraph’s run lowering via the binding-table-lookup path, with the optimize-time generation keying the TopologyChanged chunk-boundary check.

pub fn realize_many_with_optimized_env( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, optimized: &OptimizedGraph, sym_env: SymEnv, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Env-carrying sibling of Self::realize_many_with_optimized (Phase D symbolic extents). An empty env is byte-identical to it.

pub fn realize_many_with_optimized_route( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, optimized: &OptimizedGraph, route: &PickedRoute, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Multi-target PR-C1 entry — the realize_many sibling of Self::realize_with_optimized_route. Lowers each target’s runs following the picker’s chosen arms via fuel_graph::lower_picked_route.

pub fn realize_many_with_optimized_route_env( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, optimized: &OptimizedGraph, route: &PickedRoute, sym_env: SymEnv, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Env-carrying sibling of Self::realize_many_with_optimized_route (Phase D symbolic extents). An empty env is byte-identical to it.

pub fn realize_many_with_optimized_picking_env( graph: Arc<RwLock<Graph>>, targets: &[NodeId], inputs: StorageCache, optimized: &OptimizedGraph, selector: Option<Arc<dyn RuntimeSelector>>, lookup: Option<BackendRuntimeLookup>, sym_env: SymEnv, ) -> Result<Vec<(Arc<RwLock<Storage>>, Layout)>>

Multi-target sibling of Self::realize_with_optimized_picking_env — the executor resolves one arm per Op::Branch (cleanup Step C/D) over the effective targets. PR C1: a branched graph routes to the OrderSource::Streaming walk (lazy per-branch resolution at the frontier); branchless / no-selector falls to the untouched arm-0 path.