Struct NodeHandle 

pub struct NodeHandle { /* private fields */ }

A handle to a node in a shared graph. Cheap to clone. Carries the graph reference so builder methods have everything they need to append a new node.

Implementations

impl NodeHandle

pub fn from_existing(graph: SharedGraph, id: NodeId) -> Self

Wrap an existing (graph, node_id) pair as a NodeHandle handle. Used by graph-rewriting passes (e.g., opt::insert_copies) that produce new root NodeIds and need to hand them back to call sites as Tensors.

pub fn id(&self) -> NodeId

The node ID this handle points to.

pub fn graph(&self) -> &SharedGraph

The shared graph this handle belongs to. Exposed so executors and tests can walk the underlying arena.

pub fn graph_id(&self) -> GraphId

This tensor’s graph identity — see GraphId.

Two tensors can be combined iff their graph_ids match. Affinity is still checked by pointer identity (Arc::ptr_eq, which cannot alias); this is the human-readable form, so a failure can say which graphs are involved rather than only that they differ.

pub fn shape(&self) -> Shape

The shape of this tensor, read from the underlying node.

pub fn dtype(&self) -> DType

The dtype of this tensor, read from the underlying node.

pub fn storage_for(&self) -> Option<Arc<RwLock<Storage>>>

Phase 7.5 work item G: look up the realized-storage slot for this tensor’s node, if any. Returns an Arc clone so the slot stays in the map after this call and the caller’s Arc keeps the bytes alive even if a later eviction removes the map entry. None means “not realized yet” — the caller should realize the graph first.

pub fn placement(&self) -> Option<DeviceLocation>

The placement hint for this tensor’s node, if one was set. None means “inherit from the executor’s default device.”

pub fn on_device(self, loc: DeviceLocation) -> Self

Tag this tensor’s node with a target device. The executor will validate (post-Phase-1) that the node’s op can be evaluated on that device. Returns the same NodeHandle handle so this composes with builder-style graph construction:

let y = a.matmul(&b).on_device(DeviceLocation::Vulkan { gpu_id: 0 });

pub fn release(&self) -> Self

Release this tensor’s device-resident storage. Appends an Op::Release node whose output is a zero-element marker (shape [0], same dtype as self). Destructive on self — after this op runs, self’s storage is gone.

The ordering-analysis pass (opt::derive_ordering, arriving in a follow-up PR) automatically pins the Release to run after every non-destructive reader of self. Callers should NOT invoke this directly in a graph that has other readers of self until that pass is in place — today’s executor has no mechanism to stop them running in the wrong order.

pub fn move_to_device(&self, target: DeviceLocation) -> Self

Move this tensor’s data to target device — destroying the source in the process. Appends an Op::Move node. The output has the same shape and dtype as the input and lives on target; the source’s device storage is freed once the op runs.

Semantically equivalent to copy_to_device(target) followed by dropping the source. Use this when you KNOW the source won’t be needed after the transfer — it lets backends skip the intermediate alloc in fast-path cases and lets the scheduler free the source’s device memory immediately.

If the source may still be read by other ops, use Self::copy_to_device instead — Copy leaves the source resident.

The ordering-analysis pass (opt::derive_ordering) automatically pins this op to run after every non-destructive reader of the source. The caller does NOT need to worry about topo ordering — just emit the Move wherever the data-flow says it should go.

pub fn copy_to_device(&self, target: DeviceLocation) -> Self

Copy this tensor’s data to target device. Source stays resident. Appends an Op::Copy node. The output has the same shape and dtype as the input; only residency changes.

Same-device copies are still a legal thing to ask for — the backend can optimize them into a cheap clone (or the scheduler can elide the Copy node entirely). Cross-device copies resolve through the pipelined executor’s Copy arm (kernel lookup at (OpKind::Copy, [dt, dt], source backend)) — today a host round-trip; P2P comes later.

pub fn view(&self, slot: u32) -> Result<NodeHandle, Error>

Project one logical output out of a multi-output producer node.

self must have been declared as a multi-output producer via Graph::set_output_views (Session 2’s authoring contract will drive this for real fused ops; Session 1 tests drive it directly). The View’s output shape, dtype, and layout come from output_views(self.id)[slot]. Zero bytes are moved at realization time — the View shares the producer’s bundled Arc<RwLock<Storage>> and exposes the slot’s typed window.

Returns Result: the producer must have declared its slot specs; slot must be in range. Errors at graph-build time (per the validate-at-graph-build-time rule); the executor never sees a malformed View node.

pub fn view_owned(&self, slot: u32) -> Result<NodeHandle, Error>

Owned variant of Self::view — the slot’s bytes are copied into a fresh contiguous Storage at realization time, so the producer’s bundle Arc can drop independently. Output layout is always contiguous.

Same validation as view; same error surface.

pub fn write_slice( &self, source: &NodeHandle, ranges: Vec<(usize, usize)>, ) -> Result<NodeHandle, Error>

Append an Op::WriteSlice node — copies source’s bytes into self at the rectangular slab defined by ranges. Per-axis, ranges[i] = (start, end) is the half-open destination range; source.dims()[i] must equal end - start, and end must not exceed self.dims()[i]. Dtypes must match.

Destructive on self: after the write, self’s NodeId is no longer readable (the scheduler pins this op to run after every other reader of self via opt::derive_ordering). The returned tensor adopts self’s Storage and Layout; downstream consumers read post-write bytes through this NodeId.

Phase E.3.2: introduced to back persistent KV-cache writes (InferenceContext + KvCache). Non-differentiable.

Returns Result: rank/shape/range mismatches surface as a typed error.

pub fn write_slice_dyn( &self, source: &NodeHandle, ranges: Vec<(usize, usize)>, dyn_axis: usize, offset: DynScalar, ) -> Result<NodeHandle, Error>

Append an Op::WriteSlice whose start on dyn_axis is a runtime value resolved through the per-pass fuel_ir::SymEnv at realize, rather than the build-time ranges[dyn_axis].0.

This is the input-determined dynamic-offset path (a DynScalar over the one SymEnv), distinct from Self::write_slice_rotating’s data-determined position (a tensor input read mid-pass). It backs the persistent decode KV-cache write: the append offset is cached_len, a per-token runtime value over a fixed-capacity buffer, so the graph structure (and every shape) stays identical across tokens and the plan is reused (Phase D symbolic extents).

On dyn_axis, ranges[dyn_axis].0 is ignored (the start is dynamic) and ranges[dyn_axis].1 - ranges[dyn_axis].0 is the slab width, which must equal source.dims()[dyn_axis] and not exceed the destination capacity on that axis. On every other axis the same shape contract as Self::write_slice applies. The runtime offset + width is bounds-checked against the destination capacity at realize (a typed error, never a panic).

Returns Result: rank/dtype/axis-bound/width mismatches surface as a typed error at build time.

pub fn write_slice_rotating( &self, source: &NodeHandle, position: &NodeHandle, axis: usize, modulus: usize, ranges: Vec<(usize, usize)>, ) -> Result<NodeHandle, Error>

Append an Op::WriteSliceRotating node — copies source’s bytes into self at the rectangular slab defined by ranges, with the axis axis wrapping modulo modulus. The dynamic write position comes through position (a rank-0 U32 tensor); the kernel computes start = position % modulus and splits the write across the ring boundary if needed.

ranges[axis].0 is ignored (the rotating-axis start is dynamic). ranges[axis].1 - ranges[axis].0 is the write length on the rotating axis and must equal source.dims()[axis]. On every other axis the same shape contract as write_slice applies.

Destructive on self; scheduled like write_slice. Backward panics — sliding-window KV-cache writes are forward-only.

Returns Result: rank/dtype/axis-bound/modulus/range mismatches surface as a typed error at build time.

pub fn write_slice_doff( &self, source: &NodeHandle, offset: &NodeHandle, axis: usize, ranges: Vec<(usize, usize)>, ) -> Result<NodeHandle, Error>

Append an Op::WriteSliceDoff node — copies source’s bytes into self at the rectangular slab defined by ranges, with the start on axis read from offset (a rank-0 I64 tensor) device-side at kernel launch. No modulo wrap: offset is the raw start on axis.

ranges[axis].0 is ignored (the dynamic-axis start is device- resident). ranges[axis].1 - ranges[axis].0 is the write width on axis and must equal source.dims()[axis]. On every other axis the same shape contract as write_slice applies.

Bounds are the caller’s contract: offset is device-only at build time, so offset + width <= dest_dims[axis] cannot be checked here and the kernel does NOT clamp. The caller must size self’s capacity on axis to cover the largest offset (this is the CUDA-graph-capturable KV-cache append; DecodeSession sizes max_seq). The static slab width IS checked against capacity.

Destructive on self; scheduled like write_slice. Backward panics — decode-time KV-cache appends are forward-only.

Returns Result: rank / dtype / axis-bound / static-width / offset-dtype mismatches surface as a typed error at build time.

pub fn from_f32( data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build a Const tensor from an f32 slice and shape on a fresh graph. The new graph is returned via the tensor’s graph handle.

data takes impl Into<Arc<[f32]>> so both Vec<f32> (one-time conversion at the call site) and Arc<[f32]> (free clone that shares buffers across forward passes) work.

Phase 7.5 work item G2: device is the device on which the realized Storage is allocated. The graph’s storage_map slot is populated with that Storage and Op::Const is emitted — no host-side ConstData payload rides on the node.

pub fn from_f64( data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build a Const tensor from an f64 slice and shape on a fresh graph. device selects where the realized Storage is allocated.

pub fn from_bf16( data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build a Const tensor from a bf16 slice and shape on a fresh graph. device selects where the realized Storage is allocated.

pub fn from_f16( data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build a Const tensor from an f16 slice and shape on a fresh graph. device selects where the realized Storage is allocated.

pub fn from_u32( data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build a Const tensor from a u32 slice and shape on a fresh graph. Primarily used to construct index tensors for gather / scatter / index_select. u32 is the index type all real fuel backends use (Candle CPU, CUDA, Metal), so keeping the reference on the same type means oracle-equivalence tests do not need any index-type translation. device selects where the realized Storage is allocated.

pub fn from_f32_on( graph: &SharedGraph, data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

Build an f32 Const leaf on the graph you pass in, rather than on a fresh one.

This is the non-anchor route for adding a tensor to an existing graph. NodeHandle::from_f32 mints a NEW graph, so two from_* tensors can never be combined; from_f32_on puts the new leaf on graph, and const_f32_like does the same thing when what you have to hand is a sibling tensor rather than the graph itself.

let g: SharedGraph = Arc::new(RwLock::new(Graph::new()));
let a = NodeHandle::from_f32_on(&g, a_data, [2, 3], &device);
let w = NodeHandle::from_f32_on(&g, w_data, [3, 2], &device);
let y = a.matmul(&w);   // same graph ✓

pub fn from_f64_on( graph: &SharedGraph, data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

f64 sibling of from_f32_on — builds on the given graph rather than a fresh one.

pub fn from_bf16_on( graph: &SharedGraph, data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

bf16 sibling of from_f32_on. The common mixed-precision shape — f32 activations, bf16 weights — is exactly this plus from_f32_on against one shared graph.

pub fn from_f16_on( graph: &SharedGraph, data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

f16 sibling of from_f32_on.

pub fn from_u32_on( graph: &SharedGraph, data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, device: &Arc<dyn DynBackendDevice>, ) -> Result<Self, Error>

u32 sibling of from_f32_on — index tensors for gather/index_select built straight onto the graph they will be used on.

pub fn from_storage( storage: Arc<RwLock<Storage>>, shape: impl Into<Shape>, dtype: DType, ) -> Self

Phase 7.5 work item G2: build a Const leaf on a fresh graph whose realized bytes already live in storage. The graph’s storage_map slot for the new node is populated with storage directly. The executor’s slot-first dispatch returns the slot’s Arc on realize.

pub fn const_f32_like( &self, data: impl Into<Arc<[f32]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second Const tensor that lives on the same graph as self. Use this to add more inputs to an existing computation.

Phase 7.5 G2: the realized Storage is allocated on the device derived from self’s graph (any existing slot’s device — the graph always has at least one slot-bearing leaf by the time const_*_like is called). For cross-device const construction, build a fresh graph with NodeHandle::from_f32 and link via Op::Move / Op::Copy.

pub fn const_f64_like( &self, data: impl Into<Arc<[f64]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second Const f64 tensor on the same graph as self.

pub fn const_bf16_like( &self, data: impl Into<Arc<[bf16]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second Const bf16 tensor on the same graph as self.

pub fn const_f16_like( &self, data: impl Into<Arc<[f16]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second Const f16 tensor on the same graph as self.

pub fn const_u32_like( &self, data: impl Into<Arc<[u32]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a second Const u32 (index) tensor on the same graph as self.

pub fn const_u8_like( &self, data: impl Into<Arc<[u8]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a sibling U8 Const on the same graph. Used by byte-stream inputs (e.g. NF4 quantized weights packed two codes per byte).

pub fn const_i64_like( &self, data: impl Into<Arc<[i64]>>, shape: impl Into<Shape>, ) -> Result<Self, Error>

Build a sibling I64 Const on the same graph. Used by integer-target ops (e.g. cross-entropy with class indices in PyTorch convention).

pub fn const_placeholder_like( &self, shape: impl Into<Shape>, dtype: DType, ) -> Self

Push a Const node on the same graph as self without populating the graph’s storage_map. The caller is responsible for binding the storage Arc into the realize call’s initial StorageCache, typically via InferenceContext::insert.

Used by the Phase 7.6 step 9c E.3.3 forward path to bind pre-allocated KV-cache storage Arcs (held as the new Arc<RwLock<fuel_memory::Storage>> type, not the legacy fuel_backend_contract::Storage that const_like_from_storage takes) into a per-step graph without re-uploading or type-converting.

Caller contract: the same NodeId must appear in the initial StorageCache passed to the realize call. If not, the executor surfaces a clean error pointing at the missing slot — there’s no implicit fallback to graph.storage_map.

pub fn const_like_from_storage( &self, storage: Arc<RwLock<Storage>>, shape: impl Into<Shape>, dtype: DType, ) -> Self

Phase 7.5 work item G2: build a second Const leaf on the same graph as self whose realized bytes already live in storage. Companion to NodeHandle::from_storage for the multi-input case.

pub fn add(&self, other: &NodeHandle) -> NodeHandle

Append an Add node self + other to the shared graph and return a handle to the result. Requires matching shapes and matching graphs.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn mul(&self, other: &NodeHandle) -> NodeHandle

Append a Mul node self * other.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn matmul(&self, other: &NodeHandle) -> NodeHandle

Append a MatMul node. Both operands must have rank ≥ 2. The last two dims are the matrix dims; leading dims are batch dims.

• If both operands have the same rank, their batch dims must match exactly.
• If one operand is rank-2 and the other is higher rank, the rank-2 operand is auto-broadcast to match the higher-rank operand’s batch prefix. This is the common “linear layer across a batch” pattern ([batch, seq, d_model] @ [d_model, d_out]).
• Any other rank combination panics; users must reshape first.

Shape examples:

• [m, k] @ [k, n] → [m, n]
• [batch, m, k] @ [batch, k, n] → [batch, m, n]
• [batch, seq, k] @ [k, n] → [batch, seq, n] (rhs auto-broadcast)
• [k, n] @ [batch, n, m] → [batch, k, m] (lhs auto-broadcast)

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn matmul_dyn_m( &self, other: &NodeHandle, row_count: DynScalar, ) -> NodeHandle

Data-determined-M matmul: like Self::matmul, but the LHS row count M is a runtime value row_count rather than the static lhs.shape[-2]. The LHS and output are capacity buffers of lhs.shape[-2] rows; only row_count rows are computed, the rest left zeroed. This is the sparse-MoE expert-FFN primitive — gather an expert’s count_e routed tokens into a capacity buffer, then compute exactly count_e FFN rows (the FLOP saving). row_count resolves at compile if input-determined, else at execute from the producer-bound SymEnv (e.g. Op::NonZeroIndices’s count). The output shape is the capacity shape [..., lhs.shape[-2], n].

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn qmatmul( &self, weight_bytes: &NodeHandle, quant_type: QuantType, k: usize, n: usize, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::QMatMul that multiplies self (F32 activations, shape [..., M, K]) with a Q-type quantized weight matrix of logical shape [N, K] stored as a raw byte stream (passed in as a U32 tensor; length = n_bytes / 4).

Output shape: [..., M, N] F32. The backend dequantizes the weight blocks on the fly inside the matmul kernel, so the quantized blocks stay resident at their compressed size.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn conv2d( &self, weight: &NodeHandle, bias: Option<&NodeHandle>, stride: (usize, usize), padding: (usize, usize), groups: usize, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::Conv2D. self must be [N, Cin, H, W] (rank 4); weight must be [Cout, Cin/groups, Kh, Kw] (rank 4) and live on the same graph. bias is optional — when present it must be rank 1 with length Cout. Returns a rank-4 tensor of shape [N, Cout, Hout, Wout].

Panics if the input ranks don’t match, the channel counts are inconsistent with groups, or the output spatial dims would be non-positive.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn paged_attn( &self, k_cache: &NodeHandle, v_cache: &NodeHandle, block_table: &NodeHandle, context_lens: &NodeHandle, alibi_slopes: Option<&NodeHandle>, softmax_scale: f32, block_size: usize, softcap: Option<f32>, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::PagedAttn. self is the Q tensor of shape [B, Hq, Sq, D]; k_cache and v_cache are the paged caches shaped [num_blocks, block_size, Hkv, D]; block_table is [B, max_num_blocks_per_seq] (u32) mapping logical → physical blocks; context_lens is [B] (u32) of per-sequence lengths; alibi_slopes is the optional [Hq] per-head bias.

Returns a tensor with q’s shape [B, Hq, Sq, D].

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn conv_transpose2d( &self, weight: &NodeHandle, stride: (usize, usize), padding: (usize, usize), output_padding: (usize, usize), dilation: (usize, usize), groups: usize, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::ConvTranspose2D. self must be [N, Cin, H, W]; weight must be [Cin, Cout/groups, Kh, Kw] (note transposed channel order vs conv2d). Returns a rank-4 tensor [N, Cout, Hout, Wout].

Panics if ranks don’t match, channel counts are inconsistent with groups, or output spatial dims would be non-positive.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn conv_transpose1d( &self, weight: &NodeHandle, stride: usize, padding: usize, output_padding: usize, dilation: usize, groups: usize, ) -> NodeHandle

Append a transposed 1D convolution by lifting self and weight into the rank-4 path and dispatching through Self::conv_transpose2d. self is [N, Cin, Lin]; weight is [Cin, Cout/groups, K] (transposed channel order, matches PyTorch). Returns [N, Cout, Lout] where Lout = (Lin − 1)·stride − 2·pad + dil·(K − 1) + out_pad + 1.

Used by audio codec models (DAC / EnCodec / SNAC / Mimi / Parler-TTS / MetaVoice / CSM) where the decoder upsamples quantized latents back to waveform with strided transposed convs.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn flash_attn( &self, k: &NodeHandle, v: &NodeHandle, alibi_slopes: Option<&NodeHandle>, softmax_scale: f32, causal: bool, window_size_left: Option<usize>, window_size_right: Option<usize>, softcap: Option<f32>, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::FlashAttn. self is q of shape [B, Hq, Sq, D]; k and v are [B, Hkv, Sk, D] with Hq a multiple of Hkv (GQA). alibi_slopes (optional) is [Hq]. Returns a tensor with q’s shape. Build the flash_attn node.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn flash_attn_dyn( &self, k: &NodeHandle, v: &NodeHandle, alibi_slopes: Option<&NodeHandle>, softmax_scale: f32, causal: bool, window_size_left: Option<usize>, window_size_right: Option<usize>, softcap: Option<f32>, k_len: DynScalar, ) -> NodeHandle

Append an Op::Fused node carrying crate::registry::FusedOpParams::FlashAttn over a fixed-capacity K/V whose attended length is a runtime value (k_len) resolved through the per-pass fuel_ir::SymEnv at realize, decoupled from K’s allocated shape.

self is q of shape [B, Hq, Sq, D]; k/v are the capacity buffers [B, Hkv, max_seq, D] (GQA: Hq % Hkv == 0). Only the first k_len.resolve(env) rows along the K/V length axis are attended; the causal mask (when causal) is bottom-right-aligned at offset k_len - Sq — the standard FlashAttention-2 convention, equal to cached_len in autoregressive decode. Returns a tensor with q’s shape.

This is the flash counterpart of Self::write_slice_dyn: the decode KV-cache write appends at the runtime offset and flash attends the runtime prefix, so the graph structure stays identical across tokens (Phase D symbolic extents).

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn relu(&self) -> NodeHandle

Append a Relu node max(0, self).

pub fn sqr(&self) -> NodeHandle

Append a Sqr node self * self.

pub fn exp(&self) -> NodeHandle

Append an Exp node e^self.

pub fn permute(&self, axes: &[usize]) -> NodeHandle

Append a Permute node rearranging the axes according to axes. out.shape[i] = self.shape[axes[i]]. The axes vector must be a permutation of 0..rank. For a rank-3 input with axes = [2, 0, 1], the shape transform is [a, b, c] → [c, a, b].

pub fn try_permute(&self, axes: &[usize]) -> Result<NodeHandle, Error>

Result-returning sibling of Self::permute. Surfaces bad axes (wrong length, out-of-bounds entry, or duplicate) as a typed error rather than panicking.

pub fn transpose(&self) -> NodeHandle

Append a Transpose node that swaps the last two dimensions of a tensor of rank ≥ 2. Leading dims are unchanged. For a rank-2 tensor this is the ordinary matrix transpose; for higher ranks, every batch slice is transposed independently.

pub fn try_transpose(&self) -> Result<NodeHandle, Error>

Result-returning sibling of Self::transpose. Surfaces rank < 2 as a typed error rather than panicking.

pub fn sub(&self, other: &NodeHandle) -> NodeHandle

Append a Sub node self - other. Requires matching shapes and dtypes.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn div(&self, other: &NodeHandle) -> NodeHandle

Append a Div node self / other. Requires matching shapes and dtypes.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn neg(&self) -> NodeHandle

Append a Neg node -self.

pub fn sqrt(&self) -> NodeHandle

Append a Sqrt node sqrt(self).

pub fn log(&self) -> NodeHandle

Append a Log node ln(self).

pub fn sin(&self) -> NodeHandle

Append a Sin node.

pub fn cos(&self) -> NodeHandle

Append a Cos node.

pub fn tanh(&self) -> NodeHandle

Append a Tanh node.

pub fn sigmoid(&self) -> NodeHandle

Append a Sigmoid node.

pub fn silu(&self) -> NodeHandle

Append a Silu node (SiLU/Swish activation: x · sigmoid(x)).

pub fn gelu_tanh(&self) -> NodeHandle

Append a GeluTanh node (tanh-approximation GELU).

pub fn step(&self) -> NodeHandle

Append a Step node (Heaviside step: 1 where x > 0, else 0).

pub fn recip(&self) -> NodeHandle

Append a Recip node (1 / self).

pub fn abs(&self) -> NodeHandle

Append an Abs node (|self|).

pub fn relu_inplace(&self) -> NodeHandle

Append a ReluInplace node — mutates self’s storage with max(0, self). See NodeHandle::relu for the functional variant.

pub fn silu_inplace(&self) -> NodeHandle

Append a SiluInplace node — mutates self’s storage with self * sigmoid(self). See NodeHandle::silu for the functional variant.

pub fn gelu_tanh_inplace(&self) -> NodeHandle

Append a GeluTanhInplace node — mutates self’s storage with the tanh-approximation GELU. See NodeHandle::gelu_tanh for the functional variant.

pub fn tanh_inplace(&self) -> NodeHandle

Append a TanhInplace node — mutates self’s storage with tanh(self). See NodeHandle::tanh for the functional variant.

pub fn sigmoid_inplace(&self) -> NodeHandle

Append a SigmoidInplace node — mutates self’s storage with sigmoid(self). See NodeHandle::sigmoid for the functional variant.

pub fn neg_inplace(&self) -> NodeHandle

Append a NegInplace node — mutates self’s storage with -self. See NodeHandle::neg for the functional variant.

pub fn abs_inplace(&self) -> NodeHandle

Append an AbsInplace node — mutates self’s storage with |self|. See NodeHandle::abs for the functional variant.

pub fn sqr_inplace(&self) -> NodeHandle

Append a SqrInplace node — mutates self’s storage with self². See NodeHandle::sqr for the functional variant.

pub fn sqrt_inplace(&self) -> NodeHandle

Append a SqrtInplace node — mutates self’s storage with √self. See NodeHandle::sqrt for the functional variant.

pub fn rsqrt_inplace(&self) -> NodeHandle

Append a RsqrtInplace node — mutates self’s storage with 1/√self. See NodeHandle::rsqrt for the functional variant.

pub fn recip_inplace(&self) -> NodeHandle

Append a RecipInplace node — mutates self’s storage with 1/self. See NodeHandle::recip for the functional variant.

pub fn exp_inplace(&self) -> NodeHandle

Append an ExpInplace node — mutates self’s storage with exp(self). See NodeHandle::exp for the functional variant.

pub fn log_inplace(&self) -> NodeHandle

Append a LogInplace node — mutates self’s storage with ln(self). See NodeHandle::log for the functional variant.

pub fn sin_inplace(&self) -> NodeHandle

Append a SinInplace node — mutates self’s storage with sin(self). See NodeHandle::sin for the functional variant.

pub fn cos_inplace(&self) -> NodeHandle

Append a CosInplace node — mutates self’s storage with cos(self). See NodeHandle::cos for the functional variant.

pub fn sign_inplace(&self) -> NodeHandle

Append a SignInplace node — mutates self’s storage with sign(self). See NodeHandle::sign for the functional variant.

pub fn floor_inplace(&self) -> NodeHandle

Append a FloorInplace node — mutates self’s storage with ⌊self⌋. See NodeHandle::floor for the functional variant.

pub fn ceil_inplace(&self) -> NodeHandle

Append a CeilInplace node — mutates self’s storage with ⌈self⌉. See NodeHandle::ceil for the functional variant.

pub fn round_inplace(&self) -> NodeHandle

Append a RoundInplace node — mutates self’s storage with round(self). See NodeHandle::round for the functional variant.

pub fn erf_inplace(&self) -> NodeHandle

Append an ErfInplace node — mutates self’s storage with erf(self). See NodeHandle::erf for the functional variant.

pub fn gelu_inplace(&self) -> NodeHandle

Append a GeluInplace node — mutates self’s storage with the exact-GeLU formula. See NodeHandle::gelu for the functional variant.

pub fn clamp_inplace(&self, min: f64, max: f64) -> NodeHandle

Append a ClampInplace { min, max } node — mutates self’s storage with clamp(self, min, max). See NodeHandle::clamp for the functional variant.

pub fn powi_inplace(&self, exp: i32) -> NodeHandle

Append a PowIInplace(exp) node — mutates self’s storage with self.powi(exp). See NodeHandle::powi for the functional variant.

pub fn affine_inplace(&self, mul: f64, add: f64) -> NodeHandle

Append an Op::Fused(INPLACE_AFFINE, ...) node — mutates self’s storage with self = mul · self + add. Single-input fused op (no functional Op::Affine exists on fuel-graph::Op; the non-inplace equivalent is self .mul_scalar(mul).add_scalar(add)). Wired to baracuda’s affine_inplace_* symbol on CUDA when Phase 3 lands.

pub fn floor(&self) -> NodeHandle

Append a Floor node (⌊self⌋). Output dtype = input dtype. Backward is the zero distribution almost everywhere; gradient is dropped silently.

pub fn ceil(&self) -> NodeHandle

Append a Ceil node (⌈self⌉). Output dtype = input dtype. Backward drops gradient (non-differentiable almost everywhere).

pub fn round(&self) -> NodeHandle

Append a Round node — banker’s rounding (round-half-to-even, IEEE 754 roundeven). 0.5 → 0, 1.5 → 2, 2.5 → 2, 3.5 → 4. Output dtype = input dtype. Backward drops gradient.

pub fn sign(&self) -> NodeHandle

Append a Sign node (-1 / 0 / 1). sign(0) = 0 by subgradient convention. Output dtype = input dtype. Backward drops gradient.

pub fn erf(&self) -> NodeHandle

Append an Erf node (Gauss error function). Output dtype = input dtype. Differentiable.

pub fn gelu(&self) -> NodeHandle

Append a Gelu node — exact-erf formulation of GELU (0.5 * x * (1 + erf(x/√2))). Distinct from Self::gelu_tanh (tanh approximation). Output dtype = input dtype. Differentiable.

pub fn pow(&self, other: &NodeHandle) -> Result<NodeHandle, Error>

Append a Pow node pow(self, other) element-wise (real exponent). Both operands must share dtype and shape. Distinct from Self::powi (scalar i32 exponent). NaN follows IEEE-754 (pow(-2, 0.5) = NaN).

Returns Result: dtype/shape mismatch surfaces as a typed error, not a panic.

pub fn rsqrt(&self) -> NodeHandle

Append an Rsqrt node (1 / sqrt(self)). Same dtype as input. Single op rather than sqrt(x).recip() — one kernel launch and matches RMSNorm’s x * rsqrt(...) shape. Differentiable.

pub fn rem(&self, other: &NodeHandle) -> Result<NodeHandle, Error>

Append a Rem node self % other element-wise (PyTorch convention: a - floor(a/b) * b, sign of result matches divisor). Both operands must share dtype and shape. Differentiable.

Returns Result: dtype/shape mismatch surfaces as a typed error, not a panic.

pub fn flip(&self, dim: usize) -> Result<NodeHandle, Error>

Append a Flip node — reverses element order along dim. Output shape == input shape. Materializing op (real byte shuffle; not a metadata-only view). Differentiable (involutive: backward is another Flip on the same dim).

Returns Result: bad dim surfaces as a typed error.

pub fn roll(&self, dim: usize, shift: i64) -> Result<NodeHandle, Error>

Append a Roll node — cyclic shift along dim by shift positions. Positive shift moves elements to higher indices (wrapping); negative the opposite. Output shape == input shape. Differentiable (backward is Roll { dim, -shift }).

Returns Result: bad dim surfaces as a typed error.

pub fn cumsum(&self, dim: usize) -> Result<NodeHandle, Error>

Append a CumSum node — running cumulative sum along dim. Output shape == input shape. Differentiable; backward is reverse-cumsum (Flip → CumSum → Flip).

Returns Result: bad dim surfaces as a typed error.

pub fn triu(&self, diagonal: i64) -> Result<NodeHandle, Error>

Append a Triu node — upper-triangular mask along the last two dims. diagonal = 0 keeps the main diagonal and above; positive values shift the cutoff up (keeping less); negative shift down (keeping more, including subdiagonals).

Returns Result: rank < 2 surfaces as a typed error.

pub fn scan( &self, xs: &[NodeHandle], consts: &[NodeHandle], body_new_carry: &NodeHandle, body_y: &NodeHandle, bound: usize, emit: ScanEmit, ) -> Result<NodeHandle, Error>

Append an Op::Scan node — a bounded lax.scan recurrence. self is the init_carry. The body must already be built in the same graph, referencing Op::ScanPlaceholder{Carry,0} for the per-step carry and Op::ScanPlaceholder{Elem,i} for the per-step slice of xs[i]; consts are referenced by real NodeId. body_new_carry / body_y are the body’s two exit nodes. Always a 2-slot bundle (slot 0 = stacked ys [bound] ++ body_y, slot 1 = final carry); emit selects which slot this call projects. early_exit is Phase 2 (always None here).

Returns Result: a malformed body/carry/bound surfaces as a typed error — never a panic.

pub fn scan_until( &self, xs: &[NodeHandle], consts: &[NodeHandle], body_new_carry: &NodeHandle, body_y: &NodeHandle, pred_exit: &NodeHandle, bound: usize, emit: ScanEmit, ) -> Result<NodeHandle, Error>

Append an early-exit Op::Scan node — the Result-returning sibling of NodeHandle::scan that carries a convergence predicate as the extra trailing input (early_exit = Some(ScanPredicate)). self is the init_carry; pred_exit is a scalar-U8 sub-DAG over the carry (Op::ScanPlaceholder{Carry,0} for the pre-step carry, and the shared body_new_carry node for the post-step carry) that stops iteration when it fires. The runtime stop count is data-dependent under the static capacity bound.

consts must include every const referenced by the body OR the predicate — the clone/unroll machinery shares only consts; a predicate const outside consts would be re-cloned dataless and fail at realize.

Returns Result: a malformed body/carry/bound, or a non-scalar / non-U8 / Elem-referencing predicate, surfaces as a typed error — never a panic (validated at graph-build time).

pub fn tril(&self, diagonal: i64) -> Result<NodeHandle, Error>

Append a Tril node — lower-triangular mask along the last two dims. diagonal = 0 keeps the main diagonal and below. tril(diagonal = 0) is the canonical causal-attention mask.

Returns Result: rank < 2 surfaces as a typed error.

pub fn log_softmax_last_dim(&self) -> Result<NodeHandle, Error>

Append a LogSoftmaxLastDim node. Output shape == input shape. Numerically stable (max-subtracting formula) — preferred over log(softmax(x)) for NLL / cross-entropy loss.

Returns Result: rank < 1 surfaces as a typed error.

pub fn masked_fill( &self, mask: &NodeHandle, value: Scalar, ) -> Result<NodeHandle, Error>

Append a MaskedFill node. mask must be Bool (GAP-168(c)) and have the same shape as self. Every position where mask is true gets value; every position where mask == 0 passes self through. value is a scalar of self’s dtype.

Returns Result: shape or dtype mismatches surface as typed errors.

pub fn pad( &self, padding: Vec<(usize, usize)>, mode: PadMode, value: f64, ) -> Result<NodeHandle, Error>

Append a Pad node — multi-dim padding. padding[i] = (before, after) for axis i; padding.len() must equal self.rank(). Output shape: out[i] = in[i] + padding[i].0 + padding[i].1. value is the fill constant for PadMode::Constant (ignored for other modes).

Only Constant mode is wired through the executor in the v1 cut; the other modes produce a clean error at realize time. Differentiable for Constant (backward slices the gradient along each padded axis to restore the input shape).

Returns Result: rank mismatch surfaces as a typed error.

pub fn eq(&self, other: &NodeHandle) -> NodeHandle

Append an Equal node (self == other) producing a Bool mask. Both operands must share dtype and shape; output dtype is Bool (1 where equal, 0 otherwise). NaN follows IEEE-754 (NaN == NaN is false). Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn ne(&self, other: &NodeHandle) -> NodeHandle

Append a Ne node (self != other) producing a Bool mask. Same shape/dtype contract as Self::eq. NaN follows IEEE-754 (NaN != NaN is true → 1). Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn lt(&self, other: &NodeHandle) -> NodeHandle

Append an Lt node (self < other) producing a Bool mask. Same shape/dtype contract as Self::eq. NaN-on-either-side is always 0 (IEEE-754 unordered). Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn le(&self, other: &NodeHandle) -> NodeHandle

Append an Le node (self <= other) producing a Bool mask. Same shape/dtype contract as Self::eq. NaN-on-either-side is always 0. Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn gt(&self, other: &NodeHandle) -> NodeHandle

Append a Gt node (self > other) producing a Bool mask. Same shape/dtype contract as Self::eq. NaN-on-either-side is always 0. Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn ge(&self, other: &NodeHandle) -> NodeHandle

Append a Ge node (self >= other) producing a Bool mask. Same shape/dtype contract as Self::eq. NaN-on-either-side is always 0. Non-differentiable.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn where_cond(&self, a: &NodeHandle, b: &NodeHandle) -> NodeHandle

Ternary select: result[i] = if self[i] { a[i] } else { b[i] }. Receiver is the cond mask (must be DType::Bool, GAP-168(c)); a and b must share dtype with each other and shape with self. Output dtype matches a/b. Returning a new tensor where each slot is picked from a (cond=1) or b (cond=0).

Named where_cond because where is a Rust reserved keyword. Common spelling in Candle/PyTorch families.

Differentiable through a and b only; gradient through the cond mask is None (registered in crate::grad::WhereRule).

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn cast(&self, target: DType) -> NodeHandle

Append a Cast node converting this tensor’s element type to target. Shape is preserved. An identity cast (target equals the current dtype) is a no-op that the optimizer elides (see opt::optimize’s identity-Cast rule) — kept here rather than special-cased at construction so every identity cast, however it arises (decompose, autograd, user), is removed in one place.

pub fn broadcast_to(&self, target: impl Into<Shape>) -> NodeHandle

Append a BroadcastTo node that expands self to the given target shape using NumPy broadcasting rules (right-align, pad with 1s, expand size-1 dims). The new tensor has the target shape and the same dtype as self.

pub fn try_broadcast_to( &self, target: impl Into<Shape>, ) -> Result<NodeHandle, Error>

Result-returning sibling of Self::broadcast_to. Surfaces shape incompatibility as a typed error rather than panicking.

pub fn unsqueeze(&self, dim: usize) -> NodeHandle

Append an Unsqueeze node that inserts a size-1 dimension at position dim. dim must be in 0..=rank (where dim == rank appends to the end). Pure metadata — the output shares bytes with self, with a new size-1 axis layered into the Layout side-table. Strictly more efficient than reshape for the size-1-insertion case because non-contiguous (e.g. transposed or broadcast) inputs flow through without auto-Contiguize.

pub fn try_unsqueeze(&self, dim: usize) -> Result<NodeHandle, Error>

Result-returning sibling of Self::unsqueeze. Surfaces dim > rank as a typed error rather than panicking.

pub fn squeeze(&self, dim: usize) -> Result<NodeHandle, Error>

Append a Squeeze node that drops the size-1 dimension at position dim (range 0..rank). Inverse of Self::unsqueeze. Metadata-only view: the output shares bytes with self, with the named dim pruned from the Layout side-table.

Returns Result rather than panicking — production paths can recover from a bad dim instead of crashing. Bad dim (out of bounds OR shape[dim] != 1) surfaces as a typed error.

pub fn contiguize(&self) -> NodeHandle

Append an Op::Contiguize node that materializes a contiguous copy of self. Output shape and dtype match the input; only the layout changes to contiguous + zero-offset. Zero-copy when the input is already contiguous + zero-offset (the executor adopts the input Storage Arc unchanged).

First-class IR node so the optimizer (Phase 2.2) can insert layout-fixups before kernels that don’t advertise crate::KernelCaps::strided_input without overloading Self::reshape’s “change shape” semantics.

pub fn reshape(&self, target: impl Into<Shape>) -> NodeHandle

Append a Reshape node producing self’s data under a new shape. The new shape must have the same total element count as the current shape.

pub fn try_reshape(&self, target: impl Into<Shape>) -> Result<NodeHandle, Error>

Result-returning sibling of Self::reshape. Surfaces element-count mismatch as a typed error rather than panicking.

pub fn reduce_sum_to(&self, target: impl Into<Shape>) -> NodeHandle

Append a ReduceSumTo node that sum-reduces self to a smaller shape. The target must be reachable from self.shape() via reduction of dims (i.e. self.shape() could be produced from target by broadcasting).

pub fn reduce_max_to(&self, target: impl Into<Shape>) -> NodeHandle

Append a ReduceMaxTo node that max-reduces self to a smaller shape — the max-symmetric counterpart of Self::reduce_sum_to. The target must be reachable from self.shape() via reduction of dims (i.e. self.shape() could be produced from target by broadcasting).

pub fn sum_all(&self) -> NodeHandle

Append a SumAll node reducing every element to a scalar (rank-0).

pub fn max_all(&self) -> NodeHandle

Append a MaxAll node producing the max element as a scalar.

pub fn min_all(&self) -> NodeHandle

Append a MinAll node producing the min element as a scalar.

pub fn mean_all(&self) -> NodeHandle

Append a MeanAll node producing the arithmetic mean as a scalar.

pub fn sum_dim(&self, dim: usize) -> NodeHandle

Append a SumDim(dim) node. Reduces along dim; output rank is input rank - 1 (the reduced dim is removed).

pub fn max_dim(&self, dim: usize) -> NodeHandle

Append a MaxDim(dim) node.

pub fn min_dim(&self, dim: usize) -> NodeHandle

Append a MinDim(dim) node.

pub fn mean_dim(&self, dim: usize) -> NodeHandle

Append a MeanDim(dim) node.

pub fn argmax_dim(&self, dim: usize) -> NodeHandle

Append an ArgMaxDim(dim) node. Produces a U32 tensor with the reduced dim removed, whose values are the index of the maximum along that dim. Non-differentiable — trying to run backward() through an ArgMax node will panic.

pub fn argmin_dim(&self, dim: usize) -> NodeHandle

Append an ArgMinDim(dim) node. Same semantics as argmax_dim but for the minimum.

pub fn softmax_last_dim(&self) -> NodeHandle

Append a SoftmaxLastDim node. Shape is preserved.

Phase 7.6 step 3: emits Op::Fused(FusedOps::SOFTMAX_LAST_DIM, FusedOpParams::SoftmaxLastDim) — the registry-extended arm. The legacy Op::SoftmaxLastDim variant remains in the enum during the migration; step 5 drops it once nothing emits it.

pub fn causal_conv1d( &self, weight: &NodeHandle, bias: &NodeHandle, use_silu: bool, ) -> NodeHandle

Append a CausalConv1d node — depthwise 1-D causal convolution

• bias + optional fused SiLU. Three inputs:

• self (x): [batch, channels, seq + kernel - 1] F32 (caller pre-pads with kernel - 1 zeros on the left for the causal mask — matches Mamba-2’s prefill convention).
• weight: [channels, 1, kernel] F32 (depthwise — one filter per channel; groups == channels).
• bias: [channels] F32 (required — matches baracuda’s causal_conv1d_*_run signature; pass zeros if you don’t want a bias).

Output: [batch, channels, seq] F32 where seq = x.dims[2] - (kernel - 1).

Emits Op::Fused(FusedOps::CAUSAL_CONV1D, FusedOpParams::CausalConv1d { use_silu }). See fuel-graph/src/registry/causal_conv1d.rs for the registry entry. No primitive decomposition exists (fuel-graph has no Op::Conv1D); backends without a native kernel fall through to the executor’s cpu_fallback path.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn nf4_matmul( &self, w_packed: &NodeHandle, absmax: &NodeHandle, block_size: usize, ) -> NodeHandle

Append an Nf4Matmul node — bitsandbytes-style 4-bit NormalFloat quantized matrix multiply. Three inputs:

• self (activations): [..., M, K], dtype ∈ {F32, F16, BF16}
• w_packed: [N, K/2] U8 (two 4-bit codes per byte)
• absmax: [N, K/block_size] F32 (per-output-row, per-block)

Output: [..., M, N] matching activations’ dtype.

block_size is the NF4 quantization block size (typically 64 in bitsandbytes). K must be even AND a multiple of block_size.

Emits Op::Fused(FusedOps::NF4_MATMUL, FusedOpParams::Nf4Matmul { block_size }). No primitive decomposition (the dequant + matmul roundtrip is exactly what NF4’s fused dequant-in-kernel design avoids); backends without a native kernel fall through to the executor’s cpu_fallback path.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn awq_matmul( &self, qweight: &NodeHandle, qzeros: &NodeHandle, scales: &NodeHandle, group_size: usize, split_k_iters: i32, ) -> NodeHandle

Append an AwqMatmul node — AWQ (mit-han-lab) asymmetric int4 weight-only matrix multiply. Four inputs:

• self (activations): [..., M, K], dtype ∈ {F32, F16, BF16}
• qweight: [N, K/8] U32 (eight int4 codes packed per word)
• qzeros: [K/group_size, N/8] U32 (eight int4 zero-point codes packed per word)
• scales: [K/group_size, N] F32 (per-group, per-output- channel scale)

Output: [..., M, N] matching activations’ dtype.

group_size must be positive and evenly divide K. K must be a multiple of 8 (qweight‘s packing); N must be a multiple of 8 (qzeros’ packing). split_k_iters is a CUDA launch-tuning parameter carried through unused by the (current, decompose-only) execution path — see crate::registry::awq_matmul’s module doc for what is and isn’t wired/verified yet, including the packing-order caveat.

Emits Op::Fused(FusedOps::AWQ_MATMUL, FusedOpParams::AwqMatmul { group_size, split_k_iters }).

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into, or if any shape/dtype precondition above doesn’t hold — same discipline as Self::nf4_matmul.

pub fn ssd_chunk_scan( &self, dt: &NodeHandle, a: &NodeHandle, b: &NodeHandle, c: &NodeHandle, chunk_size: usize, ) -> NodeHandle

Single-output SsdChunkScan: returns the y slot of the bundled producer (View(0)). See Self::ssd_chunk_scan_bundled for the multi-output variant.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn selective_scan( &self, delta: &NodeHandle, a: &NodeHandle, b: &NodeHandle, c: &NodeHandle, delta_softplus: bool, ) -> NodeHandle

Single-output SelectiveScan: returns the y slot of the bundled producer, with the View(0) projection emitted so callers can downstream-realize / copy / etc. transparently. See Self::selective_scan_bundled for the multi-output variant that also exposes last_state for autoregressive resumption.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn selective_scan_bundled( &self, delta: &NodeHandle, a: &NodeHandle, b: &NodeHandle, c: &NodeHandle, delta_softplus: bool, ) -> Result<(NodeHandle, NodeHandle), Error>

Multi-output variant of Self::selective_scan. Returns (y, last_state) — both Op::View tensors projected from the shared bundled producer. y matches the existing single-output selective_scan shape; last_state is the final hidden state [batch, dim, dstate] for autoregressive resumption.

pub fn ssd_chunk_scan_bundled( &self, dt: &NodeHandle, a: &NodeHandle, b: &NodeHandle, c: &NodeHandle, chunk_size: usize, ) -> Result<(NodeHandle, NodeHandle), Error>

Multi-output variant of Self::ssd_chunk_scan. Returns (y, last_state).

pub fn nonzero_indices_producer(&self, count_sym: SymId) -> NodeHandle

Append an Op::NonZeroIndices producer node — the keystone primitive for data-dependent dynamic shapes. Returns the bundled producer NodeHandle (a [capacity] U32 tensor); callers usually want Self::nonzero_indices_bundled to also get the runtime count slot.

capacity = self.elem_count() is the static worst-case bound (every element nonzero). count_sym is the SymId the executor binds to the realized nonzero count in the per-pass SymEnv; the caller allocates it from their SymGen and threads it into any downstream op that consumes the count as a dynamic extent / DynScalar::Sym (the KV-cache cached_len pattern, generalized to a data-determined count).

pub fn nonzero_indices_bundled( &self, count_sym: SymId, ) -> Result<(NodeHandle, NodeHandle), Error>

Multi-output Op::NonZeroIndices: returns (indices, count) — both Op::View projections of the shared bundled producer. indices is [capacity] U32 (first count entries valid, in ascending flat order); count is [1] U32, the runtime nonzero count. The executor also publishes count’s value into the per-pass SymEnv under the producer’s count_sym.

pub fn fused_softmax_cross_entropy( &self, targets: &NodeHandle, reduction: Reduction, ignore_index: i64, ) -> NodeHandle

Append a FusedSoftmaxCrossEntropy node. Two inputs:

• self (logits): [..., V] F32 / F64 / BF16 / F16
• targets: [...] I64 (class indices; matches PyTorch / baracuda convention)

Output dtype is always F32 (the FSCE declared dtype — losses accumulate in F64 and narrow to F32, matching PyTorch and the baracuda kernel) regardless of the logits dtype. Output shape depends on reduction:

• Reduction::Mean / Reduction::Sum → scalar []
• Reduction::None → same as targets.shape

ignore_index rows are dropped from the loss accumulator and the Mean denominator. The conventional sentinel is -100.

Emits Op::Fused(FusedOps::FUSED_SOFTMAX_CROSS_ENTROPY, FusedOpParams::FusedSoftmaxCrossEntropy { reduction, ignore_index }). See fuel-graph/src/registry/fused_softmax_cross_entropy.rs for the registry entry and decompose chain.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn layer_norm_last_dim(&self, eps: f64) -> NodeHandle

Append a LayerNormLastDim node with the given epsilon. Shape is preserved.

Phase 7.6 step 4 (continued): emits Op::Fused(FusedOps::LAYER_NORM_LAST_DIM, FusedOpParams::LayerNormLastDim { eps }) through the registry-extended arm. The legacy Op::LayerNormLastDim { eps } variant remains in the enum during migration; step 5 drops it once nothing emits it.

pub fn rope(&self, base: f64, start_pos: usize) -> NodeHandle

Apply rotary position embeddings (RoPE) to self. The input is expected to have shape [..., seq, d] where seq is the sequence length and d is the per-head feature dim (which must be even). base is the frequency base — 10000.0 for LLaMA. start_pos lets callers apply RoPE at a non-zero offset, which is what KV-cached inference does when the query for position n + k is processed against a cache that already holds positions 0..n+k.

Uses the half-split layout (the one LLaMA and HuggingFace use): pair feature i with feature i + d/2. Under that layout the rotation reduces to the identity x' = x·cos + rotate_half(x)·sin where rotate_half(x) concatenates [-x[d/2:], x[:d/2]] along the last dim.

Implemented as a composition of existing graph primitives — Slice, Neg, Concat, Reshape, BroadcastTo, Mul, Add — plus two precomputed Const tensors for the cos/sin frequency tables. Every primitive already has a working backward rule, so gradients flow through RoPE without any new backward machinery.

Only f32 is supported today. Extending to other float dtypes is mechanical: build the frequency tables in the target dtype.

pub fn rope_with_tables(&self, cos: &NodeHandle, sin: &NodeHandle) -> NodeHandle

Apply RoPE using caller-supplied cos and sin tables. Each table has shape [seq, head_dim] and matches the layout build_rope_tables produces. This is the hot-path entry point: in a transformer forward pass every attention layer applies RoPE to Q and K with the same (start_pos, seq, head_dim), so the caller can build the tables once and share the const nodes across all layers rather than re-duplicating them inside each .rope() call. The classic Self::rope entry point funnels through this after building the tables itself.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn rms_norm_last_dim(&self, eps: f64) -> NodeHandle

Root-Mean-Square Normalization along the last dim, without affine parameters. y = x / sqrt(mean(x², last) + eps).

This is the normalization layer used by LLaMA-family models. Unlike LayerNorm, RmsNorm does not subtract the mean — it only divides by the root-mean-square, which is both cheaper and empirically just as effective in transformer blocks.

Emits a fused RmsNormLastDim node. Backends that have a native implementation dispatch it as one kernel; ones that don’t get a CPU fallback via the reference implementation.

Phase 7.6 step 4 (continued): emits Op::Fused(FusedOps::RMS_NORM_LAST_DIM, FusedOpParams::RmsNormLastDim { eps }) through the registry-extended arm. The legacy Op::RmsNormLastDim { eps } variant remains in the enum during migration; step 5 drops it once nothing emits it.

Use rms_norm_last_dim_decomposed instead if you need to differentiate through this op — backward through the fused op is not yet implemented.

pub fn rms_norm_last_dim_decomposed(&self, eps: f64) -> NodeHandle

Decomposed version — emits the (sqr → mean_dim → reshape → add_scalar → sqrt → broadcast_to → div) primitive chain. Has working backward rules through its primitive subgraph. Use this when you need to differentiate through RMSNorm; use rms_norm_last_dim for inference.

pub fn index_select(&self, dim: usize, indices: &NodeHandle) -> NodeHandle

Append an IndexSelect node that picks slices from self along dim using a 1-D u32 index tensor. The output has the same shape as self except dimension dim is replaced by indices.shape()[0].

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn gather(&self, dim: usize, indices: &NodeHandle) -> NodeHandle

Append a Gather node that performs an N-dimensional gather along dim. The indices tensor must be U32 with the same rank as self. Output shape equals indices.shape().

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn concat(&self, other: &NodeHandle, dim: usize) -> NodeHandle

Append a Concat node joining self and other along dim.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn slice(&self, dim: usize, start: usize, len: usize) -> NodeHandle

Append a Slice (narrow) node taking elements [start, start+len) along dim. The output has the same rank; only dim shrinks.

pub fn add_scalar(&self, c: f64) -> NodeHandle

Append an AddScalar node: y[i] = x[i] + c. Convenient for avoiding a full-shape const node when you just want to bias a tensor by a constant.

pub fn mul_scalar(&self, c: f64) -> NodeHandle

Append a MulScalar node: y[i] = x[i] * c.

pub fn powi(&self, n: i32) -> NodeHandle

Append a PowI node raising each element to an integer power.

pub fn clamp(&self, min: f64, max: f64) -> NodeHandle

Append a Clamp node restricting each element to [min, max].

pub fn maximum(&self, other: &NodeHandle) -> NodeHandle

Append a Maximum node max(self, other) element-wise. Matching shapes required.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn minimum(&self, other: &NodeHandle) -> NodeHandle

Append a Minimum node min(self, other) element-wise.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn fmax_ieee(&self, other: &NodeHandle) -> NodeHandle

The IEEE-754 NaN-suppressing maximum — the KISS-OPS §6.15-0001 fmax_ieee op, DISTINCT from maximum (which is NaN-propagating, torch parity). If exactly one operand is NaN, returns the other; if both are NaN, returns NaN; otherwise the larger.

Built by RESOLUTION through the spec-pinned §6.13 decomposition (KISS-OPS-6.15-0001), used VERBATIM — Fuel does not substitute maximum for it (that substitution is the §6.15 violation). Because the decomposition is spec-authored, its edge behaviour — including ±0 ordering under cmp_ge (cmp_ge(-0.0, +0.0) is true, so this returns -0.0 for that pair) — is the SPEC’s to define and a KISS conformance vector’s to catch. Fuel resolves; it does not re-derive. (GAP-048.)

pub fn fmin_ieee(&self, other: &NodeHandle) -> NodeHandle

The IEEE-754 NaN-suppressing minimum — the KISS-OPS §6.15-0001 fmin_ieee op, DISTINCT from minimum (which is NaN-propagating, torch parity). If exactly one operand is NaN, returns the other; if both are NaN, returns NaN; otherwise the smaller.

Built by RESOLUTION through the spec-pinned KISS-OPS §6.13 decomposition (KISS-OPS-6.15-0001), used VERBATIM — the cmp_le mirror of fmax_ieee’s cmp_ge form. Fuel resolves; it does not substitute minimum (that substitution is the §6.15 violation) and does not re-derive: the decomposition is KISS’s, so its ±0 edge behaviour is the spec’s to define and a KISS conformance vector’s to catch. (GAP-048.)

pub fn trunc(&self) -> NodeHandle

Round toward zero — the KISS-OPS §6.3 trunc floor op (GAP-300).

⚠️ Unlike fmax_ieee/fmin_ieee, which resolve through KISS’s OWN §6.13 decompositions, trunc is a §6.3 floor op: KISS-OPS-6.3-0003 forbids the standard from carrying a reference decomposition for a floor op, so THIS expansion is FUEL’s, blessed by no spec clause. It IS now checked by KISS vectors: ops-rounding.json (vendored at KISS 0af5d3b6) carries 18 f32 trunc cells (signed zero, halves and ties, 2^23, max finite, ±inf, subnormals), all of which pass through the lazy Tensor (fuel-tensor/tests/kiss_nan_conformance.rs); f64 and NaN inputs are not in that file. Beyond it, correctness is Fuel’s liability — see GAP-300, whose born-reds assert on to_bits() (a value compare 0.0 == -0.0 passes vacuously).

trunc(q) = if q >= 0 { floor(q) } else { ceil(q) }. Float dtypes only; integer dtypes are already whole, so trunc is identity for them (returned unchanged — never a panic on a non-float operand).

REJECTED-FOR-NOW alternative: copysign(floor(abs(q)), q) is cleaner (no dtype-matched zero; ±0/NaN/inf fall out of the IEEE primitives). Fuel has no copysign op today (absent from Op and the builder API), so it is not buildable — adopt it IF copysign ever lands.

pub fn rem_trunc(&self, other: &NodeHandle) -> NodeHandle

Truncated remainder — the KISS-OPS §6.15-0003 rem_trunc op, DISTINCT from rem (which is FLOORED / PyTorch, sign of the divisor). rem_trunc takes the sign of the dividend (C99 fmod / Rust %).

Built by RESOLUTION through KISS’s spec-pinned §6.13 decomposition (KISS-OPS-6.15-0003) — a - trunc(a / b) * b — used VERBATIM. Fuel does not substitute rem (that substitution is the §6.15 violation); the two diverge whenever the operands have opposite signs (e.g. -7 rem_trunc 3 = -1, where rem (floored) = 2). The trunc leaf is Fuel’s own expansion (GAP-300).

pub fn broadcast_add(&self, other: &NodeHandle) -> NodeHandle

Element-wise addition with automatic broadcasting. Unlike add, which requires matching shapes, broadcast_add computes the broadcast shape of the two operands, inserts explicit BroadcastTo nodes as needed, and then emits a regular Add. Useful for bias addition ([batch, hidden] + [hidden]) without the caller writing the broadcast out explicitly.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn broadcast_sub(&self, other: &NodeHandle) -> NodeHandle

Element-wise subtraction with automatic broadcasting.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn broadcast_mul(&self, other: &NodeHandle) -> NodeHandle

Element-wise multiplication with automatic broadcasting.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn broadcast_div(&self, other: &NodeHandle) -> NodeHandle

Element-wise division with automatic broadcasting.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn index_add( &self, dim: usize, indices: &NodeHandle, src: &NodeHandle, ) -> NodeHandle

Append an IndexAdd node — the functional inverse of IndexSelect. Returns base with src added at positions given by indices along dim.

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn scatter_add( &self, dim: usize, indices: &NodeHandle, src: &NodeHandle, ) -> NodeHandle

Append a ScatterAdd node — the functional inverse of Gather. Returns base with values from src accumulated at positions given by indices (with indices[p] substituted at dim).

Panics

Panics if any operand belongs to a different Graph than the one this builder is building into. Operand identity is graph-scoped and checked by pointer identity (Arc::ptr_eq), which cannot alias; the panic is deliberate and replaces the hang a cross-graph handle used to produce. Build every operand from one graph, or use a Result-returning entry point where one exists.

pub fn backward(&self) -> GradMap

Build the backward graph from self, treating self as the output whose gradient drives the rest of the computation.

Walks the forward graph in reverse topological order, applies the per-op gradient rule to each node, and emits new graph nodes for the gradient with respect to every node reachable from self. The returned GradMap lets you look up the gradient tensor for any forward tensor from the same graph.

The initial upstream gradient is a ones tensor of the same shape and dtype as self — the conventional dL/dL = 1 seed for backprop. For a scalar loss this is [1.0]; for a non-scalar output it is “each element of the output contributes with weight 1 to the total derivative.”

Supported ops in the MVP

Const, Add, Mul, MatMul, Transpose, Sqr, Exp all have gradient rules implemented. Relu panics — its gradient needs a Step/Where/Sign op that is not yet in the catalog and will land in a follow-up once an indicator-valued op is defined.

Ownership and the forward graph

Backward extends the same graph that held the forward pass. The gradient nodes live alongside the forward nodes in one arena. After backward(), realizing a gradient tensor re-executes whatever part of the forward pass the gradient depends on — the executor walks the combined graph as one.

Trait Implementations

impl Clone for NodeHandle

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for NodeHandle

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.