Module fused 

FusedOpRegistry — kernel-side payload. Phase 7.6 step 1 (skeleton).

Architecture v1.0 splits the fused-op registry across two crates:

• graph-side metadata in fuel-graph::registry (id, name, family, pattern, decompose, backward, shape/dtype rules);
• kernel-side payload here in fuel-storage::fused (BackendImpl, CostEstimate, PrecisionGuarantee, KernelRevisionHash).

The split exists because KernelRef lives in fuel-storage and fuel-graph cannot depend on fuel-storage (the dependency arrow goes the other way). Joining the two halves is by fuel_graph::registry::FusedOpId at runtime: the optimizer reads the metadata-side entry to reason about decomposition and shape, then asks the kernel-side FusedKernelRegistry for the per-backend BackendImpl when it needs to pre-resolve a KernelRef.

Status (step 1)

Types only. No callers; no behavior change. Subsequent steps:

• Step 3: register the SoftmaxLastDim CPU BackendImpl and dispatch Op::Fused(SOFTMAX_LAST_DIM, _) through it from the executor.
• Step 6-9: extend per-backend coverage, populate PrecisionGuarantee and cost, and migrate the binding-table lookup off the executor’s hot path.

Structs

BackendImpl

Per-backend kernel implementation for one fused op. The optimizer reads this to (1) pre-resolve KernelRef for nodes it places on this backend, (2) score routes against CostEstimate, and (3) admit candidates against the per-route tolerance budget via PrecisionGuarantee.

CostEstimate

Coarse cost model for one kernel invocation. Layer-1 of the architecture’s cost-model tower (FLOP counts + bandwidth + launch overhead). Layer-2 — empirical refinement from per-deployment telemetry — composes on top of these without changing this shape.

FusedKernelRegistry

Kernel-side registry: FusedOpId → list of per-backend BackendImpls. Joined to fuel-graph’s metadata-side fuel_graph::registry::FusedOpRegistry by id at runtime.

KernelRevisionHash

Opaque revision hash of a registered kernel. Persisted optimization caches read this to detect kernel drift between cache build and cache load (see docs/architecture/11-persistence.md). Computed from kernel source + version metadata at registration time; step 9 fills in the actual hashing function alongside the binding-table planning-time refactor.

PrecisionGuarantee

What this kernel guarantees about its numerical behavior.

Constants

ATTN_BACKWARD_CPU_PRECISION

Precision guarantee for (FLASH_ATTN_BACKWARD_{Q,K,V}, Cpu, *) impls. The CPU backward is the math-definition gradient: it recomputes the forward softmax probabilities row-by-row and accumulates dQ/dK/dV with sequential nested loops — fixed iteration order, no atomics, no parallel reductions. F32/F64 run natively; BF16/F16 accumulate in F32 and narrow on store. Bit-identical re-run on same hardware.

ATTN_CPU_PRECISION

Precision guarantee for (FLASH_ATTN | PAGED_ATTN, Cpu, *) impls. The CPU naive-attention reference is bit-stable per-hardware because both kernels iterate in fixed order. However the tiled flash-attention GPU kernel produces different numerics than the naive reference (different reduction tree); that’s a GPU-side PrecisionGuarantee concern — the CPU side stays bit-stable.

CAUSAL_CONV1D_CPU_PRECISION

Precision guarantee for (CAUSAL_CONV1D, Cpu, *) — textbook nested-loop depthwise conv with F32 accumulator; SiLU computed element-wise via x / (1 + exp(-x)). Iteration order fixed; bit-identical re-run on same hardware.

CONV2D_CPU_PRECISION

Precision guarantee shared by every (CONV2D, Cpu, *) impl in fuel-cpu-backend: textbook nested-loop accumulation in F32 (F64 kernels accumulate in F64; BF16/F16 upcast each multiply to F32 and narrow at end), bit-identical re-run on same hardware. Same shape as FUSED_LINEAR_CPU_PRECISION — the conv kernel’s numerical character is governed by the same “F32 accumulator, deterministic iteration order” properties.

CONV_TRANSPOSE2D_CPU_PRECISION

Precision guarantee for (CONV_TRANSPOSE2D, Cpu, *) impls. Same structure as Conv2D — textbook nested-loop accumulation in F32 (F64 for F64 input).

FUSED_LINEAR_CPU_PRECISION

Precision guarantee shared by every (FUSED_LINEAR, Cpu, *) impl in fuel-cpu-backend: deterministic per-element accumulation in F32 (BF16/F16 inputs upcast and narrow at the end), bit-identical on same hardware. Bounded error claims (max_ulp / max_relative) depend on the contracting dim K and so stay None at this layer — the empirical calibration framework derives them per-shape.

FUSED_SOFTMAX_CROSS_ENTROPY_CPU_PRECISION

Precision guarantee for (FUSED_SOFTMAX_CROSS_ENTROPY, Cpu, *) — stable log-sum-exp accumulated in F64, gather + NLL in F64, narrow to F32 at end. Bit-identical re-run on same hardware. The kernel’s iteration order is fixed, so the only sources of non-determinism (reduction-order changes, denormal flushing) are excluded by construction.

INPLACE_AFFINE_CPU_PRECISION

Precision guarantee for (INPLACE_AFFINE, Cpu). Single-pass x = mul · x + add per element with no parallel accumulation; deterministic iteration order; bit-identical same-hardware re-run. The half-precision dtypes (bf16/f16) pivot through f32 for the arithmetic, identical to the non-inplace Affine family.

NF4_MATMUL_CPU_PRECISION

Precision guarantee for (NF4_MATMUL, Cpu, *) — inner product accumulated in F32 regardless of activation/output dtype (F16 + BF16 variants up-cast on load and narrow on store, matching bitsandbytes’ fp16/bf16 GEMV behavior). Iteration order fixed; bit-identical re-run on same hardware. The 16-entry NF4 LUT is baked in as a const so quantization roundtrips are reproducible.

NORM_FAMILY_CPU_PRECISION

Shared precision guarantee for the norm/softmax family — softmax, rms_norm, layer_norm. All three CPU kernels use deterministic elementwise + scalar-reduction patterns with F32 accumulator (F64 for F64 input; BF16/F16 multiply in F32 and narrow at end). Same shape as FUSED_LINEAR_CPU_PRECISION — the numerical character is governed by the same “F32 accumulator, deterministic iteration order” properties.

POWI_BACKWARD_CPU_PRECISION

Precision guarantee for (POWI_BACKWARD, Cpu). Per-element grad_x = exp · x^(exp-1) · upstream with the same floating-point determinism guarantees as the forward PowI primitive: deterministic iteration order, no parallel reductions, F32 compute for half- precision dtypes via the shared powi_backward_half_kernel.

QMATMUL_CPU_PRECISION

Precision guarantee for (QMATMUL, Cpu, *) impls. Quantized matmul dequantizes inline; the dequant arithmetic is exact for the quantization scheme’s block format, and the F32 matmul accumulator stays deterministic. Bit-identical on same hardware.

REDUCE_MAX_TO_BACKWARD_CPU_PRECISION

Precision guarantee for (REDUCE_MAX_TO_BACKWARD, Cpu, *) impls. The kernel recomputes the forward max, builds a tie-mask, counts ties via reduce_sum, divides upstream by the count, then broadcasts back and gates by the mask. All deterministic on same hardware (no atomics, no parallel reductions); F32 accumulator for half-precision dtypes via the shared reduce_max_to_backward_impl adapter.

ROPE_CPU_PRECISION

Precision guarantee for (ROPE, Cpu, *) impls. Rotary position embedding is a rotation in 2D subspaces — multiplies + adds with the cos/sin tables; F32 accumulator semantics match the norm family.

SELECTIVE_SCAN_CPU_PRECISION

Precision guarantee for (SELECTIVE_SCAN, Cpu, *) — recurrence accumulated in F64 (h is Vec<f64> internally), final y narrowed to F32. Iteration order fixed; bit-identical re-run on same hardware. The F64 accumulator buys ~7 extra decimal digits of headroom vs naive F32 over the per-step state update.

SSD_CHUNK_SCAN_CPU_PRECISION

Precision guarantee for (SSD_CHUNK_SCAN, Cpu, *) — same shape as SELECTIVE_SCAN_CPU_PRECISION.

VULKAN_BYTE_LEVEL_PRECISION

Vulkan byte-level (memcpy-style) ops — Triu, Tril, Flip, Roll, WriteSlice, IndexSelect, Concat, Copy (D2H/H2D). Pure data movement with no FP arithmetic; output bytes are bit-identical to input bytes selected/permuted according to the op.

VULKAN_CAST_PRECISION

Vulkan dtype cast — f32↔f16, f32↔bf16, F8E4M3↔{f32,f16,bf16}. Pure conversion with no accumulation; output is the IEEE-754 round-to-nearest representation of the input in the target dtype.

VULKAN_FLOAT_POINTWISE_PRECISION

Vulkan f32 / f64 pointwise ops — Add, Sub, Mul, Div, Maximum, Minimum, Neg, Sqr, Sqrt, Relu, Step, Affine, Clamp.

VULKAN_HALF_POINTWISE_PRECISION

Vulkan f16 / bf16 pointwise ops — same op surface as VULKAN_FLOAT_POINTWISE_PRECISION at half precision. f16 uses shaderFloat16 native ops (where available); bf16 packs two values per u32 with software upcast/downcast through f32 for the arithmetic.

VULKAN_MATMUL_PRECISION

Vulkan matmul (f32 standard, no tensor cores) — tiled / reg-tile / matvec kernels. Accumulation order is deterministic per kernel dispatch (each output element’s reduction runs in a single thread with no cross-thread atomic adds); the kernel is bit-stable on same hardware.

VULKAN_MATMUL_TENSORCORE_PRECISION

Vulkan tensor-core mixed-precision matmul (f32 × bf16 → f32, via matmul_coop cooperative kernels on Ampere+ tensor cores).

VULKAN_QMATMUL_PRECISION

Vulkan QMatMul (Q4_0 / Q4_K_M / Q8_0 weights × F32 activation). Per-block dequant + per-output-element FMA accumulation; same deterministic structure as the CPU QMatMul kernel (bit-stable on same hardware).

VULKAN_TRANSCENDENTAL_PRECISION

Vulkan transcendentals — Exp, Log, Sin, Cos, Tanh, Sigmoid, Silu, Gelu (f32/f16/bf16/f64). f32/f16 go through GLSL.std.450’s vendor- library implementations (variable ULP per GPU/driver); f64 uses the Horner-polynomial approximations in unary_f64.slang (target ~1e-12 relative error).

Functions

cost_attn_backward_cpu

Cost model for (FLASH_ATTN_BACKWARD_{Q,K,V}, Cpu) kernels. The math-definition backward recomputes the attention probabilities (one QKᵀ matmul) and then forms dV, dP, dQ and dK — five O(B·Hq·Sq·Sk·D) matmul-shaped passes vs the forward’s two. The CPU byte wrapper additionally computes ALL THREE gradients per call and persists only the requested one (the Q/K/V OpKind split has one output each), so the per-call cost does not shrink with the selected gradient.

cost_attn_cpu

Cost model for (FLASH_ATTN, Cpu) and (PAGED_ATTN, Cpu) kernels — scaled-dot-product attention is O(B·Hq·Sq·Sk·D) multiplies for the QK matmul + same for the PV matmul, plus softmax (small).

cost_causal_conv1d_cpu

Static cost model for (CAUSAL_CONV1D, Cpu) — depthwise 1-D conv. FLOPs scale with batch × channels × seq_out × kernel; bandwidth is batch × channels × seq_in (x reads) + channels × kernel (weight reads) + channels (bias) + batch × channels × seq_out (output writes). SiLU adds one transcendental per output element.

cost_conv2d_cpu

Static cost model for (CONV2D, Cpu) kernels — FLOP + bandwidth per architecture v1.0 §“Layer-1 cost model.”

cost_conv_transpose2d_cpu

Cost model for (CONV_TRANSPOSE2D, Cpu) — FLOP count parallels Conv2D: 2·N·Cout·Hout·Wout·(Cin/g)·Kh·Kw (each output position accumulates contributions from Cin/g · Kh · Kw input positions, reading transposed).

cost_fused_linear_cpu

Static cost model for (FUSED_LINEAR, Cpu) kernels — the conservative FLOP + bandwidth model per architecture v1.0 §“Layer-1 cost model.” Identical for F32/F64/BF16/F16 (the kernels accumulate in F32 regardless of input dtype); the empirical Layer-2 refinement framework will tighten per-dtype later.

cost_fused_softmax_cross_entropy_cpu

Static cost model for (FUSED_SOFTMAX_CROSS_ENTROPY, Cpu) — row-by-row stable log-softmax + gather. FLOPs scale with n_rows × vocab (two passes per row: one for max+sum_exp, one for the loss accumulator); memory is n_rows × vocab reads plus the targets and the scalar/vector output.

cost_inplace_affine_cpu

Cost model for (INPLACE_AFFINE, Cpu). Single pass: 1 FMA per element. Bandwidth is one read + one write per element (in place, so the buffer is touched once for read + once for write through the same cache line).

cost_nf4_matmul_cpu

Static cost model for (NF4_MATMUL, Cpu) — bitsandbytes 4-bit quantized matmul. FLOPs scale with batch · m · n · k (one FMA per k step, after dequant). Bandwidth reads batch·m·k activations + n·k/2 packed weight bytes + n·k/block_size absmax scales + writes batch·m·n outputs.

cost_norm_family_cpu

Cost model for (SoftmaxLastDim | RmsNormLastDim | LayerNormLastDim, Cpu) kernels — outer-product structure: outer × last_dim elementwise pass + scalar reduction. FLOPs scale linearly with element count, bandwidth dominates for typical token-dim sizes.

cost_powi_backward_cpu

Cost model for (POWI_BACKWARD, Cpu) — single pass over the input: per-element powi(exp-1) (≈ log2(|exp|) FMAs) + 2 multiplies. Bandwidth dominated by reading x + upstream and writing grad_x.

cost_qmatmul_cpu

Cost model for (QMATMUL, Cpu) — same FLOP count as a regular matmul (the dequant adds a small per-block overhead that’s dwarfed by the FMA count for any useful K).

cost_reduce_max_to_backward_cpu

Cost model for (REDUCE_MAX_TO_BACKWARD, Cpu) — five passes over the input: forward max, mask build, count, scale, broadcast + gate. Roughly 5 · in_count FLOPs, with bandwidth dominated by 5× the input element count.

cost_rope_cpu

Cost model for (ROPE, Cpu) — per-element rotation costs 4 FMAs (2 multiplies + 2 adds in two 2D planes).

cost_selective_scan_cpu

Static cost model for (SELECTIVE_SCAN, Cpu) — Mamba’s selective state-space scan. FLOPs scale with batch · seqlen · dim · dstate (the inner triple-nested loop). Bandwidth is dominated by reading u/delta/b/c per timestep + writing y.

cost_ssd_chunk_scan_cpu

Static cost model for (SSD_CHUNK_SCAN, Cpu) — Mamba-2’s chunked SSD scan. FLOPs scale with batch · seqlen · heads · head_dim · state_dim (the inner quadruple-nested loop). Bandwidth reads x/dt/b/c per timestep + a once + writes y.

default_kernel_registry

Process-wide default kernel registry: every backend’s fused-op BackendImpls registered at startup. Built once on first access via std::sync::OnceLock; immutable thereafter (architecture v1.0: no runtime extensibility).

register_default_kernels

Populate a FusedKernelRegistry with the always-built kernels (today: fuel-cpu-backend FusedLinear F32/F64/BF16/F16). Called from default_kernel_registry’s OnceLock initializer; exposed as a free function so backend crates can compose against a custom registry in tests.