Module cost 

Phase 7.6 step 8 — Layer-1 cost model for primitive ops.

Each primitive OpKind registered in crate::kernel::KernelBindingTable gets a crate::kernel::CostFn that returns a crate::fused::CostEstimate (FLOPs + bytes moved + launch overhead). The architecture’s Layer-1 model is conservative + static: it counts FLOPs from shapes and OpParams, estimates bandwidth as operand count × element count × dtype byte width, and assigns a per-family launch-overhead constant (CPU launches measure in tens of ns). Layer-2 — empirical refinement from per-deployment telemetry — composes on top via the framework that lands with step 11 (community-aggregated cache).

Structure

• Per-family cost functions (cost_elementwise_unary_cpu, cost_elementwise_binary_cpu, cost_reduction_cpu, etc.). Each is the crate::kernel::CostFn signature and operates on whatever shapes + OpParams payload its OpKind family carries.
• default_cost_for_op_kind — the dispatcher consumed by crate::kernel::KernelBindingTable::fill_unset_cpu_cost. Maps every OpKind variant to the family function appropriate for its shape contract.

Conventions

• FLOP counts use the “FMA = 2 FLOPs” convention matching the step-6 fused-op cost functions (matmul = 2·M·N·K, etc.).
• Bandwidth assumes 4 B/element as a midpoint (F32 reference); half-precision is conservatively over-counted, F64 under. The step-8 Layer-1 model is intentionally coarse — empirical refinement tightens per-dtype later.
• Launch overhead: 50 ns for elementwise + reduction families (matches the fused-op precedent); 100-200 ns for matmul / attention (more setup work).

Functions

cost_argindex_cpu

Cost for ArgMaxDim / ArgMinDim. Inputs [in:T, out:U32]. FLOPs ≈ input element count (per-row compare); bytes are in×dsize + out×4 bytes.

cost_cast_cpu

Cost for Cast. Pure dtype conversion — no FLOPs (or ~1 conversion/element, dwarfed by bandwidth). Bytes are in_size + out_size per element with their respective dtype widths.

cost_causal_conv1d_primitive_cpu

Cost for CausalConv1d (binding-table form). Reads batch × channels × seq_out × kernel FLOPs (2 per FMA) plus an optional ~10-FLOP SiLU per output. Bandwidth approximation: x + weight + bias + out, all F32.

cost_comparison_cpu

Cost for comparison ops (Equal/Ne/Lt/Le/Gt/Ge). Same shape contract as elementwise binary but output is U8 (1 B/element).

cost_concat_cpu

Cost for Concat. Sum of N input element counts; one output of equal total size. No FLOPs (memcpy).

cost_conv2d_primitive_cpu

Cost for Conv2D (binding-table form). Reads OpParams::Conv2D for shape geometry.

cost_conv_transpose2d_primitive_cpu

Cost for ConvTranspose2D (binding-table form). Same shape algorithm as Conv2D — the transposed pass moves comparable bytes and does comparable FLOPs.

cost_elementwise_binary_cpu

Cost for elementwise binary ops (Add/Sub/Mul/Div/Maximum/Minimum/ Pow/Rem). ~1 FLOP/element (Pow/Rem run higher but the bandwidth dominates for typical sizes); 2 reads + 1 write per element.

cost_elementwise_unary_cpu

Cost for elementwise unary ops with cheap math (Neg/Sqr/Recip/ Abs/Relu/Step/Floor/Ceil/Round/Sign/Sqrt). ~1 FLOP/element; 1 read + 1 write per element.

cost_elementwise_unary_transcendental_cpu

Cost for elementwise unary transcendentals (Exp/Log/Sin/Cos/ Tanh/Sigmoid/Silu/GeluTanh/Gelu/Erf/Rsqrt). ~10 FLOPs/element (transcendentals lower to 5-15 hardware ops on average).

cost_flash_attn_primitive_cpu

Cost for FlashAttn (binding-table form). Reads geometry from OpParams::FlashAttn.

cost_flash_decoding_cuda

Cost for the CUDA flash_decoding decode arm (binding-table form).

cost_fused_linear_primitive_cpu

Cost for FusedLinear (matmul + bias-add). Reads Matmul params (same as MatMul) — bias-add contributes M·N FLOPs per batch on top of the matmul.

cost_fused_softmax_cross_entropy_primitive_cpu

Cost for FusedSoftmaxCrossEntropy (binding-table form). Reads n_rows × vocab from OpParams::FusedSoftmaxCrossEntropy. Two passes per row (max + sum_exp), plus one transcendental log per row; bandwidth is logits + targets + scalar output. The static estimate intentionally underestimates the kernel-launch / per-row overhead — the empirical layer compensates.

cost_indexing_cpu

Cost for indexing-style ops (IndexSelect, Gather, IndexAdd, ScatterAdd). FLOPs = 0 for select/gather, = src_count for add/ scatter (accumulation). Bytes ≈ (data + indices + out) × respective dsizes.

cost_masked_fill_cpu

Cost for MaskedFill (x, mask:U8 → fill where mask != 0).

cost_matmul_cpu

Cost for MatMul. Reads Matmul { m, n, k, lhs_batch_dims, rhs_batch_dims } from OpParams to compute the exact FLOP count (2·batch·M·N·K).

cost_nf4_matmul_primitive_cpu

Cost for Nf4Matmul (binding-table form). FLOPs scale with batch × m × n × k. Bandwidth: activations + w_packed (4-bit per elem, so n*k/2 bytes) + absmax + output.

cost_norm_last_dim_primitive_cpu

Cost for NormLastDim family — LayerNormLastDim, RmsNormLastDim, and their backwards. Same outer × last_dim shape as softmax.

cost_paged_attn_primitive_cpu

Cost for PagedAttn (binding-table form). Reads geometry from OpParams::PagedAttn; treats num_blocks · block_size as the effective Sk upper bound.

cost_qmatmul_primitive_cpu

Cost for QMatMul (binding-table form). Reads dimensions from OpParams::QMatMul.

cost_reduce_max_to_backward_primitive_cpu

Cost for ReduceMaxToBackward — 5-pass kernel (max recompute + tie-mask + count + scale + gate).

cost_reduce_to_cpu

Cost for ReduceSumTo / ReduceMaxTo. Same shape as reductions but the broadcast-to-target structure may iterate the input multiple times depending on which axes reduce; we approximate as in_count FLOPs.

cost_reduction_cpu

Cost for full-tensor reductions (SumReduce/MaxReduce/MinReduce/ MeanReduce). FLOPs ≈ input element count; one write per output element.

cost_rope_primitive_cpu

Cost for Rope (binding-table form).

cost_scalar_op_cpu

Cost for scalar-by-tensor ops (Affine = mul·x + add, Clamp, PowI). 1-2 FLOPs/element; 1 read + 1 write per element.

cost_selective_scan_primitive_cpu

Cost for SelectiveScan (binding-table form). FLOPs scale with batch × seqlen × dim × dstate. Conservative ~16 FLOPs per inner iteration (exp ≈ 10, plus the FMAs for h update and y accumulate).

cost_shape_op_cpu

Cost for materializing shape-rearrangement ops (Flip, Roll, Triu, Tril, CumSum, Pad, PadBackward). FLOPs are zero (CumSum is the exception — it does N adds — but bandwidth dominates).

cost_softmax_last_dim_primitive_cpu

Cost for SoftmaxLastDim / SoftmaxLastDimBackward — outer_count rows × last_dim with constant FLOPs/element.

cost_ssd_chunk_scan_primitive_cpu

Cost for SsdChunkScan (binding-table form). FLOPs scale with batch × seqlen × heads × head_dim × state_dim. Conservative ~8 FLOPs per inner iteration (mul-mul-add for h update + mul-add for y accumulate, with exp amortized per outer (b, t, h)).

cost_where_cpu

Cost for Where (ternary select). Inputs [cond:U8, a:T, b:T, out:T].

default_cost_for_op_kind

Phase 7.6 step 8: maps every OpKind variant to its default cost-family function. Consumed by crate::kernel::KernelBindingTable::fill_unset_cpu_cost at the end of register_cpu_kernels.