# Fuel CPU backend kernel inventory — `fuel-cpu-backend`
Scope: the **portable byte-shaped CPU kernels** Fuel itself provides, i.e. the
production `CpuStorageBytes` surface in `fuel-cpu-backend/src/byte_kernels.rs`
plus the trait-chassis math that backs the elementwise/reduction families
(`fuel-cpu-backend/src/chassis/{binary,unary,compare,reduction}.rs`).
This is the inventory that drives the per-kernel contracts. It is the
`CpuStorageBytes` path that the production dispatch wrapper
(`fuel_dispatch::dispatch::cpu_wrappers`) extracts and calls.
Out of scope (intentionally excluded — different surface, being retired):
- `src/ops.rs` / `src/dyn_impl.rs` — the **legacy** `HostBuffer`-based eager
`CpuStorage` (Candle-derived `Map1/Map2/StridedIndex`, layout-aware). This is
the eager path being retired, not the byte-kernel surface.
- `src/conv2d.rs` — `#[allow(dead_code)]`, not wired to dispatch.
- `src/quantized.rs` (`CpuQStorage`), `src/accelerate.rs`, `src/mkl.rs` (FFI
re-exports), `src/host_storage`, `src/byte_storage`, `src/system_memory.rs`,
`src/probe.rs` (storage / device / FFI plumbing, not compute kernels).
## Cross-cutting facts (apply to nearly every byte kernel)
- **Layout: contiguous-only, zero-offset, row-major.** With one exception
(`contiguize_cpu`, which is the strided→contiguous materializer itself),
every kernel operates on flat `CpuStorageBytes` slices via `as_slice()` /
`bytes()` and validates *byte length* against the declared shape. None of
them consult a `Layout`/strides/offset internally. The pipelined executor's
auto-Contiguize pass is responsible for realizing strided/broadcast/offset
inputs into contiguous buffers *before* these kernels run. So the input-layout
contract for all of them is: **contiguous, offset 0** (the buffer base *is*
element 0). Shape/stride semantics are passed as explicit `usize`
shape/`outer/dim/inner` parameters, not derived from a layout object.
- **Output behavior: pre-allocated, fully overwritten.** Output buffers are
caller-allocated to the exact byte size; kernels overwrite (no read of prior
contents) unless they are explicitly accumulate/seed kernels (`index_add_*`,
`scatter_add_*` copy `base`→`out` then accumulate; in-place kernels mutate
the single buffer). Output layout is always contiguous row-major. No aliasing
between inputs and output is assumed except the in-place family.
- **Dtype monomorphization.** Most families exist as one logical kernel with a
dtype list `{f32, f64, bf16, f16}`. Half floats (`bf16`/`f16`) are computed
by widening to **f32**, doing the math, narrowing back — *except* `f64`-bundle
kernels (selective_scan / ssd / fused_softmax_cross_entropy) that promote to
**f64**. Reductions/cumsum/log-softmax use an **f32 (or f64) accumulator** for
half floats even though the I/O dtype is half (load-bearing precision
invariant encoded in the chassis `ReduceOp::Acc` associated type).
- **Validation is build/runtime byte-length checks** returning `Result` (never
panics on the production path); errors carry the kernel name + expected sizes.
---
## Elementwise binary (chassis::binary)
One kernel `binary<T,U>` (`chassis/binary.rs:78`); per-(op,dtype) public thunks
generated by `binary_thunk!` (`byte_kernels.rs:54`).
- **Ops**: Add, Sub, Mul, Div, Maximum, Minimum, Pow, Rem.
- **dtypes**: f32, f64, bf16, f16 (half via f32 round-trip).
- **Layout**: contiguous-only; validates `lhs.len_bytes()==rhs==out`. Pure
positional walk `out[i]=op(lhs[i],rhs[i])` — **no broadcasting** (inputs must
already be same-shape contiguous).
- **Output**: same dtype as inputs, same element count, contiguous, overwrite.
- **Precision/semantics**: Div→IEEE inf/NaN; Max/Min follow `f32::max/min` NaN-
as-missing; Pow=`powf` (IEEE NaN); Rem=PyTorch `a - floor(a/b)*b` (sign
follows divisor). Half floats compute in f32.
- Thunks: `add/sub/mul/div_{f32,f64,bf16,f16}`, `maximum/minimum_{f32,f64,bf16,f16}`,
`pow_{f32,f64,bf16,f16}`, `rem_{f32,f64,bf16,f16}`. Source `byte_kernels.rs:68-76,144-145,160-165,186-191,374-396,3101-3102`.
## Elementwise unary (chassis::unary)
One kernel `unary<T,U>` (`chassis/unary.rs:94`); thunks via `unary_thunk!`
(`byte_kernels.rs:93`).
- **Ops**: Relu, Neg, Sqr, Sqrt, Recip, Abs, Tanh, Exp, Log, Sin, Cos, Sigmoid,
Silu, Step, Floor, Ceil, Round (round-half-to-even), Sign, Erf, GeluErf,
GeluTanh (`gelu_*`), Rsqrt.
- **dtypes**: f32, f64, bf16, f16.
- **Layout**: contiguous-only; validates `in.len_bytes()==out`. Walk
`out[i]=op(in[i])`.
- **Output**: same dtype, same count, contiguous, overwrite.
- **Precision**: half via f32. Erf/GeluErf via `libm::erf{,f}`. GeluTanh uses a
7-digit √(2/π) const for f32 and 16-digit for f64. Sign(0)=0. Round =
`round_ties_even` (banker's). Source `byte_kernels.rs:110-207,335-388,7594-7595`.
## Elementwise compare (chassis::compare)
One kernel `compare<T,U>` (`chassis/compare.rs:94`); thunks via `compare_thunk!`
(`byte_kernels.rs:233`).
- **Ops**: Eq, Ne, Lt, Le, Gt, Ge.
- **input dtypes**: f32, f64, bf16, f16. **output dtype: U8 always** (1 byte per
element: 1 if predicate holds else 0).
- **Layout**: contiguous-only; validates *element counts* (not bytes — output is
narrower). `out[i]=pred(lhs[i],rhs[i]) as u8`. No broadcasting.
- **Output**: U8, same element count, contiguous, overwrite.
- **Precision/semantics**: IEEE NaN — Ne(NaN,NaN)=1, all others involving NaN=0.
- Thunks `eq/ne/lt/le/gt/ge_{f32,f64,bf16,f16}_u8`. Source `byte_kernels.rs:247-275`.
## Where / ternary select (`where_kernel!`, byte_kernels.rs:288)
- **Op**: `out[i] = if cond[i]!=0 { a[i] } else { b[i] }`.
- **dtypes**: a/b/out f32, f64, bf16, f16; **cond is U8**.
- **Layout**: contiguous-only; validates 4 buffers same element count. No
broadcasting. Output same dtype as a/b, overwrite. Source `byte_kernels.rs:320-323`.
## In-place unary (`unary_inplace_thunk!`, byte_kernels.rs:2822)
- **Ops**: Relu, Silu, Gelu(Tanh), Tanh, Sigmoid, Neg, Abs, Sqr, Sqrt, Rsqrt,
Recip, Exp, Log, Sin, Cos, Sign, Floor, Ceil, Round, Erf, GeluErf.
- **dtypes**: f32, f64, bf16, f16.
- **Layout**: single `&mut CpuStorageBytes`; `out[i]=op(out[i])` **in place**
(aliases input==output). Same numerics as the non-in-place cousins (reuses
`UnaryOpCore`). Source `byte_kernels.rs:2834-2939`.
## In-place affine / clamp / powi (byte_kernels.rs:2946-3062)
- `affine_inplace_{f32,f64,bf16,f16}` — `out=mul*out+add` (half params arrive
as f64). `clamp_inplace_{f32,f64,bf16,f16}` — rejects min>max. `powi_inplace_{f32,f64,bf16,f16}`.
- **Layout**: in-place single buffer; scalar params; contiguous.
## Affine / clamp / powi / powi_backward (out-of-place)
- `affine_{f32,f64,bf16,f16}` (`2795,3105,affine_half_kernel!:3172-3173`):
`out=mul*in+add`; covers AddScalar (mul=1) and MulScalar (add=0). half params f32.
- `clamp_{f32,f64,bf16,f16}` (`3065,3120,3200-3201`): `clamp(in,min,max)`,
rejects min>max.
- `powi_{f32,f64,bf16,f16}` (`3087,3138,3221-3222`): `in.powi(exp)` (i32 exp).
- `powi_backward_{f32,f64,bf16,f16}` (`3230,3250,3293-3294`): two inputs (x,
upstream), `out=exp*x^(exp-1)*upstream`.
- **Layout**: contiguous-only, in/out same element count, overwrite. half via f32.
## Cast (`cast_kernel!`, `cast_kernel_to/from_fp8!`)
One logical "cast" op but **distinct directional kernels** (different in/out
dtype pairs, different element widths → output byte size differs from input).
- f32↔f64, f32↔bf16, f32↔f16 (`cast_kernel!`, byte_kernels.rs:3434-3469).
- f32↔F8E4M3, bf16↔F8E4M3, f16↔F8E4M3 (`cast_kernel_to/from_fp8!`,
3535-3570; F8E4M3 is 1 byte, handled via `from_bits/to_bits`, half↔fp8 pivots
through f32).
- **Layout**: contiguous-only; validates input bytes are a multiple of the input
elem size, output bytes == elem_count × out_elem_size.
- **Output**: **output dtype = the cast target** (differs from input); element
count preserved, byte size changes; contiguous overwrite.
- **Precision**: widening (f32→f64, bf16/f16/fp8→f32) lossless; narrowing lossy
per IEEE rounding.
## MatMul (float / int / quantized)
- `matmul_f32` (3594, hand-written), `matmul_f64` (3973, native macro mirror),
`matmul_{bf16,f16}` (`matmul_half_kernel!`:3716, **f32 accumulator**, narrow
on store), `matmul_{i8,u8}` (`matmul_int_kernel!`:3843, **i32 accumulator**,
**saturating** cast to T on store).
- **op_params**: `lhs_batch_dims`, `rhs_batch_dims`, `m, n, k`. Per-axis batch
must be equal (n_rep=1) or **GQA-divisible** (lhs>rhs && lhs%rhs==0; rhs slot =
lhs_slot/n_rep). Output batch = lhs batch.
- **Layout**: contiguous row-major, offset 0 (executor Contiguize'd). Inner loop
(i,k,j). Validates lhs/rhs/out byte sizes against batch×m×k / k×n / m×n.
- **Output**: same dtype as inputs (int saturated), shape lhs_batch+[m,n],
contiguous, zero-init then accumulate (float) / i32 acc then store (int).
### Quantized matmul (`qmatmul_*`)
- `qmatmul_q4_0_f32`, `qmatmul_q8_0_f32`, `qmatmul_q4_k_m_f32` (explicit
4202-4250) + `qmatmul_thin_wrapper!`: q4_1, q5_0, q5_1, q8_1, q2k, q3k, q5k,
q6k (4272-4279). All delegate to `qmatmul_generic_f32::<GgmlType>` (4125) →
`fuel_quantized::matmul`.
- **dtypes**: **activations F32, output F32; weights are GGML block bytes**
(reinterpreted via `block_slice_from_bytes` — requires block-type alignment).
- **op_params**: `batch_count, m, n, k`; k must be a multiple of the block's
BLCK_SIZE. weight is `[n, k/block]` blocks.
- **Layout**: contiguous; activations row-major; weight block-packed.
- **Output**: F32, `[batch,m,n]`, contiguous overwrite.
### NF4 matmul (`nf4_matmul_native!`, 5934)
- `nf4_matmul_{f32,f16,bf16}` (5988-5990).
- **dtypes**: activations/out in T (f32/f16/bf16, dequant+accumulate in **f32**);
`w_packed` U8 (2 nibbles/byte, NF4 LUT), `absmax` F32 per block.
- **op_params**: `batch, m, n, k, block_size`; k even & multiple of block_size;
w_packed=`n×k/2` bytes; absmax=`n×(k/block_size)` f32.
- **Layout**: contiguous. **Output** T, `[batch,m,n]`, overwrite.
## FusedLinear (`fused_linear_native_kernel!`/`_half_kernel!`)
- `fused_linear_{f32,f64}` (4977-4978), `_{bf16,f16}` (5049-5050, f32 acc).
- = matmul + bias-add: `out[...,i,j]=bias[j]+Σ_k a·b`. Same GQA batch contract
as matmul; bias is `[N]` broadcast over batch×M.
- **Layout**: contiguous. Output T, `[...,M,N]`, overwrite (seeded with bias).
## Reductions
### Per-axis reduce (chassis::reduction `reduce`, reduction.rs:282)
- Ops Sum, Mean, Max, Min × dtypes f32, f64, bf16, f16 → 16 thunks
`{sum,mean,max,min}_reduce_{f32,f64,bf16,f16}` (byte_kernels.rs:7306-7468).
- **op_params**: `input_shape`, `reduce_dims` (sorted-ascending+unique, in
range). Output = input with reduce_dims removed.
- **Layout**: contiguous row-major. **Acc**: f32/f64 = T; bf16/f16 = **f32**
(precision invariant). Mean rejects count==0.
- **Output**: same dtype as input, reduced shape, contiguous overwrite.
### Reduce-to-broadcast-target (chassis::reduction `reduce_to`, reduction.rs:323)
- `reduce_sum_to_{f32,f64,bf16,f16}` (4742-4789), `reduce_max_to_{f32,f64,bf16,f16}`
(4792-4838). Used by `Op::ReduceSumTo`/`ReduceMaxTo` backward of broadcast.
- **op_params**: `input_shape`, `output_shape` (left-padded to input rank; each
axis == input size or 1). **Output** same dtype, `output_shape`, overwrite.
### Argmax/Argmin (byte_kernels.rs:7486-7587)
- `argmax_dim_f32`, `argmin_dim_f32` (one logical op via `argextremum_dim_f32`).
- **input dtype f32; output dtype U32** (index of extremum along `dim`).
- **op_params**: `input_shape`, `dim` (output drops `dim`; dim size 0 rejected).
- Tie-break = first/lowest index. NaN per IEEE. Contiguous; overwrite.
## Softmax / norms (last-dim) — forward
- `softmax_last_dim_{f32,f64,bf16,f16}` (2259, 2430, `softmax_last_dim_half!`:2425-2426):
numerically stable (row-max subtract, exp, normalize). half compute f32.
- `log_softmax_last_dim_{f32,f64,bf16,f16}` (`log_softmax_last_dim_kernel!`:951 /
1002,1008,1041,1045): row-max + log-sum-exp. f32/f64 native acc; half f32 acc.
- `rms_norm_last_dim_{f32,f64,bf16,f16}` (2339, 2559, `rms_norm_last_dim_half!`:2554-2555):
`x/sqrt(mean(x²)+eps)`; **no affine params**; `eps:f64`.
- `layer_norm_last_dim_{f32,f64,bf16,f16}` (2477, 2660, `layer_norm_last_dim_half!`:2655-2656):
`(x-μ)/sqrt(var+eps)`; **no affine**; `eps:f64`.
- **op_params (all)**: `outer_count`, `last_dim` (rows×cols flat). norms add `eps`.
- **Layout**: contiguous; row = last_dim. Output same dtype/shape, overwrite.
## Softmax / norm / reduce — backward kernels
- `softmax_last_dim_backward_{f32,f64,bf16,f16}` (7610,7642,`_half!`:7708-7709):
inputs (y,g) → `y*(g - Σ(y*g))` per row; params outer_count,last_dim.
- `log_softmax_last_dim_backward_{f32,f64,bf16,f16}` (1095,1100,1130,1134):
inputs (y=fwd out, g) → `g - exp(y)*Σg`; params outer_count,last_dim.
- `layer_norm_last_dim_backward_{f32,f64,bf16,f16}` (7716,7769,`_half!`:7872-7873):
inputs (x,g)+eps; recomputes stats.
- `rms_norm_last_dim_backward_{f32,f64,bf16,f16}` (7879,7920,`_half!`:8006-8007):
inputs (x,g)+eps.
- `reduce_max_to_backward_{f32,f64,bf16,f16}` (8026,8050,`_half!`:8106-8107):
inputs (x,upstream)+input_shape,output_shape.
- `powi_backward_*` — see affine section.
- **Layout**: contiguous; half promote→f32→narrow; output = grad of the named
input, same dtype/shape, overwrite.
## Indexing / gather / scatter
- `index_select_cpu` (1788; `index_select_f32` shim 1850): **dtype-agnostic
bytes** (`dtype_size`), U32 indices. params outer,source_dim,n_indices,inner.
Out-of-bounds index → Error.
- `gather_cpu` (2132; `gather_f32` shim 2234): dtype-agnostic bytes, U32 indices,
N-D, `dim`. source/output same rank, differ only on `dim`.
- `index_add_{f32,f64}` (1255 explicit f32 / `index_add_native_kernel!`:1397 f64),
`index_add_{bf16,f16}` (`index_add_half_kernel!`:1473-1474, f32 acc add). U32
indices. **Seeds out from `base`** then accumulates `out[...,idx[i],...]+=src`.
params outer,base_dim,n_indices,inner.
- `scatter_add_{f32,f64}` (1682 / `scatter_add_native_kernel!`:1575),
`scatter_add_{bf16,f16}` (`scatter_add_half_kernel!`:1673-1674).
- **Layout**: contiguous; indices U32 contiguous. **Output**: index_select/gather
= source dtype, overwrite; index_add/scatter_add = base dtype, base-seeded
then accumulated (not pure overwrite).
## Shape/layout movement (dtype-agnostic byte kernels)
- `contiguize_cpu` (1206): **the only strided/broadcast/offset-capable kernel** —
consumes a `Layout`, iterates `layout.strided_index()` (stride 0 = broadcast
replicate, non-zero offset supported), copies `dtype_size` bytes per element to
a fresh contiguous output. Returns a new `CpuStorageBytes`.
- `flip_cpu` (408), `roll_cpu` (444): dtype-agnostic byte reorder along one dim;
params outer,dim_size,inner,(shift),dtype_size. Allocate scratch then copy out.
- `concat_cpu` (2711; `concat_f32` shim 2777): N inputs along one dim;
params outer,input_dim_sizes[],inner,dtype_size.
- `write_slice_cpu` (8300): **in-place rectangular scatter** into a slab of
`dest` defined by `ranges[(start,end)]` per axis; dtype-agnostic; only the slab
bytes are touched (partial overwrite, aliasing dest). params dest_shape,ranges,
dtype_size.
- `write_slice_rotating_cpu` (8453): in-place ring-buffer write; `axis` wraps mod
`modulus`; dynamic start read from `position_bytes` (first u32). **v1 constraint:
axis==0**. Splits across boundary into two `write_slice_cpu`-style copies.
- **Layout**: all contiguous except `contiguize_cpu`. Output dtype = input dtype.
## Padding family
- `pad_const_cpu` (497): multi-dim Constant pad; `fill_bytes` (1 element) tiles
output, then input region copied in. params in_shape,out_shape,padding[(b,a)],
dtype_size,fill_bytes. dtype-agnostic.
- `pad_reflect_cpu` (580) / `pad_replicate_cpu` (595): via `pad_walk_cpu` (631)
with reflect/replicate index maps. dtype-agnostic. Caller must keep
before/after ≤ n-1 for reflect.
- `pad_backward_{f32,f64,bf16,f16}` (`pad_backward_kernel!`:702 / 787-790):
accumulate grad_out into grad_in over the forward mapping; half acc widened to
**f64**. Modes Constant/Reflect/Replicate.
- **Layout**: contiguous; output contiguous overwrite (forward) / accumulate
(backward).
## Triangular / masked / cumsum
- `triangular_cpu` (905): Triu/Tril mask (keep_upper flag, `diagonal` offset);
dtype-agnostic bytes; zeros output then copies kept positions. params
batch_count,rows,cols,diagonal,keep_upper,dtype_size.
- `masked_fill_cpu` (1148): dtype-agnostic; **mask is U8** (count bytes); writes
`fill_bytes` where mask!=0 else copies input. params fill_bytes,dtype_size.
- `cumsum_{f32,f64}` (`cumsum_kernel!`:801 / 836-837), `cumsum_{bf16,f16}`
(840,868, **f32 accumulator**): running prefix sum along one dim; params
outer,dim_size,inner. Per-dtype (typed add). Output same dtype, overwrite.
## RoPE
- `rope_f32` (1885), `rope_f64` (2058), `rope_{bf16,f16}` (`rope_half!`:2053-2054,
f32 compute). rotate_half convention; head_dim must be even (h=head_dim/2).
- **op_params**: outer_count, seq, head_dim. Inputs (x[outer,seq,head_dim],
cos[seq,head_dim], sin[seq,head_dim]); cos/sin broadcast over outer.
- **Layout**: contiguous; output same dtype/shape as x, overwrite.
## Convolution
- `conv2d_f32` (7162, hand-written), `conv2d_f64` (`conv2d_native_kernel!`:4380),
`conv2d_{bf16,f16}` (`conv2d_half_kernel!`:4476-4477, **f32 acc**).
- `conv_transpose2d_{f32,f64}` (`conv_transpose2d_native_kernel!`:4606-4607),
`_{bf16,f16}` (`conv_transpose2d_half_kernel!`:4724-4725, f32 acc).
- **op_params**: x_shape[4]=NCHW, w_shape[4], out_shape[4], stride(h,w),
padding(h,w), dilation(h,w), groups; optional bias `[Cout]`. Group/divisibility
validated.
- **Layout**: contiguous NCHW. **Output** T, NCHW, overwrite (bias-seeded acc).
## FusedSoftmaxCrossEntropy (`fused_softmax_cross_entropy_kernel!`, 5155)
- `fused_softmax_cross_entropy_{f32,f64,bf16,f16}` (5251-5262).
- **dtypes**: logits in T; **targets I64; output always F32**. All math via
**f64** log-sum-exp (f64 lossless, others promote).
- **op_params**: n_rows, vocab, `reduction_tag` (Mean/Sum/None u8), ignore_index.
Output bytes: Mean/Sum=1 f32 scalar, None=n_rows f32. ignore_index rows skipped;
Mean over 0 valid rows = 0; target out-of-range (non-ignore) → Error.
- **Layout**: contiguous. Output F32, overwrite.
## Mamba / SSM kernels
- `causal_conv1d_{f32,f64}` (`causal_conv1d_native_kernel!`:5348), `_{bf16,f16}`
(`causal_conv1d_half_kernel!`:5411). Depthwise 1-D conv + bias + optional fused
SiLU. params batch,channels,seq_in,seq_out,kernel,use_silu; caller left-pads x
(seq_in==seq_out+kernel-1). half compute f32.
- `selective_scan_{f32,f64,bf16,f16}` (`selective_scan_kernel!`:5534 / 5627-5634):
Mamba-1 selective scan; **f64 hidden-state accumulator** regardless of T.
Inputs u,delta,a,b,c; params batch,seqlen,dim,dstate,delta_softplus.
**Multi-output bundled buffer**: out = `[y ; last_state]` concatenated.
- `ssd_chunk_scan_{f32,f64,bf16,f16}` (`ssd_chunk_scan_kernel!`:5724 / 5822-5829):
Mamba-2 SSD scan; f64 accumulator. Inputs x,dt,a,b,c; params batch,seqlen,heads,
head_dim,state_dim,chunk_size (chunk_size must divide seqlen; ABI-compat knob,
result identical for any valid value). Bundled out = `[y ; last_state]`.
- **Layout**: contiguous. Output T, bundled overwrite.
## Attention
- `flash_attn_{f32,f64}` (`flash_attn_native_kernel!`:6021 / 6166-6167),
`_{bf16,f16}` (`flash_attn_half_kernel!`:6171 / 6311-6312). Naive math-def
SDPA (not tiled). q[B,Hq,Sq,D], k/v[B,Hkv,Sk,D] (GQA Hq%Hkv==0), optional
alibi_slopes[Hq]. params b,hq,hkv,sq,sk,d,**k_len** (runtime length over a
capacity Sk),softmax_scale(f32),causal,window_left/right(Option),softcap(Option).
**Output** = q dtype/shape, overwrite.
- `flash_attn_backward_{f32,f64}` (`flash_attn_backward_native_wrapper!`:6625 /
6703-6704), `_{bf16,f16}` (`flash_attn_backward_half_wrapper!`:6707 / 6785-6786).
Inner cores `flash_attn_backward_*_inner` (6468-6469,6612-6613). Inputs q,k,v,
do_grad,alibi; `which:FaBackwardWhich` selects dQ/dK/dV output (single buffer).
- `paged_attn_{f32,f64}` (`paged_attn_native_kernel!`:6805 / 6975-6976),
`_{bf16,f16}` (`paged_attn_half_kernel!`:6979 / 7140-7141). q + paged KV cache
(k_cache/v_cache [num_blocks,block_size,Hkv,D]), block_table U32, context_lens
U32, optional alibi. params b,hq,hkv,sq,d,block_size,max_blocks_per_seq,
num_blocks,softmax_scale,softcap.
- **Layout**: contiguous; output T, overwrite.
---
### Count summary (logical kernels, dtype variants folded)
~70 logical kernels (elementwise binary/unary/compare/where/in-place families
collapsed to one entry each, cast directional kernels and the q-matmul block
variants counted distinctly because they are genuinely different kernels).