pub enum OpParams {
Show 45 variants
None,
Reduce {
dims: Vec<usize>,
keepdim: bool,
},
Matmul {
lhs_batch_dims: Vec<usize>,
rhs_batch_dims: Vec<usize>,
m: usize,
n: usize,
k: usize,
m_compute: MatmulM,
},
Conv1D(ParamsConv1D),
Conv2D {
x_shape: [usize; 4],
w_shape: [usize; 4],
out_shape: [usize; 4],
stride: (usize, usize),
padding: (usize, usize),
dilation: (usize, usize),
groups: usize,
},
ConvTranspose1D(ParamsConvTranspose1D),
ConvTranspose2D {
x_shape: [usize; 4],
w_shape: [usize; 4],
out_shape: [usize; 4],
stride: (usize, usize),
padding: (usize, usize),
output_padding: (usize, usize),
dilation: (usize, usize),
groups: usize,
},
ReduceSumTo {
input_shape: Vec<usize>,
output_shape: Vec<usize>,
},
ReduceMaxTo {
input_shape: Vec<usize>,
output_shape: Vec<usize>,
},
ReduceMaxToBackward {
input_shape: Vec<usize>,
output_shape: Vec<usize>,
},
FlashAttn {
b: usize,
hq: usize,
hkv: usize,
sq: usize,
sk: usize,
d: usize,
k_len: usize,
softmax_scale: f32,
causal: bool,
window_size_left: Option<usize>,
window_size_right: Option<usize>,
softcap: Option<f32>,
},
PagedAttn {
b: usize,
hq: usize,
hkv: usize,
sq: usize,
d: usize,
block_size: usize,
max_blocks_per_seq: usize,
num_blocks: usize,
softmax_scale: f32,
softcap: Option<f32>,
},
Slice {
dim: usize,
start: usize,
end: usize,
step: usize,
},
Cast,
Affine {
mul: f64,
add: f64,
},
Clamp {
min: f64,
max: f64,
},
PowI {
exp: i32,
},
Concat {
outer_count: usize,
input_dim_sizes: Vec<usize>,
inner_count: usize,
axis: usize,
},
SoftmaxLastDim {
outer_count: usize,
last_dim: usize,
},
NormLastDim {
outer_count: usize,
last_dim: usize,
eps: f64,
},
IndexSelect {
outer_count: usize,
source_dim_size: usize,
n_indices: usize,
inner_count: usize,
},
Gather {
source_shape: Vec<usize>,
output_shape: Vec<usize>,
dim: usize,
},
Rope {
outer_count: usize,
seq: usize,
head_dim: usize,
},
QMatMul {
quant_type: QuantType,
batch_count: usize,
m: usize,
n: usize,
k: usize,
},
IndexAdd {
outer_count: usize,
base_dim_size: usize,
n_indices: usize,
inner_count: usize,
},
ScatterAdd {
base_shape: Vec<usize>,
src_shape: Vec<usize>,
dim: usize,
},
Flip {
outer_count: usize,
dim_size: usize,
inner_count: usize,
axis: usize,
},
Roll {
outer_count: usize,
dim_size: usize,
inner_count: usize,
shift: i64,
axis: usize,
},
CumSum {
outer_count: usize,
dim_size: usize,
inner_count: usize,
axis: usize,
},
Triangular {
batch_count: usize,
rows: usize,
cols: usize,
diagonal: i64,
},
LogSoftmaxLastDim {
outer_count: usize,
last_dim: usize,
},
MaskedFill {
fill_bytes: Vec<u8>,
},
PadBackward {
in_shape: Vec<usize>,
out_shape: Vec<usize>,
padding: Vec<(usize, usize)>,
mode_tag: u8,
},
Pad {
in_shape: Vec<usize>,
out_shape: Vec<usize>,
padding: Vec<(usize, usize)>,
mode_tag: u8,
fill_bytes: Vec<u8>,
},
WriteSlice {
dest_shape: Vec<usize>,
ranges: Vec<(usize, usize)>,
deferred_dyn_offset: Option<(usize, DynScalar)>,
},
WriteSliceRotating {
dest_shape: Vec<usize>,
axis: usize,
modulus: usize,
ranges: Vec<(usize, usize)>,
},
WriteSliceDoff {
dest_shape: Vec<usize>,
axis: usize,
ranges: Vec<(usize, usize)>,
},
FusedSoftmaxCrossEntropy {
n_rows: usize,
vocab: usize,
reduction: Reduction,
ignore_index: i64,
},
CausalConv1d {
batch: usize,
channels: usize,
seq_in: usize,
seq_out: usize,
kernel: usize,
use_silu: bool,
},
SelectiveScan {
batch: usize,
seqlen: usize,
dim: usize,
dstate: usize,
delta_softplus: bool,
},
SsdChunkScan {
batch: usize,
seqlen: usize,
heads: usize,
head_dim: usize,
state_dim: usize,
chunk_size: usize,
},
Nf4Matmul {
batch: usize,
m: usize,
n: usize,
k: usize,
block_size: usize,
},
AwqMatmul {
batch: usize,
m: usize,
n: usize,
k: usize,
group_size: usize,
split_k_iters: i32,
},
JitScalars {
scalars: Vec<f64>,
},
NonZeroIndices {
capacity: usize,
count_sym: SymId,
},
}Expand description
Typed extras bag passed to kernels via KernelRef. One variant
per op family that needs auxiliary parameters; None for
parameter-less ops (most elementwise unary and binary).
Variants§
None
Op needs no auxiliary parameters. Used by elementwise unary (relu, neg, sqr, …), elementwise binary (add, mul, sub, div, …), shape-only ops (reshape, transpose), etc.
Reduce
Reduction (sum, max, mean, …) along specific dims. The input
tensor’s Layout flows through the new layouts side-channel
(layouts[0]) on KernelRef, so this variant carries only the
op-specific extras: which dims to reduce and the keepdim flag.
dims is the sorted list of dims to reduce; keepdim
controls whether reduced dims are retained as size-1 in
the output (today fuel-graph never asks for keepdim, but
the field is reserved for the future).
Matmul
Matrix multiplication. Carries the dimensions explicitly
because Storage only holds bytes + dtype;
the kernel needs the batch shape and (m, n, k) to walk
inputs and outputs.
Shape contract: lhs [..lhs_batch.., m, k] @
rhs [..rhs_batch.., k, n] → out [..lhs_batch.., m, n].
Per-axis the batch dims must either match or follow GQA-style
divisibility (lhs_dim > rhs_dim && lhs_dim % rhs_dim == 0)
— the kernel maps each lhs batch slot to the corresponding
rhs slot via rhs_axis_idx = lhs_axis_idx / n_rep_axis.
Equal-batch case: lhs_batch_dims == rhs_batch_dims. Rank-2
case: both batch vectors are empty. Both work uniformly.
Inputs are guaranteed contiguous by the executor’s
auto-Contiguize pass; transpose flags don’t appear here
because Op::Transpose is its own metadata-only op in
fuel-graph.
Fields
Conv1D(ParamsConv1D)
1D convolution geometry (forward path).
Conv2D
2D convolution geometry (forward path). Carries the tuple-shaped
stride/padding that fuel-graph’s Op::Conv2D uses (asymmetric
supported), plus the input/weight/output shapes the kernel
needs to walk the multi-index. Storage holds only bytes +
dtype, so spatial shapes flow through OpParams.
Inputs: x shape [N, Cin, Hin, Win], weight shape
[Cout, Cin/groups, Kh, Kw], optional bias shape [Cout].
Output: [N, Cout, Hout, Wout].
Fields
ConvTranspose1D(ParamsConvTranspose1D)
1D transposed-convolution geometry.
ConvTranspose2D
2D transposed-convolution geometry. Mirrors OpParams::Conv2D
in shape (inline fields, asymmetric stride/padding/dilation,
groups), with the additional output_padding parameter that
transposed conv needs to disambiguate output spatial size.
Inputs: x shape [N, Cin, Hin, Win], weight shape
[Cin, Cout/groups, Kh, Kw] (note transposed channel order
vs Conv2D), optional bias shape [Cout].
Output: [N, Cout, Hout, Wout].
Fields
ReduceSumTo
Sum-reduce a tensor to a smaller broadcast-compatible target
shape. The kernel left-pads output_shape with 1s to match
input_shape’s rank, then for each axis: if the padded output
dim equals the input dim that axis carries through; if it’s 1
the axis is summed away.
ReduceMaxTo
Max-reduce a tensor to a smaller broadcast-compatible target
shape — the max-symmetric counterpart of ReduceSumTo. Same
axis-alignment rules; per-axis reduction is max instead of
+.
ReduceMaxToBackward
Backward of ReduceMaxTo. Inputs: (x, upstream) where
x.shape == input_shape and upstream.shape == output_shape.
Output: grad_x of input_shape. The kernel recomputes the
forward max via input_shape → output_shape axis alignment,
builds a tie-count mask, and routes upstream back to argmax
positions (fair-share on ties).
FlashAttn
Multi-head scaled-dot-product attention shape + math params.
q is [B, Hq, Sq, D], k and v are [B, Hkv, Sk, D]
(Hkv ≤ Hq, GQA-divisible). Optional 4th input alibi_slopes
has shape [Hq] (presence is implicit in inputs.len() == 4).
sk is the physical K/V length axis (the allocated extent —
used for strides + byte-length checks). k_len is the
logical attended length (k_len <= sk): a kernel attends only
the first k_len rows of the K/V buffers and bottom-right-aligns
the causal mask at offset k_len - sq (Phase D symbolic extents:
flash over a fixed-capacity KV-cache with a runtime live prefix).
For the static path the lowering sets k_len == sk, so a kernel
that loops 0..k_len with the k_len - sq causal offset is
byte-identical to the old 0..sk / kj > qi form.
Fields
PagedAttn
Paged-cache attention. q is [B, Hq, Sq, D]; k_cache and
v_cache are [num_blocks, block_size, Hkv, D]. The 4th input
is block_table: [B, max_blocks_per_seq] (U32 — physical block
index per logical position). The 5th is context_lens: [B]
(U32 — true context length per sequence). Optional 6th is
alibi_slopes: [Hq].
Fields
Slice
Slice along a single dim with explicit start/end/step. The dim is implicit in the input Layout’s relabeling for multi-dim slice; this variant covers the simple case.
Cast
Cast input dtype → target dtype. The target lives on the
output Storage’s dtype field; this variant signals the
op family without requiring a re-read.
Affine
Affine transformation y = mul * x + add. Used by
Tensor::affine / scale_and_shift.
Clamp
Element-wise clamp: y = clamp(x, min, max).
PowI
Element-wise integer power: y = x.powi(exp).
Concat
Concatenate N inputs along one dim. The kernel needs the
outer/inner element counts (product of dims before/after
the concat dim) plus each input’s size along the concat
dim — that’s all that distinguishes a concat from a
sequence of slab-copies. Order matches Node::inputs.
axis is the original concat dim in the output’s shape;
kernels that want to walk strided rank-N inputs need it
to build the per-axis stride mask (the outer/inner factoring
alone loses the position info).
Fields
SoftmaxLastDim
Softmax along the last dim. The kernel walks
outer_count rows of last_dim elements each.
NormLastDim
Last-dim norm parameters shared by RMS-norm and LayerNorm (no affine flavor for both today). The OpKind selects which kernel reads this variant.
IndexSelect
Pick slices along a single dim using a rank-1 U32 index tensor. The kernel needs the outer/inner counts (dims before/after the selected axis), the source’s selected-dim size for index bounds checking, and the index count.
Gather
N-dimensional gather along dim. Source and output shapes
agree on every dim except dim; the indices tensor (U32)
has output_shape and supplies the source coord for dim
at every output position.
Rope
Rotary position embedding parameters. The kernel walks
outer_count × seq × head_dim elements; cos/sin
have [seq, head_dim] shape and broadcast across the
outer dims.
QMatMul
Quantized matmul shape: A [batch, m, k] @ dequant(W) [n, k] → out [batch, m, n]. The weight tensor is a flat U32-typed
byte stream representing n * k / elements_per_block blocks
of quant_type. Activations are F32 today; output is F32.
IndexAdd
Index-add along a single dim with rank-1 U32 indices.
Output is base with src[..., i, ...] accumulated into
base[..., indices[i], ...] for every i ∈ 0..n_indices.
ScatterAdd
N-dimensional scatter-add. Indices and src share the same
shape; base may differ only along dim. The kernel walks
every src/indices position, reads indices[p] for the
destination’s dim coord, and accumulates src[p] into
base at that destination.
Flip
Flip the order of elements along one dim. The flat-3-axis
view (outer × dim × inner) lets the kernel walk a tight loop
without re-deriving the axis split per call. axis is the
original dim index in the input’s rank-N shape — needed by
stride-aware kernels to build a per-axis flip mask.
Roll
Cyclic shift along one dim by shift positions (positive
shifts move elements to higher indices, wrapping around).
Same flat-3-axis view as Flip. shift is signed: negative
shifts move elements to lower indices.
CumSum
Running cumulative sum along one dim. Same flat-3-axis view
as Flip/Roll. Output is always the same dtype as input;
kernel needs typed addition so it’s per-dtype (unlike Flip/Roll).
Triangular
Triangular mask parameters (used by both Triu and Tril — the
op-kind picks the direction). batch_count is the product of
leading dims; rows/cols are the last two dims. diagonal
is signed (0 = main diagonal, positive shifts up, negative
shifts down).
LogSoftmaxLastDim
LogSoftmax along the last dim. Walks outer_count rows of
last_dim elements each. Per-dtype kernel (uses log/exp).
MaskedFill
MaskedFill: per-element fill where mask is nonzero. The kernel
reads the element count from the layout; fill_bytes is
pre-encoded in the output’s dtype (one element’s worth).
PadBackward
Backward helper for Pad. Carries the input shape so the kernel can size its scatter-add buffer; the output shape is implicit from the input shape + padding.
Pad
Multi-dim Pad: per-axis (before, after) plus a mode_tag
(0=Constant, 1=Reflect, 2=Replicate) and pre-encoded
fill_bytes for Constant fill. Dtype-agnostic at the byte
level: the kernel just copies bytes per the input/output shapes,
which is why fill is bytes (already encoded in the output’s
dtype) rather than f64 (which would force the kernel to
know its dtype).
in_shape.len() == out_shape.len() == padding.len(), and
out_shape[i] == in_shape[i] + padding[i].0 + padding[i].1.
fill_bytes.len() == dtype_size_in_bytes (one element’s worth).
Fields
WriteSlice
In-place scatter write parameters. The destination shape +
source shape together determine the kernel walk; ranges
gives the per-axis (start, end) slab inside the destination.
dest_shape[i] >= ranges[i].1, and source shape is implicitly
ranges[i].1 - ranges[i].0 along axis i.
Phase E.3.2: backs Op::WriteSlice for persistent KV-cache
writes.
deferred_dyn_offset is the data-determined dynamic-shapes seam
(increment 2b). Normally None — ranges is already resolved at
compile (static, or an input-determined dyn_offset bound in the
per-pass SymEnv). It is Some((axis, sym)) only when the offset is
data-determined — its SymId is UNBOUND at compile because a
producer (e.g. Op::NonZeroIndices) computes it mid-pass. The
WorkItemKind::WriteSlice executor arm resolves it from
produced_syms and overrides ranges[axis] to (base, base+width)
just before the kernel runs; the kernel wrappers ignore this field.
Fields
WriteSliceRotating
In-place ring-buffer scatter write parameters. Like
WriteSlice except axis wraps modulo modulus; the
dynamic start on that axis is read from an extra input (rank-0
U32). On the rotating axis, ranges[axis].0 is ignored
(dynamic) and ranges[axis].1 - ranges[axis].0 is the slab
width (must equal the source dim on that axis, and must not
exceed modulus).
Phase C: backs Op::WriteSliceRotating for sliding-window
KV caches.
WriteSliceDoff
In-place scatter write whose start on axis is read from a
device-resident rank-0 I64 input (inputs[2]) at kernel
launch — the CUDA-graph-capturable KV-cache append. Like
WriteSlice except ranges[axis].0 is a placeholder overridden
device-side by the offset; no modulo wrap. The CUDA wrapper
passes the offset buffer’s device pointer straight through as
dyn_start_dev (NO D2H — that would break capture); the CPU
wrapper reads it host-side and delegates to write_slice_cpu.
Bounds (offset + width <= dest_shape[axis]) are the caller’s
contract; the kernel does not clamp.
CapturedRun (2026-07): backs Op::WriteSliceDoff for
DecodeSession decode-step capture/replay.
FusedSoftmaxCrossEntropy
FusedSoftmaxCrossEntropy execution parameters. Inputs:
logits [n_rows, vocab] (F32, flattened from the original
[..., V] shape) and targets [n_rows] (I64). The kernel
walks row-by-row, computing stable log-softmax + NLL +
ignore_index masking + the requested reduction in a single
pass, allocating only an [n_rows] per-row accumulator (and
a scalar for Mean/Sum). Output is F32 scalar [] for
Mean/Sum, F32 [n_rows] for None.
n_rows and vocab are derived at translate time from the
logits layout; the kernel uses them to iterate without
re-parsing shapes.
CausalConv1d
CausalConv1d execution parameters. Inputs:
x [batch, channels, seq_in] (pre-padded by caller with
kernel - 1 left zeros), weight [channels, 1, kernel],
bias [channels]. Output [batch, channels, seq_out] where
seq_out = seq_in - (kernel - 1). All tensors share dtype.
Carries both seq_in and seq_out so the kernel walks the
time axis without re-deriving from shapes. use_silu toggles
the fused SiLU activation on the output store (matches
baracuda’s causal_conv1d_*_run signature flag).
SelectiveScan
SelectiveScan execution parameters. 5 required inputs:
u [batch, seqlen, dim], delta [batch, seqlen, dim],
a [dim, dstate], b [batch, seqlen, dstate],
c [batch, seqlen, dstate]. Output y [batch, seqlen, dim].
All tensors share dtype.
delta_softplus toggles applying softplus(delta) before use
(matches baracuda’s selective_scan_*_run flag).
SsdChunkScan
SsdChunkScan execution parameters. 5 inputs:
x [batch, seqlen, heads, head_dim],
dt [batch, seqlen, heads], a [heads],
b [batch, seqlen, heads, state_dim],
c [batch, seqlen, heads, state_dim]. Output y matches x’s
shape. All tensors share dtype.
chunk_size is the SSD block size (GPU-parallelism knob).
On CPU the kernel runs a sequential scan regardless;
validation requires chunk_size > 0 and
seqlen % chunk_size == 0.
Nf4Matmul
Nf4Matmul execution parameters. 3 inputs:
activations [batch, m, k] (rank ≥ 2; leading dims flattened
into batch), w_packed [n, k/2] U8, absmax [n, k/block_size]
F32. Output [batch, m, n] matching activations’ dtype.
block_size is the per-output-row, per-block scale granularity
(typically 64 in bitsandbytes). k must be even (w_packed
layout requirement) and a multiple of block_size.
AwqMatmul
AwqMatmul execution parameters. 4 inputs: activations [..., m, k]
(rank ≥ 2; every leading dim is flattened into batch, so the CUDA
wrapper runs a GEMM of batch * m rows), weight [n, k/8] U32,
zeros [n, zeros_w] U32, scales [n, 8 * zeros_w] F16 (baracuda W4A16
kernel layout). Output [batch, m, n] matching activations’ dtype.
group_size ∈ {64, 128}; split_k_iters is the CUDA launch-tuning
parameter (typically 8).
JitScalars
A runtime fused op’s live scalar args (the extract: values from
FusedOpParams::Runtime { scalars }), in the region’s pattern
pre-order — the same canonical slot order the synthesized kernel’s
trailing , float p0, float p1, … launch parameters are declared in
(the synthesizer’s scalar ABI appends them after long long n, always
float-typed regardless of the kernel’s element dtype). Only produced
by compile_one’s is_runtime arm; a param-free runtime op carries
OpParams::None instead.
Fields
NonZeroIndices
NonZeroIndices execution parameters. 1 input x (the value
tensor); one bundled output [indices [capacity] U32 ; count [1] U32] where capacity == x.elem_count(). The kernel writes
the ascending flat indices of x’s nonzero elements into slot 0
(first count entries) and the runtime count into slot 1.
count_sym is the fuel_ir::symbol::SymId the executor binds to the realized
count in the per-pass SymEnv after this op runs — the
data-determined dynamic-shape seam. The kernel itself ignores it.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for OpParams
impl RefUnwindSafe for OpParams
impl Send for OpParams
impl Sync for OpParams
impl Unpin for OpParams
impl UnsafeUnpin for OpParams
impl UnwindSafe for OpParams
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more