Enum TensorOp 

pub enum TensorOp {
    MulMat,
    Add,
    RmsNorm,
    Rope,
    SoftMax,
    Mul,
    Silu,
    Copy,
    None,
}

Tensor operation types for decoder inference.

Each variant maps to ONE kernel dispatch. The goal is to express an entire transformer layer as ~5 operations:

1. RmsNorm (pre-attention)
2. QKV+Attention (fused projection + attention + output projection)
3. Residual add
4. RmsNorm (pre-FFN)
5. FFN (gate+up+swiglu+down fused)
6. Residual add

Variants

MulMat

Matrix-vector multiply (Q4K dequant+GEMV or cuBLASLt GEMM)

Add

Element-wise add (residual connections)

RmsNorm

RMS normalization

Rope

Rotary position embedding

SoftMax

Softmax (attention scores)

Mul

Element-wise multiply (SwiGLU gate)

Silu

SiLU activation

Copy

Memory copy (KV cache scatter)

None

No-op (input tensor, leaf node)

Trait Implementations

impl Clone for TensorOp

fn clone(&self) -> TensorOp

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TensorOp

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TensorOp

fn eq(&self, other: &TensorOp) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for TensorOp

impl Eq for TensorOp

impl StructuralPartialEq for TensorOp