Enum GradOp 

pub enum GradOp {
    Input,
    Parameter,
    Add(usize, usize),
    Sub(usize, usize),
    Mul(usize, usize),
    Div(usize, usize),
    Neg(usize),
    MatMul(usize, usize),
    Sum(usize),
    Mean(usize),
    ScalarMul(usize, f64),
    Exp(usize),
    Ln(usize),
    StructField {
        parent: usize,
        field_index: usize,
        total_fields: usize,
    },
    MapLookup {
        map_node: usize,
        key_index: usize,
        total_keys: usize,
    },
    Sin(usize),
    Cos(usize),
    Sqrt(usize),
    Pow(usize, f64),
    Sigmoid(usize),
    Relu(usize),
    TanhAct(usize),
    Abs(usize),
    Log2(usize),
    Softmax(usize),
    CrossEntropy {
        logits: usize,
        targets: usize,
    },
    LayerNorm(usize),
    BatchNorm(usize),
    Clamp {
        input: usize,
        min: f64,
        max: f64,
    },
    Where {
        cond: usize,
        on_true: usize,
        on_false: usize,
    },
    Reshape {
        input: usize,
        original_shape: Vec<usize>,
    },
    TransposeOp(usize),
    CatOp {
        inputs: Vec<usize>,
        axis: usize,
        sizes: Vec<usize>,
    },
    GatherOp {
        input: usize,
        indices: Vec<usize>,
        axis: usize,
    },
}

Operation recorded in the reverse-mode AD computation graph.

Each variant stores the node indices of its operands so the backward pass can look up parent tensors and propagate gradients.

Variants

Input

External input data (no gradient accumulated).

Parameter

Trainable parameter (gradients are accumulated here during backward).

Add(usize, usize)

Element-wise addition of two nodes.

Sub(usize, usize)

Element-wise subtraction of two nodes.

Mul(usize, usize)

Element-wise (Hadamard) multiplication of two nodes.

Div(usize, usize)

Element-wise division of two nodes.

Neg(usize)

Element-wise negation.

MatMul(usize, usize)

Matrix multiplication of two 2-D nodes.

Sum(usize)

Sum all elements to a scalar [1] tensor.

Mean(usize)

Mean of all elements to a scalar [1] tensor.

ScalarMul(usize, f64)

Multiply every element by a constant scalar.

Exp(usize)

Element-wise exponential.

Ln(usize)

Element-wise natural logarithm.

StructField

Gradient through struct field access: parent node, field index.

Fields

parent: usize

field_index: usize

total_fields: usize

MapLookup

Gradient through map lookup: map node, key index in insertion order.

Fields

map_node: usize

key_index: usize

total_keys: usize

Sin(usize)

Element-wise sine: d/dx sin(x) = cos(x).

Cos(usize)

Element-wise cosine: d/dx cos(x) = -sin(x).

Sqrt(usize)

Element-wise square root: d/dx sqrt(x) = 1/(2*sqrt(x)).

Pow(usize, f64)

Element-wise power with a constant exponent: d/dx x^n = n * x^(n-1).

Sigmoid(usize)

Logistic sigmoid activation: sigma(x) = 1 / (1 + exp(-x)).

Relu(usize)

Rectified linear unit activation: max(0, x).

TanhAct(usize)

Hyperbolic tangent activation: tanh(x).

Abs(usize)

Element-wise absolute value with sub-gradient sign(x) at zero.

Log2(usize)

Base-2 logarithm: d/dx log2(x) = 1/(x * ln(2)).

Softmax(usize)

Softmax over the last axis, producing a probability distribution.

CrossEntropy

Cross-entropy loss between predicted logits and target labels.

Fields

logits: usize

Node index of the raw logit tensor.

targets: usize

Node index of the target (one-hot or class-index) tensor.

LayerNorm(usize)

Layer normalization over the last axis; stores statistics for backward.

BatchNorm(usize)

Batch normalization over the first axis; stores statistics for backward.

Clamp

Element-wise clamping to the range [min, max].

Fields

input: usize

Node index of the input tensor.

min: f64

Lower bound.

max: f64

Upper bound.

Where

Element-wise conditional select using a {0.0, 1.0} mask tensor.

Fields

cond: usize

Node index of the condition mask.

on_true: usize

Node index selected where condition is 1.0.

on_false: usize

Node index selected where condition is 0.0.

Reshape

Reshape a tensor, storing the original shape for backward reconstruction.

Fields

input: usize

Node index of the input tensor.

original_shape: Vec<usize>

Shape before the reshape (used during backward).

TransposeOp(usize)

Transpose a 2-D tensor (swap rows and columns).

CatOp

Concatenate tensors along an axis, storing per-input sizes for backward splitting.

Fields

inputs: Vec<usize>

Node indices of the tensors to concatenate.

axis: usize

Axis along which to concatenate.

sizes: Vec<usize>

Size of each input along the concatenation axis.

GatherOp

Gather elements along an axis by index.

Fields

input: usize

Node index of the source tensor.

indices: Vec<usize>

Indices to gather.

axis: usize

Axis along which to gather.

Trait Implementations

impl Clone for GradOp

fn clone(&self) -> GradOp

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for GradOp

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.