Trait ActivationOps 

pub trait ActivationOps: Send + Sync {
    // Required methods
    fn silu_mul(&self, gate: &TensorRef, up: &TensorRef) -> Result<TensorRef>;
    fn gelu(&self, input: &TensorRef) -> Result<TensorRef>;
}

Activation function operations (including fused variants).

Required Methods

fn silu_mul(&self, gate: &TensorRef, up: &TensorRef) -> Result<TensorRef>

Fused SiLU-multiply: silu(gate) * up.

This is the SwiGLU building block used in LLaMA/Qwen MLPs.

fn gelu(&self, input: &TensorRef) -> Result<TensorRef>

GELU activation.

Implementors