Struct MLP 

pub struct MLP {
    pub gate_proj: Linear,
    pub up_proj: Linear,
    pub down_proj: Linear,
}

SwiGLU feed-forward: down(silu(gate(x)) * up(x)).

Fields

gate_proj: Linear

Gate projection (passed through SiLU).

up_proj: Linear

Up projection.

down_proj: Linear

Down projection back to the hidden size.

Implementations

impl MLP

pub fn forward(&self, engine: &ComputeEngine, input: &Tensor) -> Tensor

Applies the feed-forward to [tokens, hidden] or [batch, seq, hidden].

pub fn params(&self) -> Vec<Tensor>

Trainable tensors (weights, then any LoRA adapters).

pub fn named_params(&self, prefix: &str) -> HashMap<String, Tensor>

Parameters keyed by Hugging Face-style names under prefix.

pub fn lora_params(&self) -> Vec<Tensor>

Only the LoRA adapter tensors (what LoRA fine-tuning updates).