Struct Qwen2MLP 

pub struct Qwen2MLP { /* private fields */ }

Qwen2 MLP with SwiGLU activation.

output = down_proj(SiLU(gate_proj(x)) * up_proj(x))

Implementations

impl Qwen2MLP

pub fn new(hidden_size: usize, intermediate_size: usize) -> Qwen2MLP

Create a new Qwen2 MLP layer.

pub fn placeholder(hidden_size: usize, intermediate_size: usize) -> Qwen2MLP

Create a placeholder MLP with minimal memory allocation.

Used for lazy initialization when loading pre-trained weights.

pub fn forward(&self, x: &Tensor) -> Tensor

Forward pass with SwiGLU activation.

pub fn gate_proj_mut(&mut self) -> &mut Linear

Get mutable reference to gate projection layer.

pub fn up_proj_mut(&mut self) -> &mut Linear

Get mutable reference to up projection layer.

pub fn down_proj_mut(&mut self) -> &mut Linear

Get mutable reference to down projection layer.

Trait Implementations

impl Debug for Qwen2MLP

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.