Struct LlamaBlock 

pub struct LlamaBlock {
    pub norm1: RMSNorm,
    pub attn: Attention,
    pub norm2: RMSNorm,
    pub mlp: MLP,
}

Pre-norm Llama decoder block: attention and MLP with residuals.

Fields

norm1: RMSNorm

Norm before attention (input_layernorm).

attn: Attention

Self-attention.

norm2: RMSNorm

Norm before the MLP (post_attention_layernorm).

mlp: MLP

Feed-forward.

Implementations

impl LlamaBlock

pub fn forward( &self, engine: &ComputeEngine, input: &Tensor, pos: usize, cache: Option<&KVCacheLayer>, mask: Option<&UnifiedTensor>, ) -> Tensor

Runs the block on input ([batch, seq, hidden]); see Attention::forward for pos, cache and mask.

pub fn params(&self) -> Vec<Tensor>

Trainable tensors (weights, then any LoRA adapters).

pub fn named_params(&self, prefix: &str) -> HashMap<String, Tensor>

Parameters keyed by Hugging Face-style names under prefix.

pub fn lora_params(&self) -> Vec<Tensor>

Only the LoRA adapter tensors (what LoRA fine-tuning updates).