Module transformer 

Transformer module with full model implementation and weight loading

This module provides:

• Transformer - Complete transformer model for language modeling
• TransformerConfig - Model architecture configuration
• load_safetensors_weights - Load weights from SafeTensors files
• Architecture - Model architecture type for weight mapping
• MultiHeadAttentionWithLoRA - Attention with deep LoRA injection
• LoRAProjection - Linear projection with LoRA adapters

Re-exports

pub use wgpu_block::WgpuForwardPass;

Modules

init

Weight initialization utilities (C-INIT-001).

wgpu_block

wgpu-accelerated transformer forward pass

Structs

BlockWeights

Downloaded weight data from a CUDA transformer block.

CudaGradWorkspace

Shared gradient workspace for weight gradients (one per model, NOT per layer).

CudaNf4TransformerBlock

CUDA-accelerated transformer block with NF4-quantized frozen weights.

CudaTransformerBlock

CUDA-accelerated transformer block

EncoderBlock

Encoder transformer block (BERT/RoBERTa).

EncoderFeedForward

Encoder feed-forward network with GELU activation (BERT/RoBERTa/CodeBERT).

EncoderModel

Complete encoder model (BERT/RoBERTa/CodeBERT).

GpuBlockOptimizerState

GPU-resident AdamW optimizer state for one transformer block.

LayerNorm

Layer Normalization with bias (used by BERT/RoBERTa/CodeBERT encoders).

LearnedPositionEmbedding

Learned absolute position embedding for encoder models (BERT, RoBERTa, CodeBERT).

LoRAProjection

LoRA-enabled linear projection

MultiHeadAttention

Multi-head self-attention layer

MultiHeadAttentionWithLoRA

Multi-head attention with deep LoRA injection

Transformer

Complete transformer model

TransformerConfig

Configuration for transformer models

Enums

Architecture

Architecture type for weight name mapping

CudaBlock

Unified enum for CUDA transformer blocks (fp32 or NF4-quantized).

ModelArchitecture

Model architecture family.

Functions

load_safetensors_weights

Load transformer weights from SafeTensors file(s)

validate_weights

Validate that loaded weights match expected architecture