Struct WgpuForwardPass 

pub struct WgpuForwardPass { /* private fields */ }

wgpu-accelerated transformer forward pass

Batches all matmul and activation operations across transformer layers into a single GpuCommandBatch::execute() call.

Current implementation accelerates FFN matmuls (gate/up/down projections) which dominate compute time (~60-70% of forward pass). Attention remains on CPU due to softmax/RoPE complexity (Phase 2 will add GPU attention).

Implementations

impl WgpuForwardPass

pub fn new( config: &TransformerConfig, adapter_index: u32, ) -> Result<Self, String>

Create a new wgpu forward pass from a transformer model

Arguments

• model - Transformer model with loaded weights
• adapter_index - wgpu adapter index to use

Errors

Returns error if GPU device creation fails

pub fn new_default(config: &TransformerConfig) -> Result<Self, String>

Create from default GPU adapter

pub fn with_resident_weights(model: &Transformer) -> Result<Self, String>

Create from model with GPU-resident FFN weights (KAIZEN-015).

Uploads all FFN weights (gate, up, down projections for every layer) to GPU memory at construction time. Subsequent forward passes use import_buffer() to reference these persistent buffers, eliminating the per-call H2D transfer overhead.

For Qwen3-4B (36 layers, hidden=3584, intermediate=18944):

• Per-layer: 3 * 3584 * 18944 * 4 bytes = ~775 MB
• Total: 36 layers * 775 MB = ~27.2 GB GPU memory (once)
• Savings: 14.6 GB/step H2D transfer eliminated

Contract (C-WGPU-RESIDENT-001)

• Precondition: Model has valid FFN weights for all layers
• Postcondition: All FFN weights GPU-resident, zero H2D transfers during forward
• Invariant: Weights are frozen (read-only) – LoRA adapters remain CPU-resident

pub fn forward_hidden( &self, model: &Transformer, token_ids: &[u32], ) -> Result<Tensor, String>

Execute forward pass through all transformer layers on GPU

Batches FFN matmuls (gate/up/down projections) per layer. Attention and normalization remain on CPU for this phase.

Arguments

• model - Transformer model with weights
• token_ids - Input token IDs

Returns

Hidden states tensor (seq_len * hidden_size)

pub fn forward_hidden_batch( &self, model: &Transformer, batch_token_ids: &[Vec<u32>], lora_layers: Option<&[LoRALayer]>, ) -> Result<Vec<Tensor>, String>

Execute batched forward pass for multiple samples (KAIZEN-008).

Processes all samples through each layer together, uploading FFN weights ONCE per layer instead of once per sample. With batch_size=20 × 36 layers, this reduces weight uploads from 720 to 36 (20× reduction, ~146 GB saved).

Attention remains per-sample on CPU SIMD. FFN inputs are concatenated across samples for a single large matmul per layer.

KAIZEN-010: LoRA integration

When lora_layers is provided, attention uses forward_with_lora() to apply LoRA adjusts to Q and V projections. Layout: [Q_0, V_0, Q_1, V_1, ...] (2 LoRA layers per transformer layer). Without this, only the classifier head trains on the wgpu path (5,122 params vs 5.9M with LoRA).

pub fn adapter_info(&self) -> String

Get the adapter info for display