Struct GpuMatmulCache 

pub struct GpuMatmulCache { /* private fields */ }

PMAT-322: Cached matmul with persistent weight buffers for LLM inference. Cached matmul state: pipeline + pre-uploaded weight buffers + persistent I/O.

PMAT-323: Three levels of buffer persistence:

1. Weight buffers — uploaded once at model init (PMAT-322)
2. I/O buffers — pre-allocated to max size, reused across calls (PMAT-323)
3. Pipeline + bind group layout — created once, reused forever

Implementations

impl GpuMatmulCache

pub fn new(device: Device, queue: Queue) -> Self

Create a new cached matmul context from an existing GpuDevice.

pub fn upload_weight( &mut self, name: &str, data: &[f32], rows: usize, cols: usize, )

Pre-upload a weight matrix (call once at model init). Weight is stored in row-major f32: shape [rows, cols]. Silently skips weights that exceed the device’s max buffer binding size.

pub fn weight_count(&self) -> usize

Number of pre-uploaded weights.

pub fn weight_bytes(&self) -> usize

Total VRAM used by weight buffers (bytes).

pub fn matmul_cached( &mut self, weight_name: &str, input: &[f32], output: &mut [f32], m: usize, ) -> Result<(), String>

PMAT-323: Zero-alloc matmul using persistent I/O buffers. Only creates bind group per call (required by WGPU — bind groups reference specific buffer instances). Everything else is reused.