Struct Q80Weight 

pub struct Q80Weight<R: Runtime> { /* private fields */ }

A weight matrix resident in VRAM in packed Q8_0 form ([n_out, k], k % 32 == 0, blocks along k).

Implementations

impl<R: Runtime> Q80Weight<R>

pub fn from_gguf_bytes( client: &ComputeClient<R>, data: &[u8], n_out: usize, k: usize, ) -> Result<Self>

Repacks a GGUF Q8_0 tensor onto the device.

pub fn vram_bytes(&self) -> usize

Bytes in VRAM: 36 per 32 weights (9.0 bits/weight).

pub fn matmul_device( &self, client: &ComputeClient<R>, x: Handle, m: usize, ) -> Handle

Device path: launch only, output handle returned. Decode (m == 1) keeps the untiled kernel; prefill (m > 1) takes the shared-memory tiled kernel unless COMBS_NO_TILED_MATMUL=1.

pub fn matmul_host( &self, client: &ComputeClient<R>, x: &[f32], m: usize, ) -> Result<Vec<f32>>

Host-slice convenience for tests.