Struct Q4KWeight 

pub struct Q4KWeight<R: Runtime> { /* private fields */ }

A weight matrix resident in VRAM in packed Q4_K form ([n_out, k], k % 256 == 0, superblocks along k).

Implementations

impl<R: Runtime> Q4KWeight<R>

pub fn from_gguf_bytes( client: &ComputeClient<R>, data: &[u8], n_out: usize, k: usize, ) -> Result<Self>

Repacks a GGUF Q4_K tensor onto the device.

pub fn vram_bytes(&self) -> usize

Bytes in VRAM: 148 per 256 weights (4.63 bits/weight).

pub fn matmul_device( &self, client: &ComputeClient<R>, x: Handle, m: usize, ) -> Handle

Device path: launch only, output handle returned. Decode (m == 1) keeps the untiled kernel; prefill (m > 1) takes the shared-memory tiled kernel unless COMBS_NO_TILED_MATMUL=1.

pub fn matmul_host( &self, client: &ComputeClient<R>, x: &[f32], m: usize, ) -> Result<Vec<f32>>

y = x @ W^T for host-side x: [m, k], returning [m, n_out].