Enum MatWeight 

pub enum MatWeight {
    F32(Vec<f32>),
    Packed {
        key: String,
        scheme: QuantScheme,
        shape: Vec<usize>,
    },
}

Storage variant for matmul weight tensors. The big projections (qkv / gate / ffn / lm_head) dominate the load footprint; the Packed variant keeps GGUF K-quant bytes in-place so the graph can emit Op::DequantMatMul instead of a full F32 dequant.

Norm weights, conv kernels, scalar params etc. stay as Vec<f32> in the layer structs (their footprint is negligible and the RmsNorm / Conv ops don’t have a packed variant).

Variants

F32(Vec<f32>)

Already dequantized to f32, row-major [out, in]. The builder transposes to [in, out] before issuing MatMul.

Packed

GGUF-packed K-quant metadata only. The actual bytes are looked up in the loader at upload time via rlx_core::weight_loader::GgufLoader::tensor_bytes_borrowed — eliminates the per-tensor Vec<u8> allocation that otherwise costs ~16 GB of memcpy on Qwen3.6-27B Q4_K_M.

key is the loader-resolvable name (post-HF↔GGUF mapping); shape is [out, in] after the safetensors-style dim reversal.

Fields

key: String

scheme: QuantScheme

shape: Vec<usize>

Implementations

impl MatWeight

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn shape(&self) -> &[usize]

[out, in] on-disk shape. For the F32 variant the caller is expected to track this externally (we return an empty Vec).

pub fn is_packed(&self) -> bool

pub fn packed_key(&self) -> Option<&str>

Loader-resolvable key for the packed variant. None for F32.

Trait Implementations

impl Clone for MatWeight

fn clone(&self) -> MatWeight

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MatWeight

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.