Struct QMatrix 

pub struct QMatrix {
    pub rows: usize,
    pub k: usize,
    pub q: Vec<i8>,
    pub scale: Vec<f32>,
}

A matrix rounded to INT8 row by row, in the layout of the module docs.

Fields

rows: usize

Rows.

k: usize

Values per row.

q: Vec<i8>

[rows.div_ceil(8)][k.div_ceil(8)][4][2][8].

scale: Vec<f32>

max |row| / 127 for each row, so a row is its values times its scale.

Implementations

impl QMatrix

pub fn quantize(x: &[f32], rows: usize, k: usize) -> Self

Rounds x, [rows, k], to INT8.

Panics

If x is not [rows, k].

pub fn dequantize(&self) -> Vec<f32>

The values as f32, [rows, k], for tests and for checking what the rounding lost.

Trait Implementations

impl Clone for QMatrix

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QMatrix

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for QMatrix

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for QMatrix

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for QMatrix