Skip to main content

Q8ActsX4

Struct Q8ActsX4 

Source
pub struct Q8ActsX4 {
    pub na: usize,
    pub n_blocks: usize,
    pub qs: Vec<i8>,
    pub d: Vec<f32>,
}
Expand description

A quad of up to Q8K_ACTS_X4_NC Q8_0 activations, pre-interleaved into the layout llama.cpp’s ggml_quantize_mat_q8_0_4x8 writes into block_q8_0x4 (arch/arm/repack.cpp): every 32-element block’s qs in 8-byte runs, plus the per-block per-row scales. Consumed by the i8mm 4x8 GEMMs; prepared once per matmul, same hoist as Q8KActsX4.

Fields§

§na: usize

Real activations in the quad (≤ 4); rows na..4 are zero padding.

§n_blocks: usize

Q8_0 blocks per activation (n_cols / 32).

§qs: Vec<i8>

Interleaved quants, n_blocks * 128 long. Block b, 8-element run c, quad row a, lane kqs[b*128 + c*32 + a*8 + k] = acts[a].q[b*32 + c*8 + k].

§d: Vec<f32>

Activation scales, n_blocks * 4 long: d[b*4 + a] = acts[a].d[b].

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.