pub struct Q8ActsX4 {
pub na: usize,
pub n_blocks: usize,
pub qs: Vec<i8>,
pub d: Vec<f32>,
}Expand description
A quad of up to Q8K_ACTS_X4_NC Q8_0 activations, pre-interleaved
into the layout llama.cpp’s ggml_quantize_mat_q8_0_4x8 writes into
block_q8_0x4 (arch/arm/repack.cpp): every 32-element block’s qs in
8-byte runs, plus the per-block per-row scales. Consumed by the i8mm
4x8 GEMMs; prepared once per matmul, same hoist as Q8KActsX4.
Fields§
§na: usizeReal activations in the quad (≤ 4); rows na..4 are zero padding.
n_blocks: usizeQ8_0 blocks per activation (n_cols / 32).
qs: Vec<i8>Interleaved quants, n_blocks * 128 long. Block b, 8-element run
c, quad row a, lane k ↦
qs[b*128 + c*32 + a*8 + k] = acts[a].q[b*32 + c*8 + k].
d: Vec<f32>Activation scales, n_blocks * 4 long: d[b*4 + a] = acts[a].d[b].
Auto Trait Implementations§
impl Freeze for Q8ActsX4
impl RefUnwindSafe for Q8ActsX4
impl Send for Q8ActsX4
impl Sync for Q8ActsX4
impl Unpin for Q8ActsX4
impl UnsafeUnpin for Q8ActsX4
impl UnwindSafe for Q8ActsX4
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more