Struct QuantizedView 

pub struct QuantizedView<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> { /* private fields */ }

View that dequantizes after loads. Scales layout should take values coordinates and map them to the corresponding scale.

Warning

Assumes only one scale maps to a single load. Adjust vector size of values or block size to ensure this. Must ensure block_size.is_multiple_of(vector_size * scheme.num_quants()).

Implementations

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> QuantizedView<'a, Q, NQ, S, F, NF, C>

pub fn new( values: View<'a, Vector<Q, NQ>, C>, scales: View<'a, S, C>, table: ComptimeOption<Box<[f32]>>, scheme: QuantScheme, ) -> Self

A view reading every scale per position through scales.

Takes a one-level scheme: the per-tensor scale of a two-level one never rides a signature here, it is either bound at launch or already held by a caller as a KnownScale passed to new_with_known_scale.

pub fn new_with_known_scale( values: View<'a, Vector<Q, NQ>, C>, scales: View<'a, S, C>, known_scale: KnownScale, table: ComptimeOption<Box<[f32]>>, scheme: QuantScheme, ) -> Self

new with whatever the caller already knows of the scale, in whichever KnownScale form it holds it. Only a caller that knows what its reads share can say so, which is why this exists on the cube side. Reads assert the register agrees with the scheme.

pub fn __expand_new( scope: &Scope, values: <View<'a, Vector<Q, NQ>, C> as CubeType>::ExpandType, scales: <View<'a, S, C> as CubeType>::ExpandType, table: <ComptimeOption<Box<[f32]>> as CubeType>::ExpandType, scheme: QuantScheme, ) -> <Self as CubeType>::ExpandType

A view reading every scale per position through scales.

Takes a one-level scheme: the per-tensor scale of a two-level one never rides a signature here, it is either bound at launch or already held by a caller as a KnownScale passed to new_with_known_scale.

pub fn __expand_new_with_known_scale( scope: &Scope, values: <View<'a, Vector<Q, NQ>, C> as CubeType>::ExpandType, scales: <View<'a, S, C> as CubeType>::ExpandType, known_scale: <KnownScale as CubeType>::ExpandType, table: <ComptimeOption<Box<[f32]>> as CubeType>::ExpandType, scheme: QuantScheme, ) -> <Self as CubeType>::ExpandType

new with whatever the caller already knows of the scale, in whichever KnownScale form it holds it. Only a caller that knows what its reads share can say so, which is why this exists on the cube side. Reads assert the register agrees with the scheme.

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> QuantizedView<'a, Q, NQ, S, F, NF, C>

pub fn view(self) -> View<'a, Vector<F, NF>, C>

pub fn __expand_view( scope: &Scope, this: QuantizedViewExpand<'a, Q, NQ, S, F, NF, C>, ) -> ViewExpand<'a, Vector<F, NF>, C>

Trait Implementations

impl<'a, Q: Clone + Scalar, NQ: Clone + Size, S: Clone + Scalar, F: Clone + Numeric, NF: Clone + Size, C: Clone + Coordinates + 'static> Clone for QuantizedView<'a, Q, NQ, S, F, NF, C>

fn clone(&self) -> QuantizedView<'a, Q, NQ, S, F, NF, C>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> CubeType for QuantizedView<'a, Q, NQ, S, F, NF, C>

type ExpandType = QuantizedViewExpand<'a, Q, NQ, S, F, NF, C>

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> LaunchArg for QuantizedView<'a, Q, NQ, S, F, NF, C>

where View<'a, Vector<Q, NQ>, C>: LaunchArg, View<'a, S, C>: LaunchArg,

type RuntimeArg<R: Runtime> = QuantizedViewLaunch<'a, Q, NQ, S, F, NF, C, R>

The runtime argument for the kernel.

type CompilationArg = QuantizedViewCompilationArg<'a, Q, NQ, S, F, NF, C>

Compilation argument.

fn register<R: Runtime>( arg: Self::RuntimeArg<R>, launcher: &mut KernelLauncher<R>, ) -> Self::CompilationArg

fn expand( arg: &Self::CompilationArg, builder: &mut KernelBuilder, ) -> <Self as CubeType>::ExpandType

Register a variable during compilation that fill the KernelBuilder.

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> Vectorized for QuantizedView<'a, Q, NQ, S, F, NF, C>

fn vector_size(&self) -> usize

fn __expand_vector_size(scope: &Scope, this: &Self::ExpandType) -> usize

impl<'a, Q: Scalar, NQ: Size, S: Scalar, F: Numeric, NF: Size, C: Coordinates + 'static> ViewOperations<Vector<F, NF>, C> for QuantizedView<'a, Q, NQ, S, F, NF, C>

fn read(&self, pos: C) -> T

fn read_checked(&self, pos: C) -> T

fn read_masked(&self, pos: C, value: T) -> T

fn read_unchecked(&self, pos: C) -> T

fn as_linear_slice(&self, pos: C, size: C) -> &[T]

Create a slice starting from pos, with size. The layout handles translation into concrete indices.

fn tensor_map_load(&self, barrier: &Barrier, shared_memory: &mut [T], pos: C)

Execute a TMA load into shared memory, if the underlying storage supports it. Panics if it’s unsupported.

fn shape(&self) -> C

fn is_in_bounds(&self, pos: C) -> bool

fn __expand_read( scope: &Scope, this: &<Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, ) -> <T as CubeType>::ExpandType

fn __expand_read_checked( scope: &Scope, this: &<Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, ) -> <T as CubeType>::ExpandType

fn __expand_read_masked( scope: &Scope, this: &<Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, value: <T as CubeType>::ExpandType, ) -> <T as CubeType>::ExpandType

fn __expand_read_unchecked( scope: &Scope, this: &<Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, ) -> <T as CubeType>::ExpandType

fn __expand_as_linear_slice<'infer, 'scope>( scope: &'scope Scope, this: &'infer <Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, size: <C as CubeType>::ExpandType, ) -> &'infer <[T] as CubeType>::ExpandType

fn __expand_tensor_map_load( scope: &Scope, this: &<Self as CubeType>::ExpandType, barrier: &<Barrier as CubeType>::ExpandType, shared_memory: &mut <[T] as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, )

fn __expand_shape( scope: &Scope, this: &<Self as CubeType>::ExpandType, ) -> <C as CubeType>::ExpandType

fn __expand_is_in_bounds( scope: &Scope, this: &<Self as CubeType>::ExpandType, pos: <C as CubeType>::ExpandType, ) -> <bool as CubeType>::ExpandType