Enum ScaleLayout 

pub enum ScaleLayout {
    PerTensor,
    BlockMxE8M0 {
        block: u32,
    },
    Nvfp4 {
        group: u32,
    },
}

How scale factors are laid out for an crate::op::Op::ScaledMatMul operand. The reconstructed value of element i is decode(code[i]) * scale(block_of(i)).

Variants

PerTensor

One f32 amax scale for the whole operand (classic per-tensor FP8 GEMM).

BlockMxE8M0

OCP microscaling: one power-of-two E8M0 scale per block consecutive elements along K (MXFP8 / MXFP6 / MXFP4). Scale tensor is DType::U8 (E8M0 bytes).

Fields

block: u32

Nvfp4

NVFP4: one FP8 E4M3 scale per group (16) elements along K, plus an optional per-tensor f32 global. Scale tensor is DType::U8.

Fields

group: u32

Implementations

impl ScaleLayout

pub const fn mx() -> ScaleLayout

OCP microscaling default (32-element blocks).

pub fn nvfp4() -> ScaleLayout

NVFP4 default (16-element groups).

pub const fn scale_dtype(self) -> DType

Element dtype of the scale tensor for this layout.

pub const fn block(self) -> u32

Number of consecutive elements sharing one scale (1 for per-tensor).

pub const fn mode_block(self) -> (u32, u32)

(scale_mode, block) for GPU kernels (scaled_lowp_general.cu): per-tensor=0, block-E8M0=1, NVFP4-E4M3=2.

Trait Implementations

impl Clone for ScaleLayout

fn clone(&self) -> ScaleLayout

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ScaleLayout

impl Debug for ScaleLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ScaleLayout

fn deserialize<__D>( __deserializer: __D, ) -> Result<ScaleLayout, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for ScaleLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Eq for ScaleLayout

impl FromStr for ScaleLayout

fn from_str(s: &str) -> Result<ScaleLayout, <ScaleLayout as FromStr>::Err>

Parse a scale-layout name: "per_tensor", "mx" (32-element E8M0 microscaling), "nvfp4" (16-element E4M3), or an explicit block size "mx/<block>" / "nvfp4/<group>". Mirrors Display round-trip.

type Err = String

The associated error which can be returned from parsing.

impl Hash for ScaleLayout

fn hash<__H>(&self, state: &mut __H)

where __H: Hasher,

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for ScaleLayout

fn eq(&self, other: &ScaleLayout) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ScaleLayout

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ScaleLayout