Enum ScaledFormat 

pub enum ScaledFormat {
    F8E4M3,
    F8E5M2,
    F8E4M3Fnuz,
    F8E5M2Fnuz,
    F6E2M3,
    F6E3M2,
    F4E2M1,
    Custom {
        exp_bits: u8,
        mant_bits: u8,
        bias: i8,
    },
}

Element format for a native low-precision tensor-core GEMM operand (see crate::op::Op::ScaledMatMul).

Distinct from QuantScheme: a QuantScheme is block-scaled storage that the CPU/GPU decodes to f32 before a normal sgemm. A ScaledFormat is the raw element encoding hardware tensor cores consume directly, with f32 accumulation — the whole point of FP8/FP6/FP4 on Hopper / Ada / Blackwell / CDNA3 / CDNA4. The per-block/per-tensor scale layout is orthogonal and lives in ScaleLayout.

Operands flow through the graph as DType::U8 byte buffers (one code per byte on the CPU oracle; packed on GPU); the format is carried on the op, not the dtype, so no DType variant is needed.

Variants

F8E4M3

OCP FP8 E4M3 — bias 7, no infinities, only S.1111.111 is NaN. Max ±448. Hopper / Ada / Blackwell; weights + forward activations.

F8E5M2

OCP FP8 E5M2 — bias 15, IEEE-like inf/NaN at max exponent. Max ±57344. Wider range — the gradient format.

F8E4M3Fnuz

AMD “FNUZ” FP8 E4M3 — bias 8, single NaN at 0x80, no inf, no −0. Max ±240. CDNA3 / MI300.

F8E5M2Fnuz

AMD “FNUZ” FP8 E5M2 — bias 16, single NaN at 0x80, no inf, no −0. Max ±57344. CDNA3 / MI300.

F6E2M3

MX FP6 E2M3 — bias 1, all 64 codes finite. Max ±7.5. Blackwell / CDNA4.

F6E3M2

MX FP6 E3M2 — bias 3, all 64 codes finite. Max ±28. Blackwell / CDNA4.

F4E2M1

FP4 E2M1 — all 16 codes finite. Max ±6. Blackwell / CDNA4. Shared by NVFP4 and MXFP4 — the ScaleLayout tells them apart.

Custom

Arbitrary sub-byte minifloat parameterized by exponent / mantissa width — the fNeXmY family (N = 1 + exp_bits + mant_bits total bits).

All-finite (no infinities, no NaN, no AMD FNUZ), IEEE-style gradual underflow (subnormals when exp == 0), sign in bit exp_bits + mant_bits, one code per byte. The whole code must fit in a byte, so 1 + exp_bits + mant_bits <= 8 and exp_bits >= 1.

This is a research / experimental encoding: it has no hardware tensor-core GEMM, so it always runs the decode-and-accumulate path (CPU oracle + generic GPU kernel). Build one with ScaledFormat::custom (IEEE bias 2^(exp_bits-1) - 1) or ScaledFormat::custom_with_bias, or parse a name via "f4e3m0".parse().

Example — f4e3m0 (exp_bits: 3, mant_bits: 0, bias: 3): the 16 codes are ±0 and ±{0.25, 0.5, 1, 2, 4, 8, 16} — a signed power-of-two (near-logarithmic) 4-bit format.

Fields

exp_bits: u8

mant_bits: u8

bias: i8

Implementations

impl ScaledFormat

pub const NAMED: [ScaledFormat; 7]

The seven named hardware element formats, for enumeration / format sweeps.

pub const fn bit_width(self) -> u32

Total bit width of one element code (4, 6, or 8).

pub const fn fields(self) -> (u32, u32, i32)

(exponent_bits, mantissa_bits, exponent_bias).

pub const fn custom(exp_bits: u8, mant_bits: u8) -> Self

Build an arbitrary all-finite minifloat with exp_bits exponent and mant_bits mantissa bits, using the IEEE-style bias 2^(exp_bits-1) - 1 — the convention every named rlx format follows (E4M3 → 7, E5M2 → 15, E2M1 → 1, …). The whole code must fit in a byte. const — usable in const contexts and pattern-free construction.

Panics if exp_bits == 0 or 1 + exp_bits + mant_bits > 8.

use rlx_ir::ScaledFormat;
const F4E3M0: ScaledFormat = ScaledFormat::custom(3, 0);
assert_eq!(F4E3M0.to_string(), "f4e3m0");
assert_eq!(F4E3M0.exp_bits(), 3);
assert_eq!(F4E3M0.mant_bits(), 0);
// Nearest representable value ("quantize" a single f32):
assert_eq!(F4E3M0.quantize(1.4), 1.0);
assert_eq!(F4E3M0.max_finite(), 16.0);

pub const fn custom_with_bias(exp_bits: u8, mant_bits: u8, bias: i8) -> Self

Like custom but with an explicit exponent bias (for formats that don’t use the IEEE 2^(e-1)-1 convention). const.

pub const fn exp_bits(self) -> u32

Exponent-field bit count (X in fNeXmY).

pub const fn mant_bits(self) -> u32

Mantissa-field bit count (Y in fNeXmY; 0 for a pure power-of-two format such as f4e3m0).

pub const fn bias(self) -> i32

Exponent bias.

pub const fn is_custom(self) -> bool

True for a parameterized Custom format (no hardware tensor core — always runs the decode path).

pub const fn is_named(self) -> bool

True for one of the seven NAMED hardware formats.

pub const fn is_fnuz(self) -> bool

AMD OCP-FNUZ encoding (single NaN at 0x80, no inf, no −0).

pub const fn has_inf(self) -> bool

True for IEEE-like formats carrying inf / NaN at the max exponent (OCP E5M2 only — E4M3 is finite-with-one-NaN, FP6/FP4 are all-finite).

pub fn max_finite(self) -> f32

Largest finite magnitude representable (used as the amax divisor when computing a per-tensor / per-block scale).

pub fn decode(self, code: u8) -> f32

Decode one packed code to f32 (bit-exact; ±inf / NaN for the formats that encode them). Method form of crate::lowp_codec::decode.

pub fn encode(self, x: f32) -> u8

Encode an f32 to its nearest representable code (round-half-to-even, saturating overflow / ±inf to ±max_finite, NaN → 0). Method form of crate::lowp_codec::encode.

pub fn quantize(self, x: f32) -> f32

Round-trip x through the format — the nearest value it can represent. Handy for eyeballing a format’s resolution without building a graph (e.g. ScaledFormat::custom(3, 0).quantize(1.4) == 1.0).

pub fn representable_values(self) -> Vec<f32>

Every finite value the format represents, ascending and deduplicated — the format’s grid. Useful for inspecting a research minifloat, e.g. ScaledFormat::custom(3, 0).representable_values() yields [-16, -8, -4, -2, -1, -0.5, -0.25, 0, 0.25, 0.5, 1, 2, 4, 8, 16].

pub const fn kernel_id(self) -> u32

The GPU dispatch word passed to scaled_lowp_general.cu (rlx_decode_lowp).

For the seven named formats this is a small stable id matching the kernel’s switch: e4m3=0, e5m2=1, e4m3fnuz=2, e5m2fnuz=3, e2m3=4, e3m2=5, e2m1=6.

For a Custom format there is no fixed id, so this returns a packed field descriptor with the top bit set as a sentinel: 0x8000_0000 | exp_bits | mant_bits<<4 | (bias as u8)<<8. The kernel detects the sentinel bit and decodes generically from the unpacked (exp_bits, mant_bits, bias) — so a new format needs no kernel edit. The named ids stay in 0..=6 (top bit clear), so the existing hardware switch path is unchanged.

pub const fn is_native_fp8(self) -> bool

True for the OCP FP8 variants that the native cublasLt / hipBLASLt FP8 GEMM accepts (per-tensor only). Other formats use the decode fallback.

Trait Implementations

impl Clone for ScaledFormat

fn clone(&self) -> ScaledFormat

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ScaledFormat

impl Debug for ScaledFormat

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ScaledFormat

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for ScaledFormat

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for ScaledFormat

impl FromStr for ScaledFormat

fn from_str(s: &str) -> Result<Self, Self::Err>

Parse a format name. The seven named formats ("f8e4m3", "f4e2m1", the …fnuz variants, …) keep their exact hardware semantics (FNUZ, inf/NaN); any other fNeXmY string becomes an all-finite Custom with the IEEE bias. N must equal 1 + X + Y and the code must fit in a byte (N <= 8, X >= 1).

type Err = String

The associated error which can be returned from parsing.

impl Hash for ScaledFormat

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for ScaledFormat

fn eq(&self, other: &ScaledFormat) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for ScaledFormat

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ScaledFormat