Struct IDynamicQuantizeLayer 

pub struct IDynamicQuantizeLayer { /* private fields */ }

IDynamicQuantizeLayer

A network layer to perform dynamic quantization.

This layer accepts a floating-point input tensor and computes the block scale factors needed to quantize the input’s data. It outputs the quantized tensor as its first output and the scale factors as its second output.

Use ILayer::setInput to add an input for the double-quantization scale factor.

Only symmetric quantization is supported. The input tensor for this layer must not be a scalar.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Implementations

impl IDynamicQuantizeLayer

pub fn setToType(self: Pin<&mut IDynamicQuantizeLayer>, toType: DataType)

Set DynamicQuantizeLayer’s quantized output type.

• toType The data type of the quantized output tensor.

Set the type of the dynamic quantization layer’s quantized output.If the network is strongly typed, setToType must be used to set the output type, and use of setOutputType is an error. Otherwise, types passed to setOutputType and setToType must be the same. Valid values for toType are DataType::kFP4 (NVFP4 quantization) and DataType::kFP8 (MXFP8 quantization).

See NetworkDefinitionCreationFlag::kSTRONGLY_TYPED

pub fn getToType(self: &IDynamicQuantizeLayer) -> DataType

Return DynamicQuantizeLayer’s quantized output type.

toType parameter set during layer creation or by setToType().

The return value is the type of the quantized output tensor. The default value is DataType::kFP4.

pub fn setScaleType(self: Pin<&mut IDynamicQuantizeLayer>, scaleType: DataType)

Set the data type of the scale factors used to quantize the data.

• scaleType The scale factors data type.

Set the scale-factors type. Valid values are DataType::kFP8, DataType::kE8M0 or DataType::kFLOAT.

pub fn getScaleType(self: &IDynamicQuantizeLayer) -> DataType

Return the scale factors data type.

scaleType parameter set during layer creation or by setScaleType().

The return value is the type of the scale factors used to quantize the dynamic data. The default value is DataType::kFP8.

pub fn setAxis(self: Pin<&mut IDynamicQuantizeLayer>, axis: i32)

Set the axis along which block quantization occurs.

The axis must be the last dimension or second to last dimension. The input’s shape along the axis must be constant.

See [getAxis()]

pub fn getAxis(self: &IDynamicQuantizeLayer) -> i32

Get the axis along which blocking occurs.

See [setAxis()]

pub fn setBlockSize(self: Pin<&mut IDynamicQuantizeLayer>, size: i32)

Set the size of the quantization block.

Note: The block size must divide the input in the blocked axis without remainder. Valid values are 16 (NVFP4 quantization) and 32 (MXFP8 quantization).

See [getBlockSize()]

pub fn getBlockSize(self: &IDynamicQuantizeLayer) -> i32

Get the size of the quantization block.

See [setBlockSize()]

pub fn setBlockShape(self: Pin<&mut IDynamicQuantizeLayer>, blockShape: &Dims64)

Set the shape of the quantization block.

Note: The block shape rank must match the input rank. The default value is empty Dims.

See [getBlockShape()]

pub fn getBlockShape(self: &IDynamicQuantizeLayer) -> Dims64

Get the shape of the quantization block.

The default value is empty Dims.

See [setBlockShape()]

Trait Implementations

impl AsLayer for IDynamicQuantizeLayer

fn as_layer(&self) -> &ILayer

fn as_layer_pin_mut(&mut self) -> Pin<&mut ILayer>

impl AsLayerTyped for IDynamicQuantizeLayer

const TYPE: LayerType = LayerType::kDYNAMIC_QUANTIZE

impl AsRef<ILayer> for IDynamicQuantizeLayer

fn as_ref(self: &IDynamicQuantizeLayer) -> &ILayer

Converts this type into a shared reference of the (usually inferred) input type.

impl ExternType for IDynamicQuantizeLayer

type Id = (n, v, i, n, f, e, r, _1, (), I, D, y, n, a, m, i, c, Q, u, a, n, t, i, z, e, L, a, y, e, r)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for IDynamicQuantizeLayer

unsafe fn allocate_uninitialized_cpp_storage() -> *mut IDynamicQuantizeLayer

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut IDynamicQuantizeLayer)

Frees a C++ allocation which has not yet had a constructor called.