Enum TensorFormat 

#[repr(i32)]
pub enum TensorFormat {
    kLINEAR = 0,
    kCHW2 = 1,
    kHWC8 = 2,
    kCHW4 = 3,
    kCHW16 = 4,
    kCHW32 = 5,
    kDHWC8 = 6,
    kCDHW32 = 7,
    kHWC = 8,
    kDLA_LINEAR = 9,
    kDLA_HWC4 = 10,
    kHWC16 = 11,
    kDHWC = 12,
}

TensorFormat

Format of the input/output tensors.

This enum is used by both plugins and network I/O tensors.

See [IPluginV2::supportsFormat()], safe::ICudaEngine::getBindingFormat()

Many of the formats are vector-major or vector-minor. These formats specify a vector dimension and scalars per vector. For example, suppose that the tensor has has dimensions [M,N,C,H,W], the vector dimension is C and there are V scalars per vector.

• A vector-major format splits the vectorized dimension into two axes in the memory layout. The vectorized dimension is replaced by an axis of length ceil(C/V) and a new dimension of length V is appended. For the example tensor, the memory layout is equivalent to an array with dimensions [M][N][ceil(C/V)][H][W][V]. Tensor coordinate (m,n,c,h,w) maps to array location [m][n][c/V][h][w][c%V].

• A vector-minor format moves the vectorized dimension to become the last axis in the memory layout. For the example tensor, the memory layout is equivalent to an array with dimensions [M][N][H][W][ceil(C/V)*V]. Tensor coordinate (m,n,c,h,w) maps array location subscript [m][n][h][w][c].

In interfaces that refer to “components per element”, that’s the value of V above.

For more information about data formats, see the topic “Data Format Description” located in the TensorRT Developer Guide. https://docs.nvidia.com/deeplearning/tensorrt/latest/inference-library/advanced.html#i-o-formats

Variants

kLINEAR = 0

Memory layout is similar to an array in C or C++. The stride of each dimension is the product of the dimensions after it. The last dimension has unit stride.

This format supports all TensorRT types. For DLA usage, the tensor sizes are limited to C,H,W in the range [1,8192].

kCHW2 = 1

Vector-major format with two scalars per vector. Vector dimension is third to last.

This format requires FP16 or BF16 and at least three dimensions.

kHWC8 = 2

Vector-minor format with eight scalars per vector. Vector dimension is third to last. This format requires FP16 or BF16 and at least three dimensions.

kCHW4 = 3

Vector-major format with four scalars per vector. Vector dimension is third to last.

This format requires INT8 and at least three dimensions. For INT8, the length of the vector dimension must be a build-time constant.

Deprecated usage:

If running on the DLA, this format can be used for acceleration with the caveat that C must be less than or equal to 4. If used as DLA input and the build option kGPU_FALLBACK is not specified, it needs to meet line stride requirement of DLA format. Column stride in bytes must be a multiple of 64 on Orin.

kCHW16 = 4

Vector-major format with 16 scalars per vector. Vector dimension is third to last.

This format is only supported by DLA and requires FP16 and at least three dimensions. This format maps to the native feature format for FP16, and the tensor sizes are limited to C,H,W in the range [1,8192].

kCHW32 = 5

Vector-major format with 32 scalars per vector. Vector dimension is third to last.

This format requires INT8, FP32, or FP16 and at least three dimensions.

For DLA usage, this format maps to the native feature format for INT8, and the tensor sizes are limited to C,H,W in the range [1,8192].

kDHWC8 = 6

Vector-minor format with eight scalars per vector. Vector dimension is fourth to last.

This format requires FP16 or BF16 and at least four dimensions.

kCDHW32 = 7

Vector-major format with 32 scalars per vector. Vector dimension is fourth to last.

This format requires FP16 or INT8 and at least four dimensions.

kHWC = 8

Vector-minor format where channel dimension is third to last and unpadded.

This format requires either FP32 or UINT8 and at least three dimensions.

kDLA_LINEAR = 9

DLA planar format. For a tensor with dimension {N, C, H, W}, the W axis always has unit stride. The stride for stepping along the H axis is rounded up to 64 bytes.

The memory layout is equivalent to a C array with dimensions [N][C][H][roundUp(W, 64/elementSize)] where elementSize is 2 for FP16 and 1 for Int8, with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c][h][w].

kDLA_HWC4 = 10

DLA image format. For a tensor with dimension {N, C, H, W} the C axis always has unit stride. The stride for stepping along the H axis is rounded up to 64 bytes on Orin. C can only be 1, 3 or 4. If C == 1, it will map to grayscale format. If C == 3 or C == 4, it will map to color image format. And if C == 3, the stride for stepping along the W axis needs to be padded to 4 in elements.

When C is {1, 3, 4}, then C’ is {1, 4, 4} respectively, the memory layout is equivalent to a C array with dimensions [N][H][roundUp(W, 64/C’/elementSize)][C’] on Orin where elementSize is 2 for FP16 and 1 for Int8. The tensor coordinates (n, c, h, w) mapping to array subscript [n][h][w][c].

kHWC16 = 11

Vector-minor format with 16 scalars per vector. Vector dimension is third to last.

This requires FP16, INT8 or FP8 and at least three dimensions.

kDHWC = 12

Vector-minor format with one scalar per vector. Vector dimension is fourth to last.

This format requires FP32 and at least four dimensions.

Trait Implementations

impl Clone for TensorFormat

fn clone(&self) -> TensorFormat

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ExternType for TensorFormat

type Id = (n, v, i, n, f, e, r, _1, (), T, e, n, s, o, r, F, o, r, m, a, t)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Trivial

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl Hash for TensorFormat

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for TensorFormat

fn eq(&self, other: &TensorFormat) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for TensorFormat

impl SharedPtrTarget for TensorFormat

impl StructuralPartialEq for TensorFormat

impl UniquePtrTarget for TensorFormat

impl VectorElement for TensorFormat

impl WeakPtrTarget for TensorFormat