Struct ITensor 

pub struct ITensor { /* private fields */ }

ITensor

A tensor in a network definition.

To remove a tensor from a network definition, use INetworkDefinition::removeTensor().

When using the DLA, the cumulative size of all Tensors that are not marked as Network Input or Output tensors, must be less than 1GB in size to fit into a single subgraph. If the build option kGPU_FALLBACK is specified, then multiple subgraphs can be created, with each subgraph limited to less than 1GB of internal tensors data.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Implementations

impl ITensor

pub unsafe fn setName(self: Pin<&mut ITensor>, name: *const c_char)

Set the tensor name.

For a network input, the name is assigned by the application. For tensors which are layer outputs, a default name is assigned consisting of the layer name followed by the index of the output in brackets. Each input and output tensor must have a unique name.

This method copies the name string.

• name The name.

The string name must be null-terminated, and be at most 4096 bytes including the terminator.

See [getName()]

pub fn getName(self: &ITensor) -> *const c_char

Get the tensor name.

The name as a null-terminated C-style string.

See [setName()]

pub fn setDimensions(self: Pin<&mut ITensor>, dimensions: &Dims64)

Set the dimensions of a tensor.

For a network input, the dimensions are assigned by the application. For a network output, the dimensions are computed based on the layer parameters and the inputs to the layer. If a tensor size or a parameter is modified in the network, the dimensions of all dependent tensors will be recomputed.

This call is only legal for network input tensors, since the dimensions of layer output tensors are inferred based on layer inputs and parameters.

• dimensions The dimensions of the tensor.

See [getDimensions()]

pub fn getDimensions(self: &ITensor) -> Dims64

Get the dimensions of a tensor.

The dimensions of the tensor.

getDimensions() returns a -1 for dimensions that are derived from a wildcard dimension.

See [setDimensions()]

pub fn setType(self: Pin<&mut ITensor>, type_: DataType)

Set the data type of a tensor.

• type The data type of the tensor when the type is not inferred.

For strongly typed networks, this method should be used only for network inputs, since the types of all other tensors are inferred. Setting the type of a network output is tolerated if the type equals the inferred type, otherwise an error occurs and the type is not updated.

For weakly typed networks, this method can be used for network outputs too, but the type merely has to be implicitly convertible from the inferred type to the specified type. In this case it does not matter whether the type is set first or the tensor is marked as an output first (via INetworkDefinition::markOutput or INetworkDefinition::markOutputForShapes).

However, marking it first has two advantages:

• It avoids warnings that the tensor is not yet a network I/O tensor.
• It causes method getType() to return the type that was set instead of the inferred type.

See [getType()]

This function does more than just set the type, so t.setType(t.getType()) is not necessarily a no-op, particularly for input and output tensors!

Repeated consecutive applications of t.setType(t.getType()) would be idempotent, provided the state of the ITensor isn’t changed between calls.

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

pub fn getType(self: &ITensor) -> DataType

Get the data type of a tensor.

The data type of the tensor.

The type is the type set by setType if the tensor is a network input or output. Otherwise the type is the inferred type.

See [setType()]

pub fn isNetworkInput(self: &ITensor) -> bool

Whether the tensor is a network input.

pub fn isNetworkOutput(self: &ITensor) -> bool

Whether the tensor is a network output.

pub fn setAllowedFormats(self: Pin<&mut ITensor>, formats: u32)

Set allowed formats for an input or output tensor. By default all formats are allowed. Shape tensors (for which isShapeTensor() returns true) may only have row-major linear format.

When running network on DLA and the build option kGPU_FALLBACK is not specified, if DLA format(kCHW4 with Int8, kCHW4 with FP16, kCHW16 with FP16, kCHW32 with Int8) is set, the input format is treated as native DLA format with line stride requirement. Input/output binding with these format should have correct layout during inference.

Tensor formats are determined at build time by TensorRT for tensors not marked as input or output.

• formats A bitmask of TensorFormat values that are supported for this tensor.

See ITensor::getAllowedFormats()

See TensorFormats

pub fn getAllowedFormats(self: &ITensor) -> u32

Get a bitmask of TensorFormat values that the tensor supports. For a shape tensor, only row-major linear format is allowed.

The value specified by setAllowedFormats or all possible formats.

See ITensor::setAllowedFormats()

pub fn isShapeTensor(self: &ITensor) -> bool

Whether the tensor is a shape tensor.

A shape tensor is a tensor that is related to shape calculations. It must have type Int32, Int64, Bool, or Float, and its shape must be determinable at build time. Furthermore, it must be needed as a shape tensor, either marked as a network shape output via markOutputForShapes(), or as a layer input that is required to be a shape tensor, such as the second input to IShuffleLayer. Some layers are “polymorphic” in this respect. For example, the inputs to IElementWiseLayer must be shape tensors if the output is a shape tensor.

The TensorRT Developer Guide gives the formal rules for what tensors are shape tensors.

The result of isShapeTensor() is reliable only when network construction is complete. For example, if a partially built network sums two tensors T1 and T2 to create tensor T3, and none are yet needed as shape tensors, isShapeTensor() returns false for all three tensors. Setting the second input of IShuffleLayer to be T3 would cause all three tensors to be shape tensors, because IShuffleLayer requires that its second optional input be a shape tensor, and IElementWiseLayer is “polymorphic”.

It is possible for a tensor to be both a shape tensor and an execution tensor.

True if tensor is a shape tensor, false otherwise.

See INetworkDefinition::markOutputForShapes()

pub fn isExecutionTensor(self: &ITensor) -> bool

Whether the tensor is an execution tensor.

Tensors are usually execution tensors. The exceptions are tensors used solely for shape calculations or whose contents are not needed to compute the outputs.

The result of isExecutionTensor() is reliable only when network construction is complete. For example, if a partially built network has no path from a tensor to a network output, isExecutionTensor() returns false. Completing the path would cause it to become true.

A tensor with isShapeTensor() == false and isExecutionTensor() == false can still show up as an input to the engine if its dimensions are required. In that case, only its dimensions need to be set at runtime and a nullptr can be passed instead of a pointer to its contents.

pub unsafe fn setDimensionName( self: Pin<&mut ITensor>, index: i32, name: *const c_char, )

Name a dimension of an input tensor.

Associate a runtime dimension of an input tensor with a symbolic name. Dimensions with the same non-empty name must be equal at runtime. Knowing this equality for runtime dimensions may help the TensorRT optimizer. Both runtime and build-time dimensions can be named.

For example, setDimensionName(0, “n”) associates the symbolic name “n” with the leading dimension.

This method copies the name string. If the function is called again, with the same index, it will overwrite the previous name. If nullptr is passed as name, it will clear the name of the dimension.

• index index of the dimension
• name of the dimension, as a pointer to a null-terminated character sequence.

The string name must be null-terminated, and be at most 4096 bytes including the terminator.

See [getDimensionName()]

pub fn getDimensionName(self: &ITensor, index: i32) -> *const c_char

Get the name of an input dimension.

• index index of the dimension

The name of the input dimension, or nullptr if the dimension has no name. The name is a pointer to a null-terminated character sequence.

See [setDimensionName()]

Trait Implementations

impl ExternType for ITensor

type Id = (n, v, i, n, f, e, r, _1, (), I, T, e, n, s, o, r)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for ITensor

unsafe fn allocate_uninitialized_cpp_storage() -> *mut ITensor

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut ITensor)

Frees a C++ allocation which has not yet had a constructor called.