Struct ILayer

Source

pub struct ILayer { /* private fields */ }

Expand description

ILayer

Base class for all layer classes in a network definition.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Implementations§

Source §

impl ILayer

Source

pub fn getType(self: &ILayer) -> LayerType

Return the type of a layer.

See LayerType

Source

pub unsafe fn setName(self: Pin<&mut ILayer>, name: *const c_char)

Set the name of a layer.

This method copies the name string.

The string name must be null-terminated, and be at most 4096 bytes including the terminator.

See [getName()]

Source

pub fn getName(self: &ILayer) -> *const c_char

Return the name of a layer.

See [setName()]

Source

pub fn getNbInputs(self: &ILayer) -> i32

Get the number of inputs of a layer.

Source

pub fn getInput(self: &ILayer, index: i32) -> *mut ITensor

Get the layer input corresponding to the given index.

index The index of the input tensor.

The input tensor, or nullptr if the index is out of range or the tensor is optional ( ISliceLayer).

Source

pub fn getNbOutputs(self: &ILayer) -> i32

Get the number of outputs of a layer.

Source

pub fn getOutput(self: &ILayer, index: i32) -> *mut ITensor

Get the layer output corresponding to the given index.

The indexed output tensor, or nullptr if the index is out of range or the tensor is optional.

Source

pub fn setInput(self: Pin<&mut ILayer>, index: i32, tensor: Pin<&mut ITensor>)

Replace an input of this layer with a specific tensor.

index the index of the input to modify.
tensor the new input tensor

Except for IFillLayer, ILoopOutputLayer, INMSLayer, IResizeLayer, IShuffleLayer, and ISliceLayer, this method cannot change the number of inputs to a layer. The index argument must be less than the value of getNbInputs().

See comments for overloads of setInput() for layers with special behavior.

Source

pub fn setPrecision(self: Pin<&mut ILayer>, dataType: DataType)

Set the preferred or required computational precision of this layer in a weakly-typed network.

Setting the precision directs TensorRT to choose an implementation that runs at this computational precision. TensorRT could still choose a non-conforming fastest implementation that ignores the requested precision. To force choosing an implementation with the requested precision, set exactly one of the following flags, which differ in what happens if no such implementation exists:

BuilderFlag::kOBEY_PRECISION_CONSTRAINTS - build fails with an error message.
BuilderFlag::kPREFER_PRECISION_CONSTRAINTS - TensorRT falls back to an implementation without the requested precision.

If precision is not set, or falling back, TensorRT will select the layer computational precision and layer input type based on global performance considerations and the flags specified to the builder.

For a IIdentityLayer: If it casts to/from float/half/int8/uint8, the precision must be one of those types, otherwise it must be either the input or output type.

Strongly-typed networks reject calls to method setPrecision. In strongly-typed networks, the computation precision is typically controlled by casting the input tensors to the desired type.

dataType the computational precision.

See [getPrecision()] precisionIsSet() resetPrecision()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub fn getPrecision(self: &ILayer) -> DataType

get the computational precision of this layer

the computational precision

See [setPrecision()] precisionIsSet() resetPrecision()

Source

pub fn precisionIsSet(self: &ILayer) -> bool

whether the computational precision has been set for this layer

whether the computational precision has been explicitly set

See [setPrecision()] getPrecision() resetPrecision()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub fn resetPrecision(self: Pin<&mut ILayer>)

reset the computational precision for this layer

See [setPrecision()] getPrecision() precisionIsSet()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub fn setOutputType(self: Pin<&mut ILayer>, index: i32, dataType: DataType)

Set the output type of this layer in a weakly-typed network.

Setting the output type constrains TensorRT to choose implementations which generate output data with the given type. If it is not set, TensorRT will select output type based on layer computational precision. TensorRT could still choose non-conforming output type based on fastest implementation. To force choosing the requested output type, set exactly one of the following flags, which differ in what happens if no such implementation exists:

BuilderFlag::kOBEY_PRECISION_CONSTRAINTS - build fails with an error message.
BuilderFlag::kPREFER_PRECISION_CONSTRAINTS - TensorRT falls back to an implementation with a non-conforming output type.

In case layer precision is not specified, or falling back, the output type depends on the chosen implementation, based on performance considerations and the flags specified to the builder.

This method cannot be used to set the data type of the second output tensor of the TopK layer. The data type of the second output tensor of the topK layer is always Int32. Also the output type of all layers that are shape operations must be DataType::kINT32, and all attempts to set the output type to some other data type will be ignored except for issuing an error message.

Note that the layer output type is generally not identical to the data type of the output tensor, as TensorRT may insert implicit reformatting operations to convert the former to the latter. Calling layer->setOutputType(i, type) has no effect on the data type of the i-th output tensor of layer, and users need to call layer->getOutput(i)->setType(type) to change the tensor data type. This is particularly relevant if the tensor is marked as a network output, since only setType() [but not setOutputType()] will affect the data representation in the corresponding output binding.

Strongly-typed networks reject calls to method setOutputType. Instead, the output type can be set only for layers that define method setToType(). Those layers are:

ICastLayer
IDequantizeLayer
IDynamicQuantizeLayer
IFillLayer
IQuantizeLayer

index the index of the output to set
dataType the type of the output

See [getOutputType()] outputTypeIsSet() resetOutputType()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub fn getOutputType(self: &ILayer, index: i32) -> DataType

get the output type of this layer

index the index of the output

the output precision. If no precision has been set, DataType::kFLOAT will be returned, unless the output type is inherently DataType::kINT32.

See [getOutputType()] outputTypeIsSet() resetOutputType()

Source

pub fn outputTypeIsSet(self: &ILayer, index: i32) -> bool

whether the output type has been set for this layer

index the index of the output

whether the output type has been explicitly set

See [setOutputType()] getOutputType() resetOutputType()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub fn resetOutputType(self: Pin<&mut ILayer>, index: i32)

reset the output type for this layer

index the index of the output

See [setOutputType()] getOutputType() outputTypeIsSet()

Unsupported in TensorRT-RTX and deprecated in 1.0. Use strong typing instead.

Source

pub unsafe fn setMetadata(self: Pin<&mut ILayer>, metadata: *const c_char)

Set the metadata for this layer.

The metadata is emitted in the JSON returned by IEngineInspector with ProfilingVerbosity set to kDETAILED.

metadata The per-layer metadata.

The string name must be null-terminated and be at most 4096 bytes including the terminator.

See [getMetadata()] See [getLayerInformation()]

Source

pub fn getMetadata(self: &ILayer) -> *const c_char

Get the metadata of the layer.

The metadata as a null-terminated C-style string. If setMetadata() has not been called, an empty string “” will be returned as a default value.

See [setMetadata()]

Source

pub fn setNbRanks(self: Pin<&mut ILayer>, nbRanks: i32) -> bool

Set the number of ranks for multi-device execution.

Setting nbRanks > 1 is only allowed for specific layer types that support multi-device execution:

IDistCollectiveLayer: Determines output shape for kALL_GATHER and kREDUCE_SCATTER operations.

For attention layers, use IAttention::setNbRanks() instead. For all other layer types, nbRanks must be 1.

nbRanks The number of ranks for multi-device execution. Must be >= 1.

true if successful, false if the layer type does not support the specified nbRanks value.

See [getNbRanks()] See IAttention::setNbRanks()

Source

pub fn getNbRanks(self: &ILayer) -> i32

Get the number of ranks for multi-device execution.

The number of ranks configured for multi-device execution. Default is 1.

See [setNbRanks()]