Struct IBuilder 

pub struct IBuilder { /* private fields */ }

IBuilder

Builds an engine from a network definition.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Implementations

impl IBuilder

pub fn getMaxDLABatchSize(self: &IBuilder) -> i32

Get the maximum batch size DLA can support. For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

getMaxDLABatchSize does not work with dynamic shapes.

pub fn getNbDLACores(self: &IBuilder) -> i32

Return the number of DLA engines available to this builder.

pub unsafe fn setGpuAllocator( self: Pin<&mut IBuilder>, allocator: *mut IGpuAllocator, )

Set the GPU allocator.

• allocator Set the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: allocateAsync uses cudaMallocAsync if cudaDevAttrMemoryPoolsSupported returns true, otherwise falls back to cudaMalloc. allocate always uses cudaMalloc.

This allocator will be passed to any engines created via the builder; thus the lifetime of the allocator must span the lifetime of those engines as well as that of the builder. If nullptr is passed, the default allocator will be used.

pub fn createBuilderConfig(self: Pin<&mut IBuilder>) -> *mut IBuilderConfig

Create a builder configuration object.

The caller owns the new IBuilderConfig, which must be destroyed with operator delete before this IBuilder is destroyed. Destroying this IBuilder before destroying the IBuilderConfig causes undefined behavior.

See IBuilderConfig

pub fn createNetworkV2( self: Pin<&mut IBuilder>, flags: u32, ) -> *mut INetworkDefinition

Create a network definition object

Creates a network definition object with immutable properties specified using the flags parameter.

createNetworkV2 supports creating network with properties from NetworkDefinitionCreationFlags.

CreateNetworkV2 supports dynamic shapes and explicit batch dimensions by default.

The network is always strongly typed: tensor data types are inferred from network input types and operator type specification. The kSTRONGLY_TYPED flag is deprecated and ignored.

The caller owns the new INetworkDefinition, which must be destroyed with operator delete before this IBuilder is destroyed. Destroying this IBuilder before destroying the INetworkDefinition causes undefined behavior.

• flags Bitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR, e.g., 1U << NetworkDefinitionCreationFlag::kPREFER_JIT_PYTHON_PLUGINS.

See INetworkDefinition, NetworkDefinitionCreationFlags

pub fn createOptimizationProfile( self: Pin<&mut IBuilder>, ) -> *mut IOptimizationProfile

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to IOptimizationProfile::setShapeValuesV2() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.

See IOptimizationProfile

pub unsafe fn setErrorRecorder( self: Pin<&mut IBuilder>, recorder: *mut IErrorRecorder, )

Set the ErrorRecorder for this interface

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

• recorder The error recorder to register with this interface.

See [getErrorRecorder()]

pub fn getErrorRecorder(self: &IBuilder) -> *mut IErrorRecorder

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not been called.

A pointer to the IErrorRecorder object that has been registered.

See [setErrorRecorder()]

pub fn reset(self: Pin<&mut IBuilder>)

Resets the builder state to default values.

pub fn buildSerializedNetwork( self: Pin<&mut IBuilder>, network: Pin<&mut INetworkDefinition>, config: Pin<&mut IBuilderConfig>, ) -> *mut IHostMemory

Builds and serializes a network for the given INetworkDefinition and IBuilderConfig.

This function allows building and serialization of a network without creating an engine.

• network Network definition.
• config Builder configuration.

A pointer to a IHostMemory object that contains a serialized network.

This function will synchronize the CUDA stream returned by config.getProfileStream() before returning.

See INetworkDefinition, IBuilderConfig, IHostMemory

pub fn buildSerializedNetworkToStream( self: Pin<&mut IBuilder>, network: Pin<&mut INetworkDefinition>, config: Pin<&mut IBuilderConfig>, writer: Pin<&mut IStreamWriter>, ) -> bool

Builds and serializes a network into stream for the given INetworkDefinition and IBuilderConfig.

This function allows building and serialization of a network without creating an engine. The engine is finally serialized into the writer stream.

• network Network definition.
• config Builder configuration.
• writer Output writer stream.

true if build succeed, otherwise false.

This function will synchronize the CUDA stream returned by config.getProfileStream() before returning.

See INetworkDefinition, IBuilderConfig, IStreamWriter

pub fn isNetworkSupported( self: &IBuilder, network: &INetworkDefinition, config: &IBuilderConfig, ) -> bool

Checks that a network is within the scope of the IBuilderConfig settings.

• network The network definition to check for configuration compliance.
• config The configuration of the builder to use when checking network.

Given an INetworkDefinition, network, and an IBuilderConfig, config, check if the network falls within the constraints of the builder configuration based on the EngineCapability, BuilderFlag, and DeviceType. If the network is within the constraints, then the function returns true, and false if a violation occurs. This function reports the conditions that are violated to the registered ErrorRecorder.

True if network is within the scope of the restrictions specified by the builder config, false otherwise.

A true return value does not guarantee that engine building will succeed, as backends may reject it for reasons not detectable with this fast validation. To definitively check whether a network can be built with a given config, use buildEngineWithConfig or buildSerializedNetwork (depending on the engine capability).

This function will synchronize the CUDA stream returned by config.getProfileStream() before returning.

pub fn getLogger(self: &IBuilder) -> *mut ILogger

get the logger with which the builder was created

the logger

pub fn setMaxThreads(self: Pin<&mut IBuilder>, maxThreads: i32) -> bool

Set the maximum number of threads.

• maxThreads The maximum number of threads that can be used by the builder.

True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

pub fn getMaxThreads(self: &IBuilder) -> i32

get the maximum number of threads that can be used by the builder.

Retrieves the maximum number of threads that can be used by the builder.

The maximum number of threads that can be used by the builder.

See [setMaxThreads()]

pub fn getPluginRegistry<'a>( self: Pin<&'a mut IBuilder>, ) -> Pin<&'a mut IPluginRegistry>

get the local plugin registry that can be used by the builder.

The local plugin registry that can be used by the builder.

Trait Implementations

impl Drop for IBuilder

fn drop(self: &mut IBuilder)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl ExternType for IBuilder

type Id = (n, v, i, n, f, e, r, _1, (), I, B, u, i, l, d, e, r)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for IBuilder

unsafe fn allocate_uninitialized_cpp_storage() -> *mut IBuilder

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut IBuilder)

Frees a C++ allocation which has not yet had a constructor called.

impl SharedPtrTarget for IBuilder

impl UniquePtrTarget for IBuilder

impl WeakPtrTarget for IBuilder