Struct IRuntime 

pub struct IRuntime { /* private fields */ }

IRuntime

Allows a serialized functionally unsafe engine to be deserialized.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Implementations

impl IRuntime

pub fn deserializeCudaEngine1( self: Pin<&mut Self>, streamReader: Pin<&mut IStreamReaderV2>, ) -> *mut ICudaEngine

Deserialize an engine from a stream. IStreamReaderV2 is expected to support reading to both host and device pointers.

If an error recorder has been set for the runtime, it will also be passed to the engine.

This deserialization path will reduce engine load time when applied with GDS (GPU Direct storage), or when weight streaming is enabled.

Destroying the IRuntime before destroying all associated ICudaEngine instances results in undefined behavior.

• streamReader a read-only stream from which TensorRT will deserialize a previously serialized engine.

The engine, or nullptr if it could not be deserialized. The pointer may not be valid immediately after the function returns.

impl IRuntime

pub fn setDLACore(self: Pin<&mut IRuntime>, dlaCore: i32)

Sets the DLA core used by the network. Defaults to -1.

• dlaCore The DLA core to execute the engine on, in the range [0,getNbDlaCores()).

This function is used to specify which DLA core to use via indexing, if multiple DLA cores are available.

if getNbDLACores() returns 0, then this function does nothing.

See [getDLACore()]

pub fn getDLACore(self: &IRuntime) -> i32

Get the DLA core that the engine executes on.

assigned DLA core or -1 for DLA not present or unset.

pub fn getNbDLACores(self: &IRuntime) -> i32

Returns number of DLA hardware cores accessible or 0 if DLA is unavailable.

pub unsafe fn setGpuAllocator( self: Pin<&mut IRuntime>, allocator: *mut IGpuAllocator, )

Set the GPU allocator.

• allocator Set the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: allocateAsync uses cudaMallocAsync if cudaDevAttrMemoryPoolsSupported returns true, otherwise falls back to cudaMalloc. allocate always uses cudaMalloc.

If nullptr is passed, the default allocator will be used.

pub unsafe fn setErrorRecorder( self: Pin<&mut IRuntime>, recorder: *mut IErrorRecorder, )

See [getErrorRecorder()]

pub fn getErrorRecorder(self: &IRuntime) -> *mut IErrorRecorder

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if an error handler has not been set.

A pointer to the IErrorRecorder object that has been registered.

See [setErrorRecorder()]

pub unsafe fn deserializeCudaEngine( self: Pin<&mut IRuntime>, blob: *const c_void, size: usize, ) -> *mut ICudaEngine

Deserialize an engine from host memory.

If an error recorder has been set for the runtime, it will also be passed to the engine.

Destroying the IRuntime before destroying all associated ICudaEngine instances results in undefined behavior.

• blob The memory that holds the serialized engine.
• size The size of the memory.

The engine, or nullptr if it could not be deserialized.

pub fn getLogger(self: &IRuntime) -> *mut ILogger

get the logger with which the runtime was created

the logger

pub fn setMaxThreads(self: Pin<&mut IRuntime>, maxThreads: i32) -> bool

Set the maximum number of threads.

• maxThreads The maximum number of threads that can be used by the runtime. True if successful, false otherwise.

The default value is 1 and includes the current thread. A value greater than 1 permits TensorRT to use multi-threaded algorithms. A value less than 1 triggers a kINVALID_ARGUMENT error.

pub fn getMaxThreads(self: &IRuntime) -> i32

Get the maximum number of threads that can be used by the runtime.

Retrieves the maximum number of threads that can be used by the runtime.

The maximum number of threads that can be used by the runtime.

See [setMaxThreads()]

pub unsafe fn setTemporaryDirectory( self: Pin<&mut IRuntime>, path: *const c_char, )

Set the directory that will be used by this runtime for temporary files.

On some platforms the TensorRT runtime may need to create and use temporary files with read/write/execute permissions to implement runtime functionality.

• path Path to the temporary directory for use, or nullptr.

If path is nullptr, then TensorRT will use platform-specific heuristics to pick a default temporary directory if required:

• On UNIX/Linux platforms, TensorRT will first try the TMPDIR environment variable, then fall back to /tmp
• On Windows, TensorRT will try the TEMP environment variable.

See the TensorRT Developer Guide for more information.

The default value is nullptr.

If path is not nullptr, it must be a non-empty string representing a relative or absolute path in the format expected by the host operating system.

The string path must be null-terminated, and be at most 4096 bytes including the terminator. Note that the operating system may have stricter path length requirements.

The process using TensorRT must have rwx permissions for the temporary directory, and the directory shall be configured to disallow other users from modifying created files (e.g. on Linux, if the directory is shared with other users, the sticky bit must be set).

See [getTemporaryDirectory()]

pub fn getTemporaryDirectory(self: &IRuntime) -> *const c_char

Get the directory that will be used by this runtime for temporary files.

• Returns A path to the temporary directory in use, or nullptr if no path is specified.

See [setTemporaryDirectory()]

pub fn setTempfileControlFlags(self: Pin<&mut IRuntime>, flags: u32)

Set the tempfile control flags for this runtime.

• flags The flags to set.

The default value is all flags set, i.e.

(1U << static_cast<uint32_t>(kALLOW_IN_MEMORY_FILES)) | (1U << static_cast<uint32_t>(kALLOW_TEMPORARY_FILES))

See [TempfileControlFlag], TempfileControlFlags, getTempfileControlFlags()

pub fn getTempfileControlFlags(self: &IRuntime) -> u32

Get the tempfile control flags for this runtime.

The flags currently set.

See [TempfileControlFlag], TempfileControlFlags, setTempfileControlFlags()

pub fn getPluginRegistry<'a>( self: Pin<&'a mut IRuntime>, ) -> Pin<&'a mut IPluginRegistry>

Get the local plugin registry that can be used by the runtime.

The local plugin registry that can be used by the runtime.

pub unsafe fn loadRuntime( self: Pin<&mut IRuntime>, path: *const c_char, ) -> *mut IRuntime

Load IRuntime from the file.

This method loads a runtime library from a shared library file. The runtime can then be used to execute a plan file built with BuilderFlag::kVERSION_COMPATIBLE and BuilderFlag::kEXCLUDE_LEAN_RUNTIME both set and built with the same version of TensorRT as the loaded runtime library.

• path Path to the runtime lean library.

the runtime library, or nullptr if it could not be loaded

The path string must be null-terminated, and be at most 4096 bytes including the terminator.

pub fn setEngineHostCodeAllowed(self: Pin<&mut IRuntime>, allowed: bool)

Set whether the runtime is allowed to deserialize engines with host executable code.

• allowed Whether the runtime is allowed to deserialize engines with host executable code.

The default value is false.

pub fn getEngineHostCodeAllowed(self: &IRuntime) -> bool

Get whether the runtime is allowed to deserialize engines with host executable code.

Whether the runtime is allowed to deserialize engines with host executable code.

pub fn getEngineHeaderSize(self: &IRuntime) -> i64

Get size of engine header in bytes.

This represents the minimum size of the buffer to be passed to getEngineValidity(). When checking the validity of large engine files, users may save time and memory by only loading the first getEngineHeaderSize() bytes of the engine file from the file system.

pub unsafe fn getEngineValidity( self: &IRuntime, blob: *const c_void, blobSize: i64, diagnostics: *mut u64, ) -> EngineValidity

Check for engine validity by inspecting the serialized engine header.

• blob The start address of a memory buffer containing either a serialized engine or the header of a serialized engine.

• blobSize The size of the memory buffer in bytes.

• diagnostics Either nullptr or the address of a bitmask where to write diagnostic information why the validity check failed.

Whether the engine is valid, suboptimal or invalid, based on the header information.

The buffer must have a length of at least getEngineHeaderSize() bytes.

This function only checks the engine header, so it cannot guarantee that engine deserialization will succeed. For example, it cannot detect whether the body of the serialized engine has become corrupted.

If a non-null pointer is passed as the diagnostics argument, *diagnostics will be set to 0 if the return value is EngineValidity::kVALID or EngineValidity::kSUBOPTIMAL. If the return value is EngineValidity::kINVALID, *diagnostics will contain a nonzero bitmask with all detected error conditions, e.g. kVERSION_MISMATCH | kUNSUPPORTED_CC.

Trait Implementations

impl Drop for IRuntime

fn drop(self: &mut IRuntime)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl ExternType for IRuntime

type Id = (n, v, i, n, f, e, r, _1, (), I, R, u, n, t, i, m, e)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for IRuntime

unsafe fn allocate_uninitialized_cpp_storage() -> *mut IRuntime

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut IRuntime)

Frees a C++ allocation which has not yet had a constructor called.

impl SharedPtrTarget for IRuntime

impl UniquePtrTarget for IRuntime

impl WeakPtrTarget for IRuntime