Struct IRuntimeConfig 

pub struct IRuntimeConfig { /* private fields */ }

IRuntimeConfig

A class for runtime configuration. This class is used during execution context creation.

See IRuntime, IBuilderConfig

Implementations

impl IRuntimeConfig

pub fn setExecutionContextAllocationStrategy( self: Pin<&mut IRuntimeConfig>, strategy: ExecutionContextAllocationStrategy, )

Set the execution context allocation strategy. Default value is kSTATIC.

• strategy The execution context allocation strategy.

pub fn getExecutionContextAllocationStrategy( self: &IRuntimeConfig, ) -> ExecutionContextAllocationStrategy

Get the execution context allocation strategy.

The execution context allocation strategy.

pub fn createRuntimeCache(self: &IRuntimeConfig) -> *mut IRuntimeCache

Create an empty Runtime cache.

A fresh IRuntimeCache object.

pub fn setRuntimeCache( self: Pin<&mut IRuntimeConfig>, cache: &IRuntimeCache, ) -> bool

Set Runtime cache to the runtime config. Enables Runtime caching.

• cache The Runtime cache to set.

pub fn getRuntimeCache(self: &IRuntimeConfig) -> *mut IRuntimeCache

Get the Runtime cache from the runtime config.

The Runtime cache.

pub fn setDynamicShapesKernelSpecializationStrategy( self: Pin<&mut IRuntimeConfig>, dynamicShapesKernelSpecializationStrategy: DynamicShapesKernelSpecializationStrategy, )

Set the dynamic shape kernel specialization strategy for this config.

• dynamicShapesKernelSpecializationStrategy The dynamic shape specialization strategy.

pub fn getDynamicShapesKernelSpecializationStrategy( self: &IRuntimeConfig, ) -> DynamicShapesKernelSpecializationStrategy

Return the dynamic shape specialization strategy of this config.

dynamicShapesKernelSpecializationStrategy The dynamic shape specialization strategy.

pub fn setCudaGraphStrategy( self: Pin<&mut IRuntimeConfig>, strategy: CudaGraphStrategy, ) -> bool

Set the strategy used for CUDA graphs for JIT (Just-In-Time) inference. Default is kDISABLED.

Even if the strategy was set to kWHOLE_GRAPH_CAPTURE, RTX CUDA graphs may be disabled internally during runtime under following circumstances:

• The given stream does not allow graph capture.
• The allocation strategy is blocking.
• Some layers in the engine use data-dependent dynamic shapes or on-device control flow.
• The engine is streaming weights. If CUDA graph is not supported for above or any other reason, it will silently ignore the set strategy and run inference without CUDA graphs.

If the stream is being captured elsewhere, RTX CUDA graphs will be disabled.

• strategy The CUDA graph strategy.

True if the strategy was within range and set successfully, false otherwise.

pub fn getCudaGraphStrategy(self: &IRuntimeConfig) -> CudaGraphStrategy

Return the strategy used for CUDA graphs for JIT (Just-In-Time) inference.

The CUDA graph strategy.

Trait Implementations

impl Drop for IRuntimeConfig

fn drop(self: &mut IRuntimeConfig)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl ExternType for IRuntimeConfig

type Id = (n, v, i, n, f, e, r, _1, (), I, R, u, n, t, i, m, e, C, o, n, f, i, g)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Opaque

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl MakeCppStorage for IRuntimeConfig

unsafe fn allocate_uninitialized_cpp_storage() -> *mut IRuntimeConfig

Allocates heap space for this type in C++ and return a pointer to that space, but do not initialize that space (i.e. do not yet call a constructor).

unsafe fn free_uninitialized_cpp_storage(arg0: *mut IRuntimeConfig)

Frees a C++ allocation which has not yet had a constructor called.

impl SharedPtrTarget for IRuntimeConfig

impl UniquePtrTarget for IRuntimeConfig

impl WeakPtrTarget for IRuntimeConfig