Enum MemoryPoolType 

#[repr(i32)]
pub enum MemoryPoolType {
    kWORKSPACE = 0,
    kDLA_MANAGED_SRAM = 1,
    kDLA_LOCAL_DRAM = 2,
    kDLA_GLOBAL_DRAM = 3,
    kTACTIC_DRAM = 4,
    kTACTIC_SHARED_MEMORY = 5,
}

MemoryPoolType

The type for memory pools used by TensorRT.

See [IBuilderConfig::setMemoryPoolLimit], IBuilderConfig::getMemoryPoolLimit

Variants

kWORKSPACE = 0

kWORKSPACE is used by TensorRT to store intermediate buffers within an operation. This defaults to max device memory. Set to a smaller value to restrict tactics whose workspace usage exceeds the threshold en masse.

kDLA_MANAGED_SRAM = 1

kDLA_MANAGED_SRAM is a fast software managed RAM used by DLA to communicate within a layer. The size of this pool must be at least 4 KiB and must be a power of 2. This defaults to 1 MiB. Orin has capacity of 1 MiB per core.

kDLA_LOCAL_DRAM = 2

kDLA_LOCAL_DRAM is host RAM used by DLA to share intermediate tensor data across operations. The size of this pool must be at least 4 KiB and must be a power of 2. This defaults to 1 GiB.

kDLA_GLOBAL_DRAM = 3

kDLA_GLOBAL_DRAM is host RAM used by DLA to store weights and metadata for execution. The size of this pool must be at least 4 KiB and must be a power of 2. This defaults to 512 MiB.

kTACTIC_DRAM = 4

kTACTIC_DRAM is the device DRAM used by the optimizer to run tactics. On embedded devices, where host and device memory are unified, this includes all host memory required by TensorRT to build the network up to the point of each memory allocation. This defaults to 75% of totalGlobalMem as reported by cudaGetDeviceProperties when cudaGetDeviceProperties.embedded is true, and 100% otherwise.

kTACTIC_SHARED_MEMORY = 5

kTACTIC_SHARED_MEMORY defines the maximum sum of shared memory reserved by the driver and used for executing CUDA kernels. Adjust this value to restrict tactics that exceed the specified threshold en masse. The default value is device max capability. This value must be less than 1GiB.

The driver reserved shared memory can be queried from cuDeviceGetAttribute(&reservedShmem, CU_DEVICE_ATTRIBUTE_RESERVED_SHARED_MEMORY_PER_BLOCK).

Updating this flag will override the shared memory limit set by HardwareCompatibilityLevel, which defaults to 48KiB - reservedShmem.

Trait Implementations

impl Clone for MemoryPoolType

fn clone(&self) -> MemoryPoolType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ExternType for MemoryPoolType

type Id = (n, v, i, n, f, e, r, _1, (), M, e, m, o, r, y, P, o, o, l, T, y, p, e)

A type-level representation of the type’s C++ namespace and type name.

type Kind = Trivial

Either cxx::kind::Opaque or cxx::kind::Trivial.

impl From<MemoryPoolType> for MemoryPoolType

fn from(value: MemoryPoolType) -> Self

Converts to this type from the input type.

impl Hash for MemoryPoolType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl Into<MemoryPoolType> for MemoryPoolType

fn into(self) -> MemoryPoolType

Converts this type into the (usually inferred) input type.

impl PartialEq for MemoryPoolType

fn eq(&self, other: &MemoryPoolType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for MemoryPoolType

impl SharedPtrTarget for MemoryPoolType

impl StructuralPartialEq for MemoryPoolType

impl UniquePtrTarget for MemoryPoolType

impl VectorElement for MemoryPoolType

impl WeakPtrTarget for MemoryPoolType