Structs§
- Dims64
- Dims Structure to define the dimensions of a tensor.
- Error
Code - autocxx bindings couldn’t be generated: This item name is used in multiple namespaces. At present, autocxx and cxx allow only one type of a given name. This limitation will be fixed in future. (Items found with this name: nvinfer1::ErrorCode, nvonnxparser::ErrorCode)
- IActivation
Layer - IActivationLayer
- IAssertion
Layer - IAssertionLayer
- IAttention
- IAttention
- IAttention
Boundary Layer - IAttentionBoundaryLayer
- IAttention
Input Layer - IAttentionInputLayer
- IAttention
Output Layer - IAttentionOutputLayer
- IBuilder
- IBuilder
- IBuilder
Config - IBuilderConfig
- ICast
Layer - ICastLayer
- IConcatenation
Layer - IConcatenationLayer
- ICondition
Layer - IConditionLayer
- IConstant
Layer - IConstantLayer
- IConvolution
Layer - IConvolutionLayer
- ICuda
Engine - ICudaEngine
- ICumulative
Layer - ICumulativeLayer
- IDebug
Listener - IDeconvolution
Layer - IDeconvolutionLayer
- IDequantize
Layer - IDequantizeLayer
- IDist
Collective Layer - IDistCollectiveLayer
- IDynamic
Quantize Layer - IDynamicQuantizeLayer
- IEinsum
Layer - IEinsumLayer
- IElement
Wise Layer - IElementWiseLayer
- IEngine
Inspector - IEngineInspector
- IError
Recorder - IExecution
Context - IExecutionContext
- IFill
Layer - IFillLayer
- IGather
Layer - IGatherLayer
- IGpu
Allocator - IGrid
Sample Layer - IGridSampleLayer
- IHost
Memory - IHostMemory
- IIdentity
Layer - IIdentityLayer
- IIfConditional
- IIfConditional
- IIfConditional
Input Layer - IIfConditionalInputLayer
- IIfConditional
Output Layer - IIfConditionalOutputLayer
- IIterator
Layer - IIteratorLayer
- IKVCache
Update Layer - IKVCacheUpdateLayer
- ILRN
Layer - ILRNLayer
- ILayer
- ILayer
- ILogger
- ILogger
- ILoop
- ILoop
- ILoop
Boundary Layer - ILoopBoundaryLayer
- ILoop
Output Layer - ILoopOutputLayer
- IMatrix
Multiply Layer - IMatrixMultiplyLayer
- IMoE
Layer ┌──────────────┐┌────────────────────────┐┌────────────────────────┐ │ hiddenStates ││selectedExpertsForTokens││scoresForSelectedExperts│ └──────────────┘└────────────────────────┘└────────────────────────┘ │ │ │ │ │ │ ┌───────────────────────────────────────────────────────────────────────────────────┐ │ │ │ ┌──────────────────────────┐ ┌──────────────────────────┐ │ │ │ │ Expert 0 │ │ MOE │ │ Expert i │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌────────┐ ┌────────┐│ │ ┌────────┐ ┌────────┐│ │ │ │ │ fcGate │ │ fcUp ││ │ │ fcGate │ │ fcUp ││ │ │ │ │ │ │ ││ │ │ │ │ ││ │ │ │ └───┬────┘ └────┬───┘│ │ └───┬────┘ └────┬───┘│ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌──────────┐ │ │ │ ┌──────────┐ │ │ │ │ │ │activation│ │ │ │ │activation│ │ │ │ │ │ └────┬─────┘ │ │ │ └────┬─────┘ │ │ │ │ │ │ │ │ ....... │ │ │ │ │ │ │ └──────┬───────┘ │ │ └──────┬───────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌────────┐ │ │ ┌────────┐ │ │ │ │ │ mul │ │ │ │ mul │ │ │ │ │ └───┬────┘ │ │ └───┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌───▼────┐ │ │ ┌───▼────┐ │ │ │ │ │ fcDown │ │ │ │ fcDown │ │ │ │ │ └───┬────┘ │ │ └───┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌───▼────┐ │ │ ┌───▼────┐ │ │ │ │ │output 0│ │ │ │output i│ │ │ │ │ └───┬────┘ │ │ └───┬────┘ │ │ │ └─────────────┼────────────┘ └─────────────┼────────────┘ │ │ │ │ │ │ └───────────────────┬───────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────┐ │ │ │ weightedSum │ │ │ └───────┬───────┘ │ └────────────────────────────────────│──────────────────────────────────────────────┘ ▼ ┌───────────────┐ │ moeOutput │ └───────────────┘
- INMS
Layer - INMSLayer
- INetwork
Definition - INetworkDefinition
- INon
Zero Layer - INonZero
- INormalization
Layer - INormalizationLayer
- IOne
HotLayer - IOneHotLayer
- IOptimization
Profile - IOptimizationProfile Optimization profile for dynamic input dimensions and shape tensors.
- IOutput
Allocator - IPadding
Layer - IPaddingLayer
- IParametric
ReLU Layer - IParametricReLULayer
- IPlugin
Registry - IPluginRegistry
- IPooling
Layer - IPoolingLayer
- IProfiler
- IProgress
Monitor - IQuantize
Layer - IQuantizeLayer
- IRagged
Soft MaxLayer - IRaggedSoftMaxLayer
- IRecurrence
Layer - IRecurrenceLayer
- IReduce
Layer - IReduceLayer
- IRefitter
- IRefitter
- IResize
Layer - IResizeLayer
- IReverse
Sequence Layer - IReverseSequenceLayer
- IRotary
Embedding Layer - IRotaryEmbeddingLayer
- IRuntime
- IRuntime
- IRuntime
Cache - IRuntimeCache
- IRuntime
Config - IRuntimeConfig
- IScale
Layer - IScaleLayer
- IScatter
Layer - IScatterLayer
- ISelect
Layer - ISelectLayer
- ISerialization
Config - ISerializationConfig
- IShape
Layer - IShapeLayer
- IShuffle
Layer - IShuffleLayer
- ISlice
Layer - Slices an input tensor into an output tensor based on the offset and strides.
- ISoft
MaxLayer - ISoftMaxLayer
- ISqueeze
Layer - ISqueezeLayer
- IStream
Reader V2 - IStream
Writer - ITensor
- ITensor
- ITiming
Cache - ITimingCache
- ITopK
Layer - ITopKLayer
- ITrip
Limit Layer - ITripLimitLayer
- IUnary
Layer - IUnaryLayer
- IUnsqueeze
Layer - IUnsqueezeLayer
- IVersioned
Interface - IVersionedInterface
- Interface
Info - InterfaceInfo
- Permutation
- Permutation
- Weights
- Weights
Enums§
- APILanguage
- APILanguage
- Activation
Type - ActivationType
- AttentionIO
Form - AttentionIOForm
- Attention
Normalization Op - AttentionNormalizationOp
- Bounding
BoxFormat - BoundingBoxFormat
- Builder
Flag - BuilderFlag
- Causal
Mask Kind - CausalMaskKind
- Collective
Operation - CollectiveOperation
- Compute
Capability - ComputeCapability
- Cuda
Graph Strategy - CudaGraphStrategy
- Cumulative
Operation - CumulativeOperation
- Data
Type - DataType The type of weights and tensors. The datatypes other than kBOOL, kINT32, and kINT64 are “activation datatypes,” as they often represent values corresponding to inference results.
- Device
Type - DeviceType The device that this layer/network will execute on.
- Dynamic
Shapes Kernel Specialization Strategy - DynamicShapesKernelSpecializationStrategy
- Element
Wise Operation - ElementWiseOperation
- Engine
Capability - EngineCapability
- Engine
Stat - EngineStat
- Engine
Validity - enum EngineValidity
Whether a TensorRT-RTX engine is likely to be valid on the current system.
The value is determined by only inspecting the header, so even an engine marked
kVALID or kSUBOPTIMAL may be invalid if the body has become corrupted.
See [
IRuntime::getEngineValidity()] - Execution
Context Allocation Strategy - ExecutionContextAllocationStrategy
- Fill
Operation - FillOperation
- Gather
Mode - Control form of IGatherLayer
- Hardware
Compatibility Level - HardwareCompatibilityLevel
- Interpolation
Mode - InterpolationMode
- KVCache
Mode - KVCacheMode
- Layer
Information Format - LayerInformationFormat
- Layer
Type - LayerType
- Loop
Output - LoopOutput
- Matrix
Operation - MatrixOperation
- Memory
Pool Type - MemoryPoolType
- MoEAct
Type - MoEActType
- Network
Definition Creation Flag - NetworkDefinitionCreationFlag
- OptProfile
Selector - OptProfileSelector
- Padding
Mode - PaddingMode
- Pooling
Type - PoolingType
- Preview
Feature - PreviewFeature
- Profiling
Verbosity - ProfilingVerbosity
- Reduce
Operation - ReduceOperation
- Resize
Coordinate Transformation - ResizeCoordinateTransformation
- Resize
Round Mode - ResizeRoundMode
- Resize
Selector - ResizeSelector
- Runtime
Platform - RuntimePlatform
- Sample
Mode - Controls how ISliceLayer and IGridSample handle out-of-bounds coordinates.
- Scale
Mode - Controls how shift, scale and power are applied in a Scale layer.
- Scatter
Mode - ScatterMode
- Seek
Position - SeekPosition Controls the seek mode of IStreamReaderV2.
- Serialization
Flag - SerializationFlag
- Tactic
Source - TacticSource
- Tensor
Format - TensorFormat
- TensorIO
Mode - TensorIOMode
- Tensor
Location - TensorLocation
- Tiling
Optimization Level - TilingOptimizationLevel
- TopK
Operation - TopKOperation
- Trip
Limit - TripLimit
- Unary
Operation - UnaryOperation
- Weights
Role - WeightsRole
Type Aliases§
- Allocator
Flags - Builder
Flags - Represents one or more BuilderFlag values using binary OR operations, e.g., 1U << BuilderFlag::kDEBUG.
- Dims
- Alias for Dims64.
- IError
Recorder_ Error Desc - A typedef of a C-style string for reporting error descriptions.
- IError
Recorder_ RefCount - A typedef of a 32-bit integer for reference counting.
- Network
Definition Creation Flags - Represents one or more NetworkDefinitionCreationFlag flags using binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kPREFER_JIT_PYTHON_PLUGINS
- Serialization
Flags - Represents one or more SerializationFlag values using binary OR operations, e.g., 1U << SerializationFlag::kEXCLUDE_LEAN_RUNTIME
- Tactic
Sources - Represents a collection of one or more TacticSource values combine using bitwise-OR operations.
- Tempfile
Control Flags - Represents a collection of one or more TempfileControlFlag values combined using bitwise-OR operations.
- Tensor
Formats - It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormat::kCHW4 | 1U << TensorFormat::kCHW32.