Struct TensorRtOptions 

pub struct TensorRtOptions {
    pub device_id: i32,
    pub fp16: bool,
    pub tf32: bool,
    pub cuda_graph: bool,
    pub workspace_bytes: usize,
    pub builder_optimization_level: u8,
    pub auxiliary_streams: i8,
    pub sparsity: bool,
    pub min_shapes: Option<String>,
    pub opt_shapes: Option<String>,
    pub max_shapes: Option<String>,
}

Build and execution options used only by TensorRT.

Fields

device_id: i32

GPU ordinal, default 0.

fp16: bool

Allow FP16 computations internally; does not change model I/O types. Default false.

tf32: bool

Allow TF32 for TensorRT FP32 operations (default true). The process-wide NVIDIA_TF32_OVERRIDE must be unset for true, or 0 for false.

cuda_graph: bool

Enable CUDA graphs for fixed-shape, supported tensor I/O.

workspace_bytes: usize

Maximum builder workspace in bytes, default 4 GiB on 64-bit targets.

builder_optimization_level: u8

Builder optimization level from 0 through 5, default 3.

auxiliary_streams: i8

Maximum auxiliary streams; -1 lets TensorRT choose (the default).

sparsity: bool

Allow sparse weights where supported, default true.

min_shapes: Option<String>

ORT syntax, e.g. images:1x3x640x640. Set all three for dynamic models.

opt_shapes: Option<String>

Preferred dynamic shapes, in the same ORT syntax as Self::min_shapes.

max_shapes: Option<String>

Maximum dynamic shapes, in the same ORT syntax as Self::min_shapes.

Trait Implementations

impl Clone for TensorRtOptions

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TensorRtOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TensorRtOptions

fn default() -> Self

Returns the “default value” for a type.