Struct InferenceConfig 

pub struct InferenceConfig {

    pub confidence_threshold: f32,
    pub iou_threshold: f32,
    pub max_det: usize,
    pub imgsz: Option<(usize, usize)>,
    pub batch: Option<usize>,
    pub num_threads: usize,
    pub quantize: Option<Quantization>,
    pub device: Option<Device>,
    pub save: bool,
    pub save_frames: bool,
    pub rect: bool,
    pub classes: Option<Vec<usize>>,
    pub cuda_preprocess: bool,
    pub cuda_memory_limit: Option<usize>,
    /* private fields */
}

Configuration for YOLO inference.

This struct is used to customize the behavior of the inference engine. It uses a builder pattern for convenient construction.

Examples

Basic configuration:

use ultralytics_inference::InferenceConfig;

let config = InferenceConfig::new()
    .with_confidence(0.5)
    .with_iou(0.45)
    .with_max_det(300)
    .with_imgsz(640, 640);

With specific hardware device:

use ultralytics_inference::{InferenceConfig, Device};

let config = InferenceConfig::new()
    .with_confidence(0.5)
    .with_device(Device::Cuda(0));

Fields

confidence_threshold: f32

Confidence threshold for detections (0.0 to 1.0). Detections with confidence scores lower than this value will be discarded.

iou_threshold: f32

Intersection over Union (IoU) threshold for Non-Maximum Suppression (NMS) (0.0 to 1.0). Used to merge overlapping boxes. Lower values filter more duplicates.

max_det: usize

Maximum number of detections to return per image. The top-k detections sorted by confidence will be returned.

imgsz: Option<(usize, usize)>

Explicit input image size (height, width). If None, the model’s metadata will be used to determine input size.

batch: Option<usize>

Batch size for inference when using BatchProcessor. If None, defaults to 1 (single-image inference).

num_threads: usize

Number of intra-op threads for ONNX Runtime. Setting this to 0 lets YOLOModel::load resolve it to std::thread::available_parallelism when it builds the session, falling back to 4 if that cannot be determined.

quantize: Option<Quantization>

Requested inference precision. None uses the model’s native precision.

device: Option<Device>

Hardware device to use for inference. If None, the best available device will be automatically selected.

save: bool

Whether to save annotated results. Defaults to true.

save_frames: bool

Whether to save individual frames instead of a video file when input is video. Defaults to false (save as video).

rect: bool

Whether to use minimal padding (rectangular inference). Defaults to true.

classes: Option<Vec<usize>>

Class IDs to filter predictions. If None, all classes are returned. Useful for focusing on specific objects in multi-class detection tasks.

cuda_preprocess: bool

Use the CUDA preprocess fast path when available.

Defaults to true. The flag is only consulted when the crate was compiled with the cuda-preprocess feature and the selected device is CUDA or TensorRT (or one of those EPs is registered by default). In every other configuration the value is ignored and the standard CPU preprocess path runs.

cuda_memory_limit: Option<usize>

Upper bound, in bytes, on the CUDA execution provider’s memory arena.

None (default) lets the arena grow as far as the device allows. Only consulted with the cuda feature and a CUDA device: a TensorRT device ignores it, as does a limit of 0. The cap covers the arena alone, so peak device memory stays well above it.

Implementations

impl InferenceConfig

pub const DEFAULT_CONF: f32 = 0.25

Default confidence threshold (0.0 to 1.0).

pub const DEFAULT_IOU: f32 = 0.7

Default IoU threshold for NMS (0.0 to 1.0).

pub const DEFAULT_MAX_DET: usize = 300

Default maximum number of detections per image.

pub const DEFAULT_QUANTIZE: Option<Quantization> = None

Default inference precision. None uses the model’s native precision.

pub const DEFAULT_SAVE: bool = true

Default for saving annotated results.

pub const DEFAULT_SAVE_FRAMES: bool = false

Default for saving individual frames (vs video).

pub const DEFAULT_RECT: bool = true

Default for rectangular (minimal padding) inference.

pub const DEFAULT_IMGSZ: (usize, usize)

Default input image size for standard YOLO models (height, width).

pub const DEFAULT_OBB_IMGSZ: (usize, usize)

Default input image size for OBB models (height, width).

pub const DEFAULT_CUDA_PREPROCESS: bool = true

Default for the CUDA preprocess fast path: on whenever the crate is built with the cuda-preprocess feature and the device permits it.

pub fn new() -> Self

Create a new configuration with default values.

Returns

• A new InferenceConfig instance with default settings.

pub const fn with_batch(self, batch: usize) -> Self

Set the batch size.

Arguments

• batch - The batch size.

Returns

• The modified InferenceConfig.

pub const fn with_confidence(self, threshold: f32) -> Self

Set the confidence threshold.

Detections with a confidence score below this threshold will be filtered out.

Arguments

• threshold - The minimum confidence score (0.0 to 1.0).

Returns

• The modified InferenceConfig.

pub const fn with_iou(self, threshold: f32) -> Self

Set the IoU threshold for Non-Maximum Suppression (NMS).

NMS suppresses overlapping bounding boxes. This threshold determines how much overlap is allowed before boxes are considered duplicates.

Arguments

• threshold - The IoU threshold (0.0 to 1.0).

Returns

• The modified InferenceConfig.

pub const fn with_max_det(self, max: usize) -> Self

Set the maximum number of detections to return.

Only the top max detections (sorted by confidence) will be kept after NMS.

Arguments

• max - The maximum number of detections.

Returns

• The modified InferenceConfig.

pub const fn with_imgsz(self, height: usize, width: usize) -> Self

Set the input image size.

This explicitly sets the size to resize images to before inference. If not set, the model’s internal metadata size will be used.

Arguments

• height - The target image height.
• width - The target image width.

Returns

• The modified InferenceConfig.

pub const fn with_threads(self, threads: usize) -> Self

Set the number of threads for inference.

Arguments

• threads - The number of intra-op threads. Set to 0 to use every available core.

Returns

• The modified InferenceConfig.

pub const fn with_quantize(self, quantize: Quantization) -> Self

Set the requested inference precision.

The accepted schemes match the Python package’s quantize argument. On CPU this selects the requested precision where the execution provider supports it; an FP16 ONNX model always runs at FP32 weights there, because ONNX Runtime widens the graph while building the session.

Returns

• The modified InferenceConfig.

pub const fn with_cuda_preprocess(self, enabled: bool) -> Self

Enable or disable the CUDA preprocess fast path.

When true (default), and the crate was built with the cuda-preprocess feature, and the selected device is CUDA or TensorRT, YOLOModel::predict_image dispatches to a fused CUDA kernel for letterbox + normalize + HWC→CHW and feeds the result to ORT as a zero-copy device tensor. Set to false to force the standard CPU preprocess path even when the feature is available.

In any configuration where the fast path can’t run (feature off, non-CUDA device, or runtime fallback), the value is silently ignored.

pub const fn with_device(self, device: Device) -> Self

Set the hardware device for inference.

Arguments

• device - The device to use (e.g. Device::Cpu, Device::Cuda(0), Device::CoreMl, Device::IntelGpu).

Example

use ultralytics_inference::{Device, InferenceConfig};

let config = InferenceConfig::new()
    .with_device(Device::CoreMl); // CoreML on Apple Silicon

// OpenVINO on Intel hardware (intel:cpu, intel:gpu, intel:npu)
let intel = InferenceConfig::new()
    .with_device(Device::IntelGpu);

Returns

• The modified InferenceConfig.

pub const fn with_save(self, save: bool) -> Self

Set whether to save annotated results.

Arguments

• save - true to save results, false to skip saving.

Returns

• The modified InferenceConfig.

pub const fn with_save_frames(self, save_frames: bool) -> Self

Set whether to save individual frames for video inputs.

Arguments

• save_frames - true to save frames, false to save as video.

Returns

• The modified InferenceConfig.

pub const fn with_rect(self, rect: bool) -> Self

Set whether to use minimal padding (rectangular inference).

Arguments

• rect - true to enable, false to disable.

Returns

• The modified InferenceConfig.

pub fn with_classes(self, classes: Vec<usize>) -> Self

Set the class IDs to filter predictions.

Only detections belonging to the specified classes will be returned.

Arguments

• classes - A vector of class IDs to keep.

Example

use ultralytics_inference::InferenceConfig;

// Only detect persons (class 0) and cars (class 2)
let config = InferenceConfig::new()
    .with_classes(vec![0, 2]);

Returns

• The modified InferenceConfig.

pub fn keep_class(&self, class_id: usize) -> bool

Check if a class should be included in the results.

Arguments

• class_id - The class index to check.

Returns

• true if the class should be kept.
• false if the class should be filtered out.

pub const fn with_cuda_memory_limit(self, limit: usize) -> Self

Cap the CUDA execution provider’s memory arena at limit bytes.

Use this to stop ONNX Runtime from reserving most of the GPU so the device can be shared with other processes. Has no effect unless the crate is built with the cuda feature and a CUDA device is selected. A limit the graph cannot fit in fails the load with an ONNX Runtime arena error, so leave the model room to run.

Example

use ultralytics_inference::InferenceConfig;

// Limit the CUDA arena to 2 GiB.
let config = InferenceConfig::new().with_cuda_memory_limit(2 * 1024 * 1024 * 1024);

Returns

• The modified InferenceConfig.

Trait Implementations

impl Clone for InferenceConfig

fn clone(&self) -> InferenceConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferenceConfig

fn default() -> Self

Returns the “default value” for a type.