🦀 Ultralytics YOLO Rust Inference

High-performance YOLO inference library written in Rust. This library provides a fast, safe, and efficient interface for running YOLO models using ONNX Runtime, with an API designed to match the Ultralytics Python package.

✨ Features

🚀 High Performance - Pure Rust implementation with zero-cost abstractions
🎯 Ultralytics API Compatible - Results, Boxes, Masks, Keypoints, Probs classes matching Python
🔧 Multiple Backends - CPU, CUDA, TensorRT, CoreML, OpenVINO, and more via ONNX Runtime
📦 Dual Use - Library for Rust projects + standalone CLI application
🏷️ Auto Metadata - Automatically reads class names, task type, and input size from ONNX models
⬇️ Auto Download - Automatically downloads YOLO11 and YOLO26 ONNX models (all sizes: n/s/m/l/x) when not found locally
🖼️ Multiple Sources - Images, directories, glob patterns, video files, webcams, and streams
🪶 Lean Runtime - No PyTorch, TensorFlow, or Python runtime required

🚀 Quick Start

Prerequisites

Rust 1.88+ (install via rustup)
A YOLO ONNX model (export from Ultralytics: yolo export model=yolo26n.pt format=onnx)

Installation

# Install CLI globally from crates.io
cargo install ultralytics-inference

# Install CLI globally with custom features
# Minimal build (no default features)
cargo install ultralytics-inference --no-default-features

# Enable video support
cargo install ultralytics-inference --features video

# Enable multiple accelerators
cargo install ultralytics-inference --features "cuda,tensorrt"

Development install

# Install CLI directly from the git repository
cargo install --git https://github.com/ultralytics/inference.git ultralytics-inference

# Or clone, build, and install from source
git clone https://github.com/ultralytics/inference.git
cd inference
cargo build --release

# Install from local checkout
cargo install --path . --locked

cargo install places binaries in Cargo's default bin directory:

macOS/Linux: ~/.cargo/bin
Windows: %USERPROFILE%\\.cargo\\bin

Ensure this directory is in your PATH, then run from anywhere:

ultralytics-inference help

Export a YOLO Model to ONNX

# Using Ultralytics CLI
yolo export model=yolo26n.pt format=onnx

# Or with Python
from ultralytics import YOLO
model = YOLO("yolo26n.pt")
model.export(format="onnx")

Run Inference

# With defaults (auto-downloads yolo26n.onnx and sample images)
ultralytics-inference predict

# Select task — auto-downloads the nano model for that task
ultralytics-inference predict --task segment  # downloads yolo26n-seg.onnx
ultralytics-inference predict --task pose     # downloads yolo26n-pose.onnx
ultralytics-inference predict --task obb      # downloads yolo26n-obb.onnx
ultralytics-inference predict --task classify # downloads yolo26n-cls.onnx

# With explicit model (task is read from model metadata)
ultralytics-inference predict --model yolo26n.onnx --source image.jpg

# Auto-download any supported size (n/s/m/l/x)
ultralytics-inference predict --model yolo26l.onnx --source image.jpg
ultralytics-inference predict --model yolo11x-seg.onnx --source image.jpg

# On a directory of images
ultralytics-inference predict --model yolo26n.onnx --source assets/

# With custom thresholds
ultralytics-inference predict -m yolo26n.onnx -s image.jpg --conf 0.5 --iou 0.45

# Filter by class IDs
ultralytics-inference predict --model yolo26n.onnx --source image.jpg --classes 0
ultralytics-inference predict --model yolo26n.onnx --source image.jpg --classes "0,1,2"

# With visualization and custom image size
ultralytics-inference predict --model yolo26n.onnx --source video.mp4 --show --imgsz 1280

# Save individual frames for video input
ultralytics-inference predict --model yolo26n.onnx --source video.mp4 --save-frames

# Rectangular inference
ultralytics-inference predict --model yolo26n.onnx --source image.jpg --rect

Example Output

# ultralytics-inference predict

WARNING ⚠️ 'model' argument is missing. Using default '--model=yolo26n.onnx'.
WARNING ⚠️ 'source' argument is missing. Using default images: https://ultralytics.com/images/bus.jpg, https://ultralytics.com/images/zidane.jpg
Ultralytics 0.0.13 🚀 Rust ONNX FP32 CPU
Using ONNX Runtime CPUExecutionProvider
YOLO26n summary: 80 classes, imgsz=(640, 640)

image 1/2 /home/ultralytics/inference/bus.jpg: 640x480 640x480 4 persons, 1 bus, 36.4ms
image 2/2 /home/ultralytics/inference/zidane.jpg: 384x640 2 persons, 1 tie, 28.6ms
Speed: 1.5ms preprocess, 32.5ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
Results saved to runs/detect/predict1
💡 Learn more at https://docs.ultralytics.com/modes/predict

With --task (auto-downloads the matching nano model):

# ultralytics-inference predict --task segment

WARNING ⚠️ 'model' argument is missing. Using default '--model=yolo26n-seg.onnx'.
WARNING ⚠️ 'source' argument is missing. Using default images: https://ultralytics.com/images/bus.jpg, https://ultralytics.com/images/zidane.jpg
Ultralytics 0.0.13 🚀 Rust ONNX FP32 CPU
Using ONNX Runtime CPUExecutionProvider
YOLO26n-seg summary: 80 classes, imgsz=(640, 640)

image 1/2 /home/ultralytics/inference/bus.jpg: 640x480 4 persons, 1 bus, 48.2ms
image 2/2 /home/ultralytics/inference/zidane.jpg: 384x640 2 persons, 1 tie, 38.1ms
Speed: 1.6ms preprocess, 44.3ms inference, 1.2ms postprocess per image at shape (1, 3, 384, 640)
Results saved to runs/segment/predict1
💡 Learn more at https://docs.ultralytics.com/modes/predict

📚 Usage

As a CLI Tool

# Show help
ultralytics-inference help

# Show version
ultralytics-inference version

# Run inference
ultralytics-inference predict --model <model.onnx> --source <source>

--help and --version are also supported as standard flag aliases.

CLI Options:

Option	Short	Description	Default
`--model`	`-m`	Path to ONNX model file; auto-downloaded if a known YOLO11/YOLO26 name	`yolo26n.onnx`
`--task`		Task type (`detect`, `segment`, `pose`, `obb`, `classify`); selects nano model when `--model` is omitted	`detect`
`--source`	`-s`	Input source (image, directory, glob, video, webcam index, or URL)	`Task dependent Ultralytics URL assets`
`--conf`		Confidence threshold	`0.25`
`--iou`		IoU threshold for NMS	`0.7`
`--max-det`		Maximum number of detections	`300`
`--imgsz`		Inference image size	`Model metadata`
`--rect`		Enable rectangular inference (minimal padding)	`true`
`--batch`		Batch size for inference	`1`
`--half`		Use FP16 half-precision inference	`false`
`--save`		Save annotated results to runs/<task>/predict	`true`
`--save-frames`		Save individual frames for video	`false`
`--show`		Display results in a window	`false`
`--device`		Device (cpu, cuda:0, mps, coreml, directml:0, openvino, tensorrt:0, xnnpack)	`cpu`
`--verbose`		Show verbose output	`true`
`--classes`		Filter by class IDs, e.g. `0` or `"0,1,2"` or `"[0, 1, 2]"`	all classes

Task and Model Resolution:

Invocation	Model used	Notes
`predict`	`yolo26n.onnx`	Default detect model, auto-downloaded
`predict --task segment`	`yolo26n-seg.onnx`	Nano seg model, auto-downloaded
`predict --task pose`	`yolo26n-pose.onnx`	Nano pose model, auto-downloaded
`predict --task obb`	`yolo26n-obb.onnx`	Nano OBB model, auto-downloaded
`predict --task classify`	`yolo26n-cls.onnx`	Nano classify model, auto-downloaded
`predict --model yolo26l-seg.onnx`	`yolo26l-seg.onnx`	Task read from model metadata
`predict --task segment --model yolo26l-seg.onnx`	`yolo26l-seg.onnx`	`--task` matches metadata, proceeds normally
`predict --task segment --model yolo26n.onnx`	error	`--task` conflicts with model metadata (`detect`), exits with error

Auto-downloadable models:

All YOLO11 and YOLO26 ONNX models in sizes n / s / m / l / x across all five task variants are supported for auto-download:

Family	Variants
YOLO26	`yolo26{n,s,m,l,x}.onnx`, `yolo26{n,s,m,l,x}-seg.onnx`, `-pose`, `-obb`, `-cls`
YOLO11	`yolo11{n,s,m,l,x}.onnx`, `yolo11{n,s,m,l,x}-seg.onnx`, `-pose`, `-obb`, `-cls`

Source Options:

Source Type	Example Input	Description
Image	`image.jpg`	Single image file
Directory	`images/`	Directory of images
Glob	`images/*.jpg`	Glob pattern for images
Video	`video.mp4`	Video file
Webcam	`0`,`1`	Webcam index (0 = default webcam)
URL	`https://example.com/image.jpg`	Remote image URL

As a Rust Library

Add to your Cargo.toml (choose one):

# Stable release from crates.io
[dependencies]
ultralytics-inference = "0.0.13"

# Development version (latest unreleased code from GitHub)
[dependencies]
ultralytics-inference = { git = "https://github.com/ultralytics/inference.git" }

Basic Usage:

use ultralytics_inference::{YOLOModel, InferenceConfig};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load model - metadata (classes, task, imgsz) is read automatically
    let mut model = YOLOModel::load("yolo26n.onnx")?;

    // Run inference
    let results = model.predict("image.jpg")?;

    // Process results
    for result in &results {
        if let Some(ref boxes) = result.boxes {
            println!("Found {} detections", boxes.len());
            for i in 0..boxes.len() {
                let cls = boxes.cls()[i] as usize;
                let conf = boxes.conf()[i];
                let name = result.names.get(&cls).map(|s| s.as_str()).unwrap_or("unknown");
                println!("  {} {:.2}", name, conf);
            }
        }
    }

    Ok(())
}

With Custom Configuration:

use ultralytics_inference::{YOLOModel, InferenceConfig};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = InferenceConfig::new()
        .with_confidence(0.5)
        .with_iou(0.45)
        .with_max_det(300);

    let mut model = YOLOModel::load_with_config("yolo26n.onnx", config)?;
    let results = model.predict("image.jpg")?;

    Ok(())
}

Accessing Detection Data:

if let Some(ref boxes) = result.boxes {
    // Bounding boxes in different formats
    let xyxy = boxes.xyxy();      // [x1, y1, x2, y2]
    let xywh = boxes.xywh();      // [x_center, y_center, width, height]
    let xyxyn = boxes.xyxyn();    // Normalized [0-1]
    let xywhn = boxes.xywhn();    // Normalized [0-1]

    // Confidence scores and class IDs
    let conf = boxes.conf();      // Confidence scores
    let cls = boxes.cls();        // Class IDs
}

Selecting a Device:

use ultralytics_inference::{Device, InferenceConfig, YOLOModel};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Select a device (e.g., CUDA, MPS, CPU)
    let device = Device::Cuda(0);

    // Configure the model to use this device
    let config = InferenceConfig::new().with_device(device);

    let mut model = YOLOModel::load_with_config("yolo26n.onnx", config)?;
    let results = model.predict("image.jpg")?;

    Ok(())
}

🗂️ Project Structure

inference/
├── src/
│   ├── lib.rs              # Library entry point and public exports
│   ├── main.rs             # CLI application
│   ├── model.rs            # YOLOModel - ONNX session and inference
│   ├── results.rs          # Results, Boxes, Masks, Keypoints, Probs, Obb
│   ├── preprocessing.rs    # Image preprocessing (letterbox, normalize, SIMD)
│   ├── postprocessing.rs   # Detection post-processing (NMS, decode, SIMD)
│   ├── metadata.rs         # ONNX model metadata parsing
│   ├── source.rs           # Input source handling (images, video, webcam)
│   ├── task.rs             # Task enum (Detect, Segment, Pose, Classify, Obb)
│   ├── inference.rs        # InferenceConfig
│   ├── batch.rs            # Batch processing pipeline
│   ├── device.rs           # Device enum (CPU, CUDA, MPS, CoreML, etc.)
│   ├── download.rs         # Model and asset downloading
│   ├── annotate.rs         # Image annotation (bounding boxes, masks, keypoints)
│   ├── io.rs               # Result saving (images, videos)
│   ├── logging.rs          # Logging macros
│   ├── error.rs            # Error types
│   ├── utils.rs            # Utility functions (NMS, IoU)
│   ├── cli/                # CLI module
│   │   ├── mod.rs          # CLI module exports
│   │   ├── args.rs         # CLI argument parsing
│   │   └── predict.rs      # Predict command implementation
│   └── visualizer/         # Real-time visualization (minifb)
├── tests/
│   └── integration_test.rs # Integration tests
├── assets/                 # Test images
│   ├── bus.jpg
│   └── zidane.jpg
├── Cargo.toml              # Rust dependencies and features
├── LICENSE                 # AGPL-3.0 License
└── README.md               # This file

⚡ Hardware Acceleration

Enable hardware acceleration by adding features to your build:

# NVIDIA GPU (CUDA)
cargo build --release --features cuda

# NVIDIA TensorRT
cargo build --release --features tensorrt

# Apple CoreML (macOS/iOS)
cargo build --release --features coreml

# Intel OpenVINO
cargo build --release --features openvino

# Multiple features
cargo build --release --features "cuda,tensorrt"

Available Features:

Feature	Description
`cuda`	NVIDIA CUDA support
`tensorrt`	NVIDIA TensorRT optimization
`coreml`	Apple CoreML (macOS/iOS)
`openvino`	Intel OpenVINO
`onednn`	Intel oneDNN
`rocm`	AMD ROCm
`directml`	DirectML (Windows)
`nnapi`	Android Neural Networks API
`xnnpack`	XNNPACK (cross-platform)
`nvidia`	Convenience: CUDA + TensorRT
`intel`	Convenience: OpenVINO + oneDNN
`mobile`	Convenience: NNAPI + CoreML + QNN

📦 Dependencies

One of the key benefits of this library is a Rust/ONNX Runtime stack with no PyTorch, TensorFlow, or Python runtime required.

Core Dependencies (always included)

Crate	Purpose
`ort`	ONNX Runtime bindings
`ndarray`	N-dimensional arrays
`image`	Image loading/decoding
`jpeg-decoder`	JPEG decoding
`fast_image_resize`	SIMD-optimized resizing
`half`	FP16 support
`lru`	LRU cache for preprocessing LUT
`wide`	SIMD for fast preprocessing

Optional Dependencies (for `--save` feature)

Crate	Purpose
`imageproc`	Drawing boxes and shapes
`ab_glyph`	Text rendering (embedded font)

Optional Dependencies (for Video & Visualization)

Crate	Purpose
`minifb`	Window creation and buffer display
`video-rs`	Video decoding/encoding (ffmpeg)

Video Support (FFmpeg)

Video features require FFmpeg (7 or 8) installed on your system:

# macOS
brew install ffmpeg

# Ubuntu/Debian
apt-get install -y ffmpeg libavutil-dev libavformat-dev libavfilter-dev libavdevice-dev libclang-dev

# Build with video support
cargo build --release --features video

To build without annotation support (smaller binary):

cargo build --release --no-default-features

🧪 Testing

# Run all tests
cargo test

# Run with output
cargo test -- --nocapture

# Run specific test
cargo test test_boxes_creation

📊 Performance

Benchmarks on Apple M4 MacBook Pro (CPU, ONNX Runtime):

YOLO26n Detection Model (640x640)

Precision	Model Size	Preprocess	Inference	Postprocess	Total
FP32	10.2 MB	~9ms	~21ms	<1ms	~31ms
FP16	5.2 MB	~9ms	~24ms	<1ms	~34ms

Key findings:

FP16 models are ~50% smaller (5.2 MB vs 10.2 MB)
FP32 is slightly faster on CPU (~21ms vs ~24ms) due to CPU's native FP32 support
FP16 requires upcasting to FP32 for computation on most CPUs, adding overhead
Use FP32 for CPU inference, FP16 for GPU (where it provides speedup)

Threading Optimization

ONNX Runtime threading is set to auto (num_threads: 0) which lets ORT choose optimal thread count:

Manual threading (4 threads): ~40ms inference
Auto threading (0 = ORT decides): ~21ms inference

🔮 Roadmap

Completed

Detection, Segmentation, Pose, Classification, OBB inference
ONNX model metadata parsing (auto-detect classes, task, imgsz)
Hardware acceleration support (CUDA, TensorRT, CoreML, OpenVINO, XNNPACK)
Ultralytics-compatible Results API (Boxes, Masks, Keypoints, Probs, Obb)
Multiple input sources (images, directories, globs, URLs)
Video file support and webcam/RTSP streaming
Image annotation and visualization
FP16 half-precision inference
Batch inference support
Rectangular inference support and optimization
Class filtering support
Auto-download all YOLO11 and YOLO26 ONNX models (all sizes n/s/m/l/x, all tasks)
--task CLI flag: selects and auto-downloads the matching nano model when --model is omitted; errors on task/model metadata conflict

In Progress

Python bindings (PyO3)
WebAssembly (WASM) support for browser inference

💡 Contributing

Ultralytics thrives on community collaboration, and we deeply value your contributions! Whether it's reporting bugs, suggesting features, or submitting code changes, your involvement is crucial.

Report Issues: Found a bug? Open an issue
Feature Requests: Have an idea? Share it
Pull Requests: Read our Contributing Guide first
Feedback: Take our Survey

A heartfelt thank you 🙏 goes out to all our contributors! Your efforts help make Ultralytics tools better for everyone.

📜 License

Ultralytics offers two licensing options to suit different needs:

AGPL-3.0 License: This OSI-approved open-source license is perfect for students, researchers, and enthusiasts. It encourages open collaboration and knowledge sharing. See the LICENSE file for full details.
Ultralytics Enterprise License: Designed for commercial use, this license allows for the seamless integration of Ultralytics software and AI models into commercial products and services, bypassing the open-source requirements of AGPL-3.0. If your use case involves commercial deployment, please contact us via Ultralytics Licensing.

📮 Contact

GitHub Issues: Bug reports and feature requests
Discord: Join our community
Documentation: docs.ultralytics.com

ultralytics-inference 0.0.13