warden-vision

Reusable computer vision building blocks for detection, tracking, and annotation — detection/keypoint/classification data structures (including oriented bounding boxes), geometry and counting zones, three multi-object trackers, ~20 annotators, evaluation metrics, dataset I/O, and video/image utilities — in the spirit of Roboflow's Python supervision library, reimplemented as a dependency-light, feature-gated Rust crate.

This is an independent project, not affiliated with or endorsed by Roboflow.

Why

Python supervision is the de facto toolkit for gluing together an object detector's output, a tracker, and on-screen annotation. warden-vision covers the same ground natively in Rust — no Python runtime, no PyTorch — so it fits into Rust inference pipelines (e.g. using ort, tract, or candle to run the model itself) without a foreign-language boundary between inference and post-processing.

Features at a glance

• Core data model — Detections/Detection (boxes, masks, oriented bounding boxes via Detection::from_obb, confidence, class, tracker id), KeyPoints (pose/landmarks), Classifications (whole-image classification).
• Detector-output parsing — Detections::from_ultralytics_onnx and Detections::from_transformers_onnx parse the raw tensor layouts YOLOv8+ and DETR-family models produce, independent of which Rust inference runtime you used to run them.
• Geometry — Point, Rect, anchor Positions, polygon utilities, rotated-rectangle IoU (oriented_box_iou), LineZone / PolygonZone crossing/occupancy counters.
• Tracking — SortTracker (Kalman + IoU), ByteTracker (two-stage high/low-confidence matching), and DeepSortTracker (motion + caller- supplied appearance embeddings, for when paths cross), plus DetectionsSmoother for jitter reduction and InferenceSlicer for SAHI-style tiled inference (with a rayon-parallel run_parallel behind the parallel feature).
• Annotation — box, mask, polygon, label (bitmap font or, via ab_glyph, real TTF/OTF), circle, dot, triangle, ellipse, round-box, box-corner, trace, heat map, halo, percentage bar, icon overlay, background dim, blur/pixelate redaction, and keypoint skeleton annotators — all working against a pure-Rust image::RgbaImage backend and (optionally) an OpenCV Mat backend, sharing one configuration type.
• Metrics — Precision, Recall, F1, ConfusionMatrix, MeanAveragePrecision, MeanAverageRecall.
• Dataset I/O — COCO, YOLO, and Pascal VOC detection formats, plus an image-folder classification format.
• Video/image utilities — FPSMonitor, letterbox/resize/crop/overlay, ImageSink, and (via OpenCV videoio) VideoInfo/VideoSink/frame iteration.

Installation

[dependencies]
warden-vision = { version = "0.1", features = ["annotate-image"] }

Feature flags

core has no dependencies beyond ndarray/nalgebra/lapjv/thiserror and builds anywhere Rust does. annotate-opencv needs a system OpenCV install (imgproc + videoio modules) and a working bindgen/libclang toolchain to build its native bindings. parallel is worth it once a frame has hundreds to thousands of candidate boxes/tiles (e.g. merging InferenceSlicer output) — see the benchmark below for why the sequential versions stay the better default at typical detector output sizes.

Quick example

use warden_vision::annotators::{Annotator, BoxAnnotator};
use warden_vision::core::{Detection, Detections};
use warden_vision::tracker::SortTracker;

// Detections from your detector of choice for this frame (xyxy pixel
// boxes). See `Detections::from_ultralytics_onnx` /
// `from_transformers_onnx` if you're running a model directly.
let mut detections = Detections::new(vec![
    Detection::new([120.0, 40.0, 260.0, 300.0], 0.91, 0),
    Detection::new([400.0, 80.0, 520.0, 260.0], 0.76, 1),
]);

// Assign stable identities across frames.
let mut tracker = SortTracker::default();
tracker.update(&mut detections);

// Draw boxes + labels onto an RGBA frame.
let mut frame = image::RgbaImage::new(640, 480);
BoxAnnotator::default().annotate(&mut frame, &detections).unwrap();

Runnable version: cargo run --example quickstart --features annotate-image (see examples/quickstart.rs).

More examples in examples/:

Module overview

Full API docs: cargo doc --open --features annotate-image,datasets.

Relationship to Python supervision

This crate mirrors supervision's API shape and covers its core detection/tracking/annotation/metrics/dataset surface, but is not a line-for-line port:

• Detector adapters parse raw ONNX output tensors (from_ultralytics_onnx, from_transformers_onnx) instead of binding to ultralytics/transformers Python objects, which have no Rust equivalent — see the core::adapters module docs for the reasoning.
• ByteTrack is a two-stage (high/low-confidence) implementation of the core idea, not a byte-for-byte port of the reference codebase.
• DeepSortTracker is a single-pass, motion-gated approximation of DeepSORT's matching-cascade idea (combined IoU + nearest-neighbor cosine distance over a per-track embedding gallery), not a port of the reference implementation either. It also doesn't run the embedding model — you supply one feature vector per detection each frame, same division of responsibility as the ONNX adapters not owning inference.
• Oriented bounding boxes live on Detection::obb/Detection::from_obb rather than a separate OrientedBoxes type; from_obb also populates mask with the same four corners so the existing PolygonAnnotator/ MaskAnnotator render them, instead of shipping a duplicate OrientedBoxAnnotator that would be identical code.
• Not (yet) implemented: RichLabelAnnotator's OpenCV backend (OpenCV has no built-in TTF rendering without FreeType), and a handful of Detections.from_* adapters for detector libraries beyond Ultralytics/transformers.

Benchmark vs Python supervision

examples/bench_vs_supervision.rs and scripts/bench_vs_supervision.py run the same three workloads — NMS, ByteTrack, box+label annotation — over identical synthetic data (a shared xorshift PRNG keeps both sides bit-for-bit comparable; NMS keeps the same 9,394/10,000 boxes on both sides, confirming matching semantics). One data point, single-threaded, no SIMD tuning either side (Intel i9-9880H @ 2.30GHz, supervision 0.29.1):

Reproduce:

cargo run --release --example bench_vs_supervision --features annotate-image,parallel

python3 -m venv .venv && source .venv/bin/activate
pip install supervision numpy
python scripts/bench_vs_supervision.py

Caveats: this is one machine, one run each, no warmup loop, and the NMS case in particular is a worst-case stress test (10k boxes is far more than a single detector head produces) rather than a typical-frame workload — treat the numbers as directional, not a formal benchmark suite.

The NMS row also doubles as a lesson in how not to parallelize greedy NMS: the first non_max_suppression_parallel attempt dispatched a rayon job per surviving candidate at every step of the sequential suppression loop and measured slower than plain sequential NMS — thousands of small, shrinking parallel dispatches lose to their own overhead. Splitting it into one large parallel pass that precomputes the full pairwise IoU relation, followed by a cheap sequential bookkeeping pass over the precomputed booleans, is what actually gets the ~3.5x from 16 threads shown above. See Detections::non_max_suppression_parallel's doc comment in src/core/detections.rs for the full writeup.

Contributing

Issues and PRs welcome. Run cargo fmt, cargo clippy --all-targets --features annotate-image,datasets,parallel, and cargo test --features annotate-image,datasets,parallel before submitting — CI runs the same checks, plus an annotate-opencv job that's allowed to fail (it needs a system OpenCV install and isn't well-covered yet). If you touch that backend, check its CI result and mention in your PR how you tested it.

License

MIT — see LICENSE.