Crate ados_vision 

ados-vision

Single-object visual tracker for drones: Kalman motion model with appearance re-identification, YOLOv8 head decoding and OSNet embedding helpers.

Object detectors are noisy: boxes jitter, frames drop, and a second object of the same class can wander into view. When a drone follows or frames one target, it needs one stable identity, not a fresh list of boxes every frame. This crate turns per-frame detections into a single locked track with a persistent id, coasts through short misses, and refuses to silently swap onto a lookalike. It also decodes raw YOLO output tensors and prepares re-identification crops, so a detector and an appearance model can feed it directly.

use ados_protocol::framebus::{BoundingBox, Detection, LockState};
use ados_vision::tracker::{SingleObjectTracker, TrackState, TrackerConfig};

fn person(x: f32) -> Detection {
    Detection {
        bbox: Some(BoundingBox { x, y: 100.0, width: 40.0, height: 80.0 }),
        class_label: "person".into(),
        confidence: 0.9,
        track_id: None,
        assoc_confidence: None,
        lock_state: None,
        attributes: None,
        mask: None,
        keypoints: None,
        depth: None,
        world_pos: None,
    }
}

let mut tracker = SingleObjectTracker::new(TrackerConfig::default());
let mut last = None;
// The target drifts right by 3 px per frame.
for frame in 0..5 {
    last = Some(tracker.update(&[person(200.0 + 3.0 * frame as f32)]));
}
let update = last.unwrap();
assert_eq!(update.state, TrackState::Confirmed);
let reported = update.detection.expect("a confirmed track is reported");
assert_eq!(reported.track_id, update.track_id);
println!("track {:?}, lock {:?}", update.track_id, reported.lock_state);

// A missed frame coasts on the prediction and keeps the id.
let coast = tracker.update(&[]);
assert_eq!(coast.state, TrackState::Coasting);
assert_eq!(coast.track_id, update.track_id);
assert_ne!(coast.detection.and_then(|d| d.lock_state), Some(LockState::Locked));

Highlights

• Pure and deterministic: no camera, socket or clock. The same input sequence always yields the same id assignment.
• Never-silent identity: when two candidates look alike, the lock goes Uncertain and raises a re-designation request instead of guessing.
• Appearance-first association with a pluggable AppearanceModel; a colour/gradient histogram ships, a learned embedding plugs in through the same trait.
• YOLOv8/v11 and YOLOv5/v7 head decoding with class-agnostic non-maximum suppression.
• 63 tests, including adversarial suites that try to provoke identity swaps, locks onto noise and distractor capture.
• One dependency: ados-protocol for the shared Detection type.

How it works

The crate has three modules.

tracker

SingleObjectTracker takes the per-frame detections (the shared Detection type, never a parallel one), predicts where the locked object should be next with a constant-velocity Kalman model over [cx, cy, w, h] and their rates, associates the single best detection, and stamps a persistent track_id, lock_state and assoc_confidence on it. A brief miss is coasted on the prediction and keeps the id; only a real loss drops the lock, and a re-acquire then mints a fresh id.

The lifecycle is Idle, Tentative (too few hits to report), Confirmed, and Coasting (held on the prediction through a miss or an ambiguous frame, reported Uncertain). A track can be seeded automatically or by an explicit operator pick with designate; TrackUpdate::operator_designated tells the two apart, so a consumer can present only operator-picked tracks as locked.

Geometry alone cannot separate two overlapping boxes: a distractor riding the prediction fits the motion gate as well as the true target that maneuvered off it. So update_with_appearance takes Candidates that carry an appearance descriptor beside the detection. The tracker captures a template at confirm and updates it slowly, so it cannot drift onto a distractor. The descriptor travels only through Candidate, never on the wire detection.

Association is appearance-first. Among candidates that clear the absolute appearance gate, the one that looks most like the template wins; motion fit and detector confidence only break ties within an ambiguity margin of the best. When two admissible candidates fall inside that margin, the tracker holds on the prediction, reports Uncertain, lowers its confidence and sets needs_redesignation. The one case vision cannot solve, a lone appearance-identical impostor while the true target is out of frame, is left to that operator backstop rather than a guess.

All thresholds live in TrackerConfig.

yolo

decode turns a raw output tensor into detections in source-frame pixels. It supports the transposed [1, 4+nc, anchors] YOLOv8/v11 head (no objectness) and the [1, anchors, 5+nc] YOLOv5/v7 head, transposes a head delivered the other way round automatically, scales boxes by frame / input, applies the confidence gate and runs class-agnostic NMS. preprocess_rgb24_nchw does the matching stretch resize into an NCHW float tensor.

reid

Helpers for a re-identification model such as OSNet: crop and resize a detection box out of an rgb24 frame, normalize the crop with ImageNet statistics into an NCHW tensor, and L2-normalize the returned embedding so cosine similarity becomes a dot product.

Part of ADOS

Used in production by the ADOS Drone Agent, the onboard software of the ADOS software-defined drone platform. All ADOS crates · Docs · Discord

Other ADOS crates

Platform (versioned with the ADOS Drone Agent)

Crate	What it does	Docs
ados-protocol	Wire contracts of the ADOS Drone Agent: IPC framing, plugin RPC envelope, capability tokens and the vision frame bus.	docs.rs
ados-sdk	Write Rust plugins for the ADOS Drone Agent: IPC client, plugin context, driver traits and lifecycle runner.	docs.rs
ados-gpio	Drive status buzzers and LEDs on Linux GPIO lines, with bounded software-PWM beep patterns.	docs.rs
ados-hal-probe	Find out what a board’s hardware can really do by probing it, plus bundled board profiles.	docs.rs

Standalone

Crate	What it does	Docs
ados-sensorhub	Sensor hub wire protocol: COBS framing, CRC-16 and fixed-layout IMU, baro, mag and GNSS packets. no_std.	docs.rs
ados-offload	Choose where perception runs (onboard, offloaded or hybrid) and hold the vehicle when results go stale.	docs.rs
ados-macpin	Stable MAC addresses for Linux network adapters that randomize theirs every boot.	docs.rs
ados-lora	LoRa transmit governor with cited regional band plans, duty cycle, dwell time and airtime accounting.	docs.rs
ados-crsf-codec	no_std CRSF / ExpressLRS codec for RC channels, telemetry and parameter frames.	docs.rs
ados-positioning	Split mixed UBX, NMEA and RTCM3 GNSS streams, configure u-blox receivers and run an RTK base survey-in.	docs.rs
ados-rtcm	RTCM3 framing, CRC-24Q and MAVLink GPS_RTCM_DATA fragmentation for RTK corrections.	docs.rs
ados-clock	Map remote clocks onto the host clock: least-squares and envelope fits, TIMESYNC and a chrony sink.	docs.rs
ados-swarm	Swarm control laws: separation, flocking, formations, CBBA task allocation and a point-mass simulator.	docs.rs
ados-mavlink-tunnel	Send large messages over MAVLink TUNNEL frames with chunking and bounded reassembly.	docs.rs
ados-display-driver	Small-display output on Linux: SSD1306/SH1106 OLED over I2C, pixel packers and a framebuffer writer.	docs.rs
ados-hmi	Front-panel input: touchscreen calibration and gestures, button presses and rotary encoders.	docs.rs

ADOS repositories

Repo	Runs on	What it does	Link
ADOS Mission Control	Browser, desktop (Electron)	Ground control station: FC setup, mission planning, flight, 3D simulation, fleet	ADOSMissionControl
ADOS Drone Agent	Companion computer on the drone, or a ground node	Onboard agent: MAVLink routing, HD video, long-range radio link, plugins, remote control	ADOSDroneAgent
ADOS Android GCS	Android phones and tablets	Native ground control app for agents on the LAN, direct flight controllers and the cloud	ADOSAndroidGCS
ADOS Extensions	Drone Agent and Mission Control	First-party signed plugins, each with an agent half and a GCS half	ADOSExtensions
ADOS MCP	Your machine	Model Context Protocol server that lets AI clients operate ADOS drones under scopes and a safety gate	ADOS-MCP
Documentation	docs.altnautica.com	Public docs for every ADOS product	Documentation

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Modules

reid

Re-id (appearance) embedding helpers: crop a detection’s box out of a frame, resize it to the embedding model’s input, and (for the in-process ONNX path) normalize it into the model’s expected tensor. The embedding the model returns is L2-normalized so the tracker’s cosine similarity is a clean dot product, the standard re-id distance.

tracker

A single-object visual tracker with an appearance model.

yolo

YOLO detection-head decoding.