Skip to main content

Crate rusttorch_vision

Crate rusttorch_vision 

Source
Expand description

Decode local images and keep annotations aligned through geometric transforms.

Images are RGB Uint8 CHW tensors until VisionSample::to_float. Datasets read local files only; no download or archive extraction happens implicitly.

use rusttorch_data::{DataLoader, ResourceLimits};
use rusttorch_vision::ImageFolder;
let dataset = ImageFolder::open("images/train", ResourceLimits::default())?;
let mut loader = DataLoader::builder(dataset).batch_size(8).workers(2).build()?;
for batch in loader.iter() {
    let batch = batch?;
    // Same-sized inputs stack to [batch, 3, height, width].
    println!("{:?}", batch.images.size());
}

§RustTorch Vision

Turn local image folders, digit datasets and detection annotations into typed training batches. Image decoding produces RGB Uint8 tensors in CHW order; to_float() converts bytes to Float values in [0, 1]. Decoding does not apply EXIF orientation, color-profile conversion or automatic normalization.

§Installation

Install through rusttorch = { version = "0.4", features = ["vision"] }, which exposes rusttorch::vision, or use the package directly:

[dependencies]
rusttorch-vision = { version = "0.4", features = ["download-libtorch"] }
rusttorch-data = "0.4"

The package uses the same LibTorch runtime as the rest of RustTorch. PNG/JPEG codecs are included only when this optional package is selected.

§Features

The vision facade feature enables PNG/JPEG decoding, local dataset readers and target-aware image transforms. The direct package has no default features. download-libtorch obtains the tensor runtime; doc-only builds documentation without native linking and cannot execute tensor operations.

§Native runtime

Tensor operations require LibTorch 2.13.0. Enable download-libtorch, or set LIBTORCH to an extracted distribution and add its library directory to PATH on Windows or the platform’s shared-library search path. All RustTorch packages in one application must share the same runtime. Documentation can be checked with cargo doc -p rusttorch-vision --no-default-features --features doc-only.

§Example: Train from class folders

Arrange inputs as train/cats/a.jpg, train/dogs/b.png, and so on. Class names and file paths are sorted, so label assignments are stable. Nested directories inside a class are supported. Empty classes, symlinks and unsupported files produce errors. Keep the dataset files unchanged during an epoch.

use std::num::NonZeroUsize;
use rusttorch_data::{DataLoader, FnTransform, ResourceLimits, TaskContext};
use rusttorch_vision::{ImageFolder, VisionSample};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let dataset = ImageFolder::open("train", limits)?;
    println!("label names: {:?}", dataset.classes());
    let mut loader = DataLoader::builder(dataset)
        .shuffle(42)?
        .batch_size(16)
        .workers(2)
        .prefetch_bytes(NonZeroUsize::new(32 * 1024 * 1024).unwrap())
        .transform(FnTransform::new(move |image: VisionSample, _: &TaskContext| {
            image.resize(224, 224, limits)?.to_float()
        }))
        .build()?;

    for batch in loader.iter() {
        let batch = batch?;
        // Pass batch.images to an image model. The final batch may be smaller.
        assert_eq!(&batch.images.size()[1..], &[3, 224, 224]);
        assert_eq!(batch.labels.len(), batch.images.size()[0] as usize);
    }
    Ok(())
}

The default collator returns VisionBatch: a stacked image tensor plus ordered per-image labels, boxes, keypoints and optional masks. Resize differing input sizes before stacking. For variable-size detection images, explicitly select rusttorch_data::VecCollate and process each owned sample separately.

§Keep detection targets aligned

Rectangles use continuous XYXY pixel-edge coordinates. Cropping clips boxes and removes empty boxes; keypoints outside the crop become absent. Resizing uses bilinear image interpolation and nearest-neighbor mask indexing, preserving integer category IDs exactly. Transformations consume the sample and keep all its targets together.

use rusttorch_data::{Dataset, ResourceLimits};
use rusttorch_vision::CocoDetection;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let dataset = CocoDetection::open("images", "annotations.json", limits)?;
    let sample = dataset.get(0)?.resize(256, 256, limits)?;
    let sample = sample.crop(16, 16, 224, 224)?.to_float()?;
    println!("{} objects", sample.boxes.len());
    Ok(())
}

horizontal_flip() updates coordinates, boxes and masks. It preserves keypoint order; a pose model with named left/right joints must additionally permute those joint entries according to its own skeleton. COCO category IDs are preserved, not automatically made contiguous. The COCO reader supports bounding boxes and keypoints; polygon/RLE segmentation returns an explicit unsupported-data error. Attach an integer [H,W] mask to VisionSample.mask for mask transforms.

§Read local MNIST and CIFAR records

use rusttorch_data::{Dataset, ResourceLimits};
use rusttorch_vision::{Cifar, Mnist};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let limits = ResourceLimits::default();
    let digits = Mnist::open("train-images-idx3-ubyte", "train-labels-idx1-ubyte", limits)?;
    let digit = digits.get(0)?.to_float()?;
    assert_eq!(digit.image.size()[0], 3); // Grayscale repeated into RGB channels.

    let images = Cifar::open("data_batch_1.bin", false, limits)?;
    let first = images.get(0)?;
    assert_eq!(first.image.size(), [3, 32, 32]);
    // Pass true to read CIFAR-100 binary records and return the fine label.
    Ok(())
}

IDX inputs must be uncompressed and have matching header counts and exact file sizes. CIFAR inputs must contain complete binary records. Both read one sample at a time, validate label ranges and work with the same loader as image folders. No dataset download or archive extraction happens implicitly.

§Supported formats and limits

Input or operationContractExecutable evidence
PNG/JPEGRGB byte decoding, explicit float conversiontests/pipelines.rs
ImageFolderStable classes/paths, lazy worker decodingimagefolder_workers_decode_classify_and_bound_payload
MNIST/CIFAR-10/CIFAR-100Local IDX/binary samples and labelsnamed_dataset_families_read_real_local_formats
COCO detectionLocal boxes/keypoints and safe-root pathsnamed_dataset_families_read_real_local_formats
Resize/crop/flipImage, boxes, keypoints and integer maskstarget_geometry_tracks_image_mask_and_keypoints

ResourceLimits checks encoded bytes, image dimensions, tensor elements, decoded payload and record counts. Decoder scratch allocations belong to the image library; these limits are not a process-RSS guarantee. Loader prefetch_bytes bounds retained sample payload independently. Existing direct rusttorch::data paths are unchanged; this package adds optional APIs.

Structs§

BoundingBox
An axis-aligned rectangle in continuous pixel-edge XYXY coordinates.
Cifar
One local CIFAR-10 or CIFAR-100 binary file; RGB planar bytes are preserved.
CocoDetection
COCO-style bounding-box/keypoint dataset. Polygon/RLE segmentation is rejected explicitly; integer masks can be attached to VisionSample by callers.
ImageFolder
Images under root/class-name/image.png, with sorted classes and paths. Symlinks are rejected; traversal is bounded by the record limit.
Keypoint
A keypoint in pixel-edge coordinates, with COCO visibility (0, 1 or 2).
Mnist
Local uncompressed MNIST IDX image/label pair, read one sample at a time.
VisionBatch
Dense image batch with variable-size annotations retained per sample.
VisionSample
Owned CHW RGB bytes and associated classification/detection targets.

Enums§

VisionError
Image decoding, annotation, I/O or tensor failure.

Functions§

decode_image
Decodes a bounded PNG/JPEG into RGB Uint8 CHW, ignoring EXIF orientation.

Type Aliases§

Result
Result returned by vision operations.