Crate edgefirst_decoder 

EdgeFirst HAL - Decoders

This crate provides decoding utilities for YOLOobject detection and segmentation models, and ModelPack detection and segmentation models. It supports both floating-point and quantized model outputs, allowing for efficient processing on edge devices. The crate includes functions for efficient post-processing model outputs into usable detection boxes and segmentation masks, as well as utilities for dequantizing model outputs..

For general usage, use the Decoder struct which provides functions for decoding various model outputs based on the model configuration. If you already know the model type and output formats, you can use the lower-level functions directly from the yolo and modelpack modules.

Quick Example

// Create a decoder for a YOLOv8 model with quantized int8 output with 0.25 score threshold and 0.7 IOU threshold
let decoder = DecoderBuilder::new()
    .with_config_yolo_det(configs::Detection {
        anchors: None,
        decoder: configs::DecoderType::Ultralytics,
        quantization: Some(configs::QuantTuple(0.012345, 26)),
        shape: vec![1, 84, 8400],
        dshape: Vec::new(),
        normalized: Some(true),
    },
    Some(DecoderVersion::Yolov8))
    .with_score_threshold(0.25)
    .with_iou_threshold(0.7)
    .build()?;

// Get the model output from the model. Here we load it from a test data file for demonstration purposes.
let model_output: Vec<i8> = include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/../../testdata/yolov8s_80_classes.bin"))
    .iter()
    .map(|b| *b as i8)
    .collect();
let model_output_array = ndarray::Array3::from_shape_vec((1, 84, 8400), model_output)?;

// THe capacity is used to determine the maximum number of detections to decode.
let mut output_boxes: Vec<_> = Vec::with_capacity(10);
let mut output_masks: Vec<_> = Vec::with_capacity(10);

// Decode the quantized model output into detection boxes and segmentation masks
// Because this model is a detection-only model, the `output_masks` vector will remain empty.
decoder.decode_quantized(&[model_output_array.view().into()], &mut output_boxes, &mut output_masks)?;

Overview

The primary components of this crate are:

• Decoder/DecoderBuilder struct: Provides high-level functions to decode model outputs based on the model configuration.
• yolo module: Contains functions specific to decoding YOLO model outputs.
• modelpack module: Contains functions specific to decoding ModelPack model outputs.

The Decoder supports both floating-point and quantized model outputs, allowing for efficient processing on edge devices. It also supports mixed integer types for quantized outputs, such as when one output tensor is int8 and another is uint8. When decoding quantized outputs, the appropriate quantization parameters must be provided for each output tensor. If the integer types used in the model output is not supported by the decoder, the user can manually dequantize the model outputs using the dequantize functions provided in this crate, and then use the floating-point decoding functions. However, it is recommended to not dequantize the model outputs manually before passing them to the decoder, as the quantized decoder functions are optimized for performance.

The yolo and modelpack modules provide lower-level functions for decoding model outputs directly, which can be used if the model type and output formats are known in advance.

Re-exports

pub use error::DecoderError;

pub use error::DecoderResult;

Modules

byte

config

configs

error

float

modelpack

yolo

Structs

BoundingBox

A bounding box with f32 coordinates in XYXY format

ConfigOutputs

Used to represent the outputs in the model configuration.

Decoder

DecoderBuilder

DetectBox

A detection box with f32 bbox and score

DetectBoxQuantized

A detection box with a f32 bbox and quantized score

ProtoData

Raw prototype data for fused decode+render pipelines.

Quantization

Describes the quantization parameters for a tensor

Segmentation

A segmentation result with a segmentation mask, and a normalized bounding box representing the area that the segmentation mask covers

XYWH

Converts XYWH bounding boxes to XYXY. The XY values are the center of the box

XYXY

Converts XYXY bounding boxes to XYXY

Enums

ArrayViewDQuantized

ConfigOutput

ConfigOutputRef

DecoderVersion

Decoder version for Ultralytics models.

Nms

NMS (Non-Maximum Suppression) mode for filtering overlapping detections.

ProtoTensor

Prototype tensor variants for fused decode+render pipelines.

Traits

BBoxTypeTrait

Trait to convert bounding box formats to XYXY float format

Functions

dequant_detect_box

Turns a DetectBoxQuantized into a DetectBox by dequantizing the score.

dequantize_cpu

Dequantizes a slice from quantized values to float values using the given quantization parameters

dequantize_cpu_chunked

Dequantizes a slice from quantized values to float values using the given quantization parameters, using chunked processing. This is around 5% faster than dequantize_cpu for large slices.

dequantize_ndarray

Dequantizes an ndarray from quantized values to f32 values using the given quantization parameters

segmentation_to_mask

Converts a segmentation tensor into a 2D mask If the last dimension of the segmentation tensor is 1, values equal or above 128 are considered objects. Otherwise the object is the argmax index