Crate edgefirst_image

Expand description

§EdgeFirst HAL - Image Converter

The edgefirst_image crate is part of the EdgeFirst Hardware Abstraction Layer (HAL) and provides functionality for converting images between different formats and sizes. The crate is designed to work with hardware acceleration when available, but also provides a CPU-based fallback for environments where hardware acceleration is not present or not suitable.

What the crate covers:

Packed and planar RGB families (Rgb, Rgba, Bgra, Grey, PlanarRgb, PlanarRgba), semi-planar YUV sources (Nv12, Nv16, Nv24), and packed YUV (Yuyv, Vyuy).
Source crop, resize, letterbox, 90° rotation, and flipping in one call.
Segmentation-mask rendering, including a fused GPU proto path.
SAHI-style input tiling for small-object detection in large frames (TilingConfig, ImageProcessor::tile_into).
Hardware acceleration (OpenGL, NXP G2D) with a CPU fallback that is always available.

The crate uses TensorDyn from edgefirst_tensor to represent images, with PixelFormat metadata describing the pixel layout. The ImageProcessor struct manages the conversion process, selecting the appropriate conversion method based on the available hardware.

Geometry is source-side: Crop says which sub-rectangle of the source to sample and how to fit it into the destination’s shape. Where the output lands is the destination itself — pass a view or batch of a larger tensor to render into one tile of a batch.

§Examples

let image = edgefirst_bench::testdata::read("zidane.jpg");
// The codec emits the source's native format (a colour JPEG decodes to NV12)
// and configures the destination tensor's dims+format during the decode.
let info = peek_info(&image).expect("peek");
// Heap tensors are CPU-touched on both sides (codec decode writes, the
// engine's upload/CPU-fallback paths read), so declare ReadWrite. Pure
// hardware pipelines allocate their convert destinations with
// `CpuAccess::None` instead.
let mut src = Tensor::<u8>::image(info.width, info.height, info.format,
                                   Some(TensorMemory::Mem), edgefirst_tensor::CpuAccess::ReadWrite)?;
let mut decoder = ImageDecoder::new();
src.load_image(&mut decoder, &image).expect("decode");
// Convert the native NV12 frame to packed RGB for downstream processing.
let mut converter = ImageProcessor::new()?;
let mut dst =
    converter.create_image(640, 480, PixelFormat::Rgb, DType::U8, None, CpuAccess::ReadWrite)?;
converter.convert(&src.into(), &mut dst, None)?;

§Environment Variables

The behavior of the edgefirst_image::ImageProcessor struct can be influenced by the following environment variables:

EDGEFIRST_FORCE_BACKEND: When set to cpu, g2d, or opengl (case-insensitive), only that single backend is initialized and no fallback chain is used. If the forced backend fails to initialize, an error is returned immediately. This is useful for benchmarking individual backends in isolation. When this variable is set, the EDGEFIRST_DISABLE_* variables are ignored.
EDGEFIRST_DISABLE_GL: If set to 1, disables the use of OpenGL for image conversion, forcing the use of CPU or other available hardware methods.
EDGEFIRST_DISABLE_G2D: If set to 1, disables the use of G2D for image conversion, forcing the use of CPU or other available hardware methods.
EDGEFIRST_DISABLE_CPU: If set to 1, disables the use of CPU for image conversion, forcing the use of hardware acceleration methods. If no hardware acceleration methods are available, an error will be returned when attempting to create an ImageProcessor.

Additionally the TensorMemory used by default allocations can be controlled using the EDGEFIRST_TENSOR_FORCE_MEM environment variable. If set to 1, default tensor memory uses system memory. This will disable the use of specialized memory regions for tensors and hardware acceleration. However, this will increase the performance of the CPU converter.

Re-exports§

pub use edgefirst_codec as codec;

Modules§

fit_code: Wire codes for FitMode, one const per variant and identical to .code() for the same variant – both are generated from the same declared literal here, so a const-only consumer (cbindgen, FFI) and code() cannot independently drift apart.
interpolation_code: Wire codes for Interpolation, one const per variant and identical to .code() for the same variant – both are generated from the same declared literal here, so a const-only consumer (cbindgen, FFI) and code() cannot independently drift apart.
letterbox_placement_code: Wire codes for LetterboxPlacement, one const per variant and identical to .code() for the same variant – both are generated from the same declared literal here, so a const-only consumer (cbindgen, FFI) and code() cannot independently drift apart.
normalization_code: Wire codes for NormalizationPreset, one const per variant and identical to .code() for the same variant – both are generated from the same declared literal here, so a const-only consumer (cbindgen, FFI) and code() cannot independently drift apart.

Structs§

CPUProcessor: CPUConverter implements the ImageProcessor trait using the fallback CPU implementation for image processing.
CacheStats: Snapshot of one EGLImage cache’s hit/miss counters.
ContentRegion: Where the image content sits in a destination: in pixels, and as [xmin, ymin, xmax, ymax] normalized by the destination size, the form unletter_norm and the semantic decoder take.
ConvertOptions: Options for convert. The default converts the whole source, stretched, with no rotation or flip, resamples with the processor’s interpolation, and maps pixels as Normalization::unsigned for float destinations.
ConvertStats: Per-processor zero-copy telemetry: how convert sources reached the GPU and how often a zero-copy opportunity was declined into a copy path.
Crop: Source-side convert geometry: which sub-rectangle of the source to sample (source) and how to fit it into the destination (fit). Destination placement is the destination itself — a tensor, or a Region view / batch tile of one — not a field here.
EglDisplayInfo: A validated, available EGL display discovered by probe_egl_displays.
G2DProcessor: G2DConverter implements the ImageProcessor trait using the NXP G2D library for hardware-accelerated image processing on i.MX platforms.
GLProcessorThreaded: OpenGL multi-threaded image converter. The actual conversion is done in a separate rendering thread, as OpenGL contexts are not thread-safe. This can be safely sent between threads. The convert() call sends the conversion request to the rendering thread and waits for the result.
GlCacheStats: Combined snapshot of every EGLImage cache on the GL processor (source, destination, the Path-B NV R8 source cache, and its chroma-plane view for hardware-filtered NV resizes).
ImageProcessor: Image converter that uses available hardware acceleration or CPU as a fallback.
ImageProcessorConfig: Configuration for ImageProcessor construction.
MaskOverlay: Options for mask overlay rendering.
Normalization: Per-channel (x - mean) / std on 0–255 pixel values, in RGB order.
PreprocessSpec: A model’s input preprocessing as plain values: what the image crate needs to build a convert, produced by the decoder from model metadata.
Region: A rectangular sub-region of a tensor’s leading spatial frame, in pixels.
RenderDtypeSupport: Reports which float-color-buffer extensions the GPU backend detected. Returned by ImageProcessor::supported_render_dtypes; the two flags are independent.
TilePlacement: One tile’s placement within a frame grid.
TileSpec: One tile’s native-frame crop rectangle and its grid coordinates.
TilingConfig: Static tiling configuration for one model. Independent of frame size.

Enums§

ColorMode: Controls how the color palette index is chosen for each detected object.
ColorimetryMode: How convert() trades colorimetric exactness against speed on platforms where the exact path is expensive.
ComputeBackend: Compute backend selection for ImageProcessor.
EglDisplayKind: Identifies the type of EGL display used for headless OpenGL ES rendering.
Error
Fit: How a source is fit into the requested destination shape.
FitMode: How a source is fit into the destination, without the letterbox pad colour. The code is the one the C EF_FIT_* enumerators and the Python Fit enum use.
Flip: A mirror applied during ImageProcessorTrait::convert.
Int8InterpolationMode: Interpolation mode for int8 proto textures (GL_R8I cannot use GL_LINEAR).
Interpolation: Resampling filter. The code is the one the C EF_INTERPOLATION_* enumerators use; the string is the edgefirst.json and Python name.
LetterboxPlacement: Where a letterboxed image sits inside the destination. The code is the one the C EF_LETTERBOX_PLACEMENT_* enumerators and the Python LetterboxPlacement enum use; the string is the edgefirst.json name.
MaskResolution: Controls the resolution and coordinate frame of masks produced by ImageProcessor::materialize_masks.
NormalizationPreset: A named Normalization. The code is the one the C EF_NORMALIZATION_* enumerators and the Python NormalizationPreset enum use; the names match the edgefirst.json presets.
PadSpace: The space a letterbox pad value is expressed in. The code is the one the C EF_PAD_SPACE_* enumerators use; the string is the edgefirst.json name.
Rotation: A quarter-turn rotation applied during ImageProcessorTrait::convert.

Constants§

DEFAULT_BOX_LINE_WIDTH: Default bounding-box outline thickness in pixels, used until ImageProcessorTrait::set_box_line_width sets another.
GPU_DMA_BUF_PITCH_ALIGNMENT_BYTES: Pitch alignment requirement for DMA-BUF tensors that may be imported as EGLImages by the GL backend. Mali Valhall (i.MX 95 / G310) rejects eglCreateImageKHR with EGL_BAD_ALLOC for any DMA-BUF whose row pitch is not a multiple of 64 bytes; Vivante GC7000UL (i.MX 8MP) accepts any pitch so the constant is harmless on that path. 64 is the smallest alignment that satisfies every embedded ARM GPU we ship to.

Traits§

ContentRegionSource: Something that knows where it puts the image content in a destination, such as the image crate’s ConvertOptions or a PreprocessSpec. The decoder takes it to map its output back to the source image with the same geometry the convert used.
ImageProcessorTrait

Functions§

align_width_for_gpu_pitch: Round width (in pixels) up so the resulting row stride width * bpp is a multiple of GPU_DMA_BUF_PITCH_ALIGNMENT_BYTES AND a multiple of bpp (so the rounded width is an integer pixel count).
primary_plane_bpp: Bytes-per-pixel for the primary plane of format at element size elem. Returns None for formats that don’t have a single packed BPP (semi-planar chroma is handled separately, returning the luma-plane bpp).
probe_egl_displays: Probe for available EGL displays supporting headless OpenGL ES 3.0.
save_jpeg: Save a TensorDyn image as a JPEG file.
tile_grid: Uniform overlapping EvenDist tile grid covering a frame_h×frame_w frame. Row-major (all columns of row 0, then row 1, …). Every tile is full-size (tile_w×tile_h) unless the frame is smaller than the tile on an axis, in which case that axis yields a single whole-frame crop.

Type Aliases§

CompletionFence: Handle returned by convert_with_fence: a sync_file fd on Unix, an event handle on Windows that is set when the GPU work completes.
Result