edgefirst-tensor

Zero-copy tensor memory management for edge AI applications.

This crate provides a unified interface for managing multi-dimensional arrays (tensors) with support for different memory backends optimized for ML inference pipelines.

Role in edgefirst-hal

edgefirst-tensor is the foundation of the data-plane crates in the EdgeFirst HAL workspace. The image, decoder, capi, and gpu-probe crates all depend on it; the tracker and bench crates are independent and operate on their own types.

• edgefirst-image consumes Tensor<u8> / TensorDyn for image processor input/output buffers and provides the PboOps trait impl that backs PboTensor.
• edgefirst-decoder reads model output via Tensor<T> and TensorMap.
• edgefirst-tensor-capi crosses the FFI boundary using from_fd, clone_fd, and from_planes.
• gpu-probe uses it to allocate the DMA-BUF round-trip buffer the probe verifies.
• edgefirst-tracker and edgefirst-bench do not depend on this crate — tracker works against DetectionBox and nalgebra, bench wraps serde_json for benchmark IO.

This crate has no internal edgefirst-* dependencies.

Memory Types

TensorMemory::DmaBuf is one variant on every platform. Ask for it and you get whichever of the three backings the OS provides, so portable code does not branch on the mechanism.

So is every other variant: TensorMemory names storage codes rather than listing what this build can allocate, so a descriptor recorded on one platform is still nameable on another. TensorMemory::is_available() answers the "can I allocate this here, now" question at runtime — /dev/dma_heap can exist while the process lacks permission on it — and a request for a backing this build cannot serve returns an error rather than panicking. IoSurface and Cuda are defined as codes but no backend reports them yet.

On Linux a DMA-BUF comes from the CMA heap, which is physically contiguous, or from the system heap, which is not, when there is no CMA heap or it is full. Tensor::contiguity() reports which (Unknown for an imported fd). An image that a device without an IOMMU will write into, such as a camera capture buffer, can require CMA with ImageDesc::with_contiguous(true); the allocation then fails instead of falling back.

Features

• Automatic memory selection — new() tries DMA → SHM → Mem; the image constructors try DMA → Mem (SHM buys an image nothing)
• Zero-copy sharing — share tensors between processes via file descriptors
• Memory mapping — CPU access with the direction declared up front
• ndarray integration — optional conversion to/from ndarray::Array (feature: ndarray)

Quick Start

use edgefirst_tensor::{CpuAccess, PixelFormat, Tensor, TensorMapTrait, TensorMemory, TensorTrait};

# fn main() -> Result<(), edgefirst_tensor::Error> {
// Plain tensor, automatic memory selection (DMA → SHM → Mem).
let tensor = Tensor::<f32>::new(&[1, 3, 224, 224], None, None)?;
println!("Memory type: {:?}", tensor.memory());

// Map for CPU access. `map()` is ReadWrite; map_read() / map_write() are
// cheaper when you only need one direction.
let mut map = tensor.map()?;
map.as_mut_slice().fill(0.0);

// Image tensors take a required CpuAccess declaration — see below.
let frame = Tensor::<u8>::image(1920, 1080, PixelFormat::Rgba,
                                Some(TensorMemory::DmaBuf), CpuAccess::None)?;

// Share via file descriptor (Linux DMA-BUF / POSIX shm).
#[cfg(target_os = "linux")]
let fd = frame.clone_fd()?;
# Ok(())
# }

CPU Access Is Declared, Hardware Access Is Assumed

Every image constructor — Tensor::image, TensorDyn::image, ImageProcessor::create_image, and the C and Python equivalents — takes a CpuAccess argument (in C and Python it defaults to ReadWrite). There is no usage or role enum beyond it.

The crate assumes image buffers are produced and consumed by hardware (ISP, codec, GPU, NPU), so every image tensor allocates GPU-readable and GPU-writable and no declaration is needed for that. What the CPU intends to do is the one thing the allocator cannot guess:

The declaration covers all CPU access, including the HAL's own CPU work. A map, pin or sync beyond the declaration fails with Error::CpuAccessDenied, naming the declared and requested access, on both the static and dynamic backends. map() asks for ReadWrite, so on a Read- or Write-declared tensor use map_read() to read and TensorTrait::map_for_write() to fill. A write-only map is valid only for a pass that overwrites the whole buffer; map_for_write() maps a view ReadWrite for that reason.

On Android any access other than None pins the layout linear and forfeits tile compression: the ReadWrite default asks gralloc for CPU_READ_OFTEN | CPU_WRITE_OFTEN, which may give linear, CPU-cached memory. Pass CpuAccess::None for a buffer you never map, so gralloc can keep a tiled or compressed allocation. Linux DMA-BUF allocation does not depend on the declaration.

restrict_cpu_access narrows a declaration and never widens it; import_descriptor uses it to carry a producer's declaration onto an imported alias.

For anything past the four common parameters — notably a compression request — use the ImageDesc builder and Tensor::image_desc.

DMA Images Carry a 64-Byte-Aligned Row Stride

Every DMA-backed image tensor has a 64-byte-aligned row stride, in packed, planar, and semi-planar layouts alike. This is not a tuning choice: Mali rejects an eglCreateImage DMA-BUF import at an unaligned pitch with EGL_BAD_ALLOC, and Vivante with EGL_BAD_ACCESS. Neither error mentions the stride, which is what makes the bug expensive to find.

The alignment holds in both image() (allocation) and configure_image() (reconfiguring a pooled tensor for a new geometry). If you touch either path, keep it. Widths that are merely not a multiple of 16 are enough to trip it — 321 pixels of RGBA is 1284 bytes, 322 is 1288, and neither divides by 64 — and the resulting convert succeeds on V3D and Tegra while failing on imx95 and imx8mp.

Platform Support

On macOS and Android the GL backend renders into the platform GPU buffer directly, so it has no need of PBOs. On Windows the DMA column is a TensorMemory::DmaBuf tensor backed by an ID3D11Texture2D on the HAL's D3D11 device (src/d3d11/); PBO tensors still allocate there and are the fallback when a format has no texture layout. Probe with is_gpu_buffer_available() rather than the per-platform probes when all you need to know is whether TensorMemory::DmaBuf will work.

Feature Flags

• ndarray (default) - Enable ndarray integration for array conversions

Environment Variables

• EDGEFIRST_TENSOR_FORCE_MEM - Set to 1 or true to force heap allocation

PlaneDescriptor

PlaneDescriptor wraps a duplicated file descriptor for use with ImageProcessor::import_image(). It captures optional stride and offset metadata alongside the fd so that the importer gets a complete picture of the plane layout without additional out-of-band parameters.

use edgefirst_tensor::PlaneDescriptor;
use std::os::fd::BorrowedFd;

// SAFETY: replace 42 with a real, valid fd from a DMA-BUF allocation.
let pd = unsafe { PlaneDescriptor::new(BorrowedFd::borrow_raw(42)) }
    .expect("failed to duplicate fd — check that the fd is valid")
    .with_stride(2048)  // optional: row stride in bytes
    .with_offset(0);    // optional: plane offset in bytes

The fd is duplicated eagerly in new() — a bad fd fails immediately rather than inside import_image. The caller retains ownership of the original fd.

DMA-BUF fd Accessors

TensorDyn exposes two fd accessors for DMA-backed tensors (Linux only):

• dmabuf(&self) -> Result<BorrowedFd<'_>> — Borrow the DMA-BUF fd tied to the tensor's lifetime.
• dmabuf_clone(&self) -> Result<OwnedFd> — Duplicate the DMA-BUF fd. Fails with Error::NotImplemented if the tensor is not DMA-backed.

// Share the buffer with an external consumer (e.g. NPU delegate)
let fd = tensor.dmabuf_clone()?;
delegate.register_buffer(fd)?;

Pixel Format Metadata

Attach a PixelFormat to any tensor for image processing:

• set_format(format: PixelFormat) -> Result<()> — Validates shape compatibility and stores the format.
• with_format(format: PixelFormat) -> Result<Self> — Builder-style consuming variant.

let mut t = TensorDyn::new(&[480, 640, 3], DType::U8, None, None)?;
t.set_format(PixelFormat::Rgb)?;

Row Stride

For externally allocated buffers with row padding (e.g. V4L2 camera frames):

• row_stride(&self) -> Option<usize> — Stored stride, None if tightly packed.
• effective_row_stride(&self) -> Option<usize> — Stored stride, or computed from format and width if not set.
• set_row_stride(stride: usize) -> Result<()> — Set stride in bytes. Format must be set first.
• with_row_stride(stride: usize) -> Result<Self> — Builder-style consuming variant.

Plane Offset

For buffers where image data does not start at byte 0 of the fd:

• plane_offset(&self) -> Option<usize> — Offset in bytes, None if zero.
• set_plane_offset(offset: usize) — Set byte offset.
• with_plane_offset(offset: usize) -> Self — Builder-style consuming variant.

BufferIdentity

BufferIdentity provides a stable cache key for a tensor's underlying buffer. It is created fresh on every allocation or import and carries:

• id() -> u64 — Monotonically increasing integer. Changes whenever the buffer changes. Suitable as a HashMap key or EGL image cache key.
• weak() -> Weak<()> — Goes dead when the owning tensor (and all clones) are dropped, allowing caches to detect stale entries without holding a strong reference.

buffer_identity() is accessible on typed tensors via TensorTrait:

use edgefirst_tensor::{Tensor, TensorTrait};

let t = Tensor::<u8>::new(&[1920, 1080, 3], None, None)?;
let key = t.buffer_identity().id();
let guard = t.buffer_identity().weak();
// Later: guard.upgrade().is_none() means the tensor was dropped.

BufferIdentity is used internally by the image processing backends as an EGL image cache key to avoid redundant GPU texture imports across frames.

CUDA Tensor Mapping

On CUDA-capable devices (e.g. Jetson Orin-series) the float PBO produced by ImageProcessor::convert() can be mapped directly to a CUDA device pointer. No link-time dependency on libcudart — the symbols are resolved at runtime via dlopen.

Availability probe

use edgefirst_tensor::is_cuda_available;

if is_cuda_available() {
    println!("CUDA runtime present; zero-copy path available");
}

Usage — try CUDA map, fall back to host

use edgefirst_tensor::{TensorTrait, TensorMapTrait};

// Per-frame: prefer zero-copy CUDA, fall back to host map
if let Some(cuda) = dst.cuda_map() {
    // cuda.device_ptr() — raw CUDA device pointer, valid until `cuda` drops.
    // cuda.len()        — byte length of the mapped region.
    trt_enqueue(cuda.device_ptr(), cuda.len());
    // Drop `cuda` here — releases the PBO before the next convert().
} else {
    let host = dst.map()?;
    trt_enqueue_host(host.as_slice());
}

cuda_map() returns None when:

• libcudart is not present at runtime.
• The tensor is not PBO- or DMA-BUF-backed.
• CUDA registration of the backing buffer failed (logged at warn).

The CudaMap guard must be dropped before the next ImageProcessor::convert() call that writes into the same tensor — the GL pipeline must not touch a PBO while CUDA has it mapped. See ARCHITECTURE.md § Zero-copy CUDA tensor mapping for the full aliasing rules, DMA-BUF import path, and drop-order contract.

C API

if (ef_is_cuda_available()) {
    void *map = ef_tensor_cuda_map(tensor);
    if (map) {
        size_t size   = 0;
        void *dev_ptr = ef_tensor_cuda_device_ptr(map, &size);
        trt_enqueue(dev_ptr, size);
        ef_tensor_cuda_unmap(map);  // must call before next convert()
    }
}

Python

from edgefirst.tensor import is_cuda_available

if is_cuda_available():
    cm = dst.cuda_map()          # returns CudaMap or None
    if cm is not None:
        with cm:                 # context manager — unmap on __exit__
            trt_context.execute(cm.device_ptr)

Documentation

• Architecture overview: ARCHITECTURE.md
• Testing guide: TESTING.md
• Full API reference: docs.rs/edgefirst-tensor
• Project README: ../../README.md

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.