Struct Tensor

Source

pub struct Tensor<T>where
    T: Num + Clone + Debug + Send + Sync,
{ /* private fields */ }

Expand description

Multi-backend tensor with optional image format metadata.

When format is Some, this tensor represents an image. Width, height, and channels are derived from shape + format. When format is None, this is a raw tensor (identical to the pre-refactoring behavior).

Implementations§

Source §

impl<T> Tensor<T>
where T: Num + Clone + Debug + Send + Sync,

Source

pub fn from_slice(values: &[T], shape: &[usize]) -> Result<Self>
where T: Copy,

Construct a tensor from a row-major element slice + shape. Allocates a new buffer (TensorMemory::Mem) and memcpys the contents; caller retains ownership of the input slice.

§Errors

Error::InvalidShape if values.len() != shape.iter().product().
Propagates any allocation error from Self::new.

Source

pub unsafe fn from_foreign( ptr: *mut T, shape: &[usize], owner: Option<ForeignOwner>, name: Option<&str>, ) -> Result<Self>

Wrap externally-owned memory as a tensor without copying. The tensor borrows [ptr, ptr + shape.product() * size_of::<T>()) as TensorMemory::Mem; owner, when Some, co-owns the source so it outlives the tensor (and all derived views/maps). See ForeignOwner.

The canonical use is CUDA zero-copy: allocate host-coherent memory (cudaHostAlloc), wrap the host pointer here, and bind the matching device pointer to the inference engine — reads and writes hit the same physical buffer with no host copy. The identical primitive backs the Python Tensor.from_numpy zero-copy borrow (owner = the NumPy object).

§Safety

ptr must be non-null, aligned to align_of::<T>(), and valid for shape.product() elements of T for as long as the returned tensor — and every view/map sharing its backing — is alive. Pass an owner that co-owns the source to uphold that contract.

§Errors

Error::InvalidSize if shape is empty.

Source

pub fn from_arrayview3(view: ArrayView3<'_, T>) -> Result<Self>
where T: Copy,

Construct a tensor from a 3-D ndarray view. Respects strides — one copy in all cases; contiguous views take a memcpy fast path.

Only available when the ndarray feature is enabled.

Source

pub fn new( shape: &[usize], memory: Option<TensorMemory>, name: Option<&str>, ) -> Result<Self>

Create a new tensor with the given shape, memory type, and optional name. If no name is given, a random name will be generated. If no memory type is given, the best available memory type will be chosen based on the platform and environment variables.

On Linux platforms, the order of preference is: Dma -> Shm -> Mem. On other Unix platforms (macOS), the order is: Shm -> Mem. On non-Unix platforms, only Mem is available.

§Environment Variables

EDGEFIRST_TENSOR_FORCE_MEM: If set to a non-zero and non-false value, forces the use of regular system memory allocation (TensorMemory::Mem) regardless of platform capabilities.

§Example

use edgefirst_tensor::{Error, Tensor, TensorMemory, TensorTrait};
let tensor = Tensor::<f32>::new(&[2, 3, 4], Some(TensorMemory::Mem), Some("test_tensor"))?;
assert_eq!(tensor.memory(), TensorMemory::Mem);
assert_eq!(tensor.name(), "test_tensor");

Source

pub fn image( width: usize, height: usize, format: PixelFormat, memory: Option<TensorMemory>, ) -> Result<Self>
where T: 'static,

Create an image tensor with the given format.

Source

pub fn image_with_stride( width: usize, height: usize, format: PixelFormat, row_stride_bytes: usize, memory: Option<TensorMemory>, ) -> Result<Self>

Create a DMA-backed image tensor with an explicit row stride that may exceed the natural width * channels * sizeof(T) pitch.

Used for image tensors that need GPU pitch alignment padding: the underlying DMA-BUF is sized to row_stride * height bytes, but the tensor’s logical shape stays at [height, width, channels]. width() / height() / shape() continue to report the user-requested values; the padding is visible only via row_stride() / effective_row_stride() and is automatically propagated to the GL backend’s EGLImage import so Mali Valhall accepts the buffer.

§Supported formats

Currently only packed pixel layouts (RGBA8, BGRA8, RGB888, Grey, etc.) are supported — the formats the GL backend uses as render destinations. Semi-planar formats (NV12, NV16) come from external allocators (camera capture, video decoders) and are imported via TensorDyn::from_fd + set_row_stride, which already supports padded strides.

§Supported memory

Currently only TensorMemory::Dma is supported. PBO and Mem storage don’t go through EGLImage import so they don’t need pitch alignment; if you pass any other memory type this returns NotImplemented. None (auto-select) is treated as Dma.

§Errors

InvalidArgument if row_stride_bytes < width * channels * sizeof(T) (the requested stride would not fit a single row)
NotImplemented for non-packed formats or non-DMA memory
IoError if the DMA-heap allocation fails (propagated from DmaTensor::new_with_byte_size)

Source

pub fn set_format(&mut self, format: PixelFormat) -> Result<()>

Attach format metadata to an existing tensor.

§Arguments

format - The pixel format to attach

§Returns

Ok(()) on success, with the format stored as metadata on the tensor.

§Errors

Returns Error::InvalidShape if the tensor shape is incompatible with the format’s layout (packed expects [H, W, C], planar expects [C, H, W], semi-planar expects [H*k, W] with format-specific height constraints).

Source

pub fn configure_image( &mut self, width: usize, height: usize, format: PixelFormat, ) -> Result<()>

Set this tensor’s logical dimensions and pixel format to a decoded image, reusing the existing allocation. The shape is derived from the format layout; fails with Error::InsufficientCapacity if the allocation cannot hold width×height in format, or Error::InvalidArgument if the dimensions are invalid for the format.

For NV12/NV16/NV24 the buffer width is rounded up to even (a chroma-plane interleaving requirement); the true odd width is reported by the decoder in ImageInfo and trimmed by a convert() crop. See PixelFormat::image_shape.

When the backing has a fixed physical row pitch (an IOSurface’s 64-aligned bytesPerRow) that exceeds the new format’s natural row stride — i.e. a reused max-sized pool tensor reconfigured to a smaller image — the physical pitch is preserved as the tensor’s row_stride. This keeps the physical grid (allocation stride/surface) fixed while the logical ROI (this image’s W×H) changes, so the decode writes rows at the surface’s real stride and the GPU samples them at the same stride. Exact-sized buffers (pitch == natural) stay tightly packed unchanged.

Source

pub fn image_with_capacity( width: usize, height: usize, format: PixelFormat, memory: Option<TensorMemory>, ) -> Result<Self>
where T: 'static,

Allocate an image tensor sized to hold up to width×height in format, reusable for any smaller image via configure_image.

Source

pub fn format(&self) -> Option<PixelFormat>

Pixel format (None if not an image).

Source

pub fn width(&self) -> Option<usize>

Image width (None if not an image).

Source

pub fn height(&self) -> Option<usize>

Image height (None if not an image).

For semi-planar formats the combined-plane shape row count is divided by the format’s luma-to-total ratio to recover logical height. This returns the exact logical height (including odd heights) only because the logical dimensions are tracked separately from the physical shape — configure_image stores the actual (width, height) in the format’s image_shape, which round-trips losslessly via these accessors.

Source

pub fn from_planes( luma: Tensor<T>, chroma: Tensor<T>, format: PixelFormat, ) -> Result<Self>

Create from separate Y and UV planes (multiplane NV12/NV16).

Source

pub fn is_multiplane(&self) -> bool

Whether this tensor uses separate plane allocations.

Source

pub fn chroma(&self) -> Option<&Tensor<T>>

Access the chroma plane for multiplane semi-planar images.

Source

pub fn chroma_mut(&mut self) -> Option<&mut Tensor<T>>

Mutable access to the chroma plane for multiplane semi-planar images.

Source

pub fn row_stride(&self) -> Option<usize>

Row stride in bytes (None = tightly packed).

Source

pub fn effective_row_stride(&self) -> Option<usize>

Effective row stride in bytes: the stored stride if set, otherwise the minimum stride computed from the format, width, and element size. Returns None only when no format is set and no explicit stride was stored via set_row_stride.

GREY note: effective_row_stride() for a GREY tensor returns the tight width bytes (no padding), which is what normalize_to_numpy and the CPU convert path expect. The codec’s internal native_row_stride (64-byte-aligned) is used only during decoding and is not propagated to the tensor’s stored stride, so callers reading via effective_row_stride() always see the tight value for GREY.

Source

pub fn set_row_stride(&mut self, stride: usize) -> Result<()>

Set the row stride in bytes for externally allocated buffers with row padding (e.g. V4L2 or GStreamer allocators).

The stride is propagated to the EGL DMA-BUF import attributes so the GPU interprets the padded buffer layout correctly. Must be called after set_format and before the tensor is first passed to [ImageProcessor::convert]. The stored stride is cleared automatically if the pixel format is later changed.

No stride-vs-buffer-size validation is performed because the backing allocation size is not reliably known: external DMA-BUFs may be over-allocated by the allocator, and internal tensors store a logical (unpadded) shape. An incorrect stride will be caught by the EGL driver at import time.

§Arguments

stride - Row stride in bytes. Must be >= the minimum stride for the format (width * channels * sizeof(T) for packed, width * sizeof(T) for planar/semi-planar).

§Errors

InvalidArgument if no pixel format is set on this tensor
InvalidArgument if stride is less than the minimum for the format and width

Source

pub fn set_row_stride_unchecked(&mut self, stride: usize)

Set the row stride without format validation.

Use this for raw sub-tensors (e.g. chroma planes) that don’t carry format metadata. The caller is responsible for ensuring the stride is valid.

Source

pub fn with_row_stride(self, stride: usize) -> Result<Self>

Builder-style variant of set_row_stride, consuming and returning self.

§Errors

Same conditions as set_row_stride.

Source

pub fn plane_offset(&self) -> Option<usize>

Byte offset within the DMA-BUF where image data starts (None = 0).

Source

pub fn view_origin(&self) -> Option<ViewOrigin>

The parent-image snapshot if this tensor is a view/ batch sub-region; None for a whole tensor. The GL backend keys its import on the parent geometry and renders this view as a glViewport/glScissor ROI at (x, y, width, height). See ViewOrigin.

Source

pub fn set_plane_offset(&mut self, offset: usize)

Set the byte offset within the DMA-BUF where image data starts.

Propagated to EGL_DMA_BUF_PLANE0_OFFSET_EXT on GPU import. Unlike set_row_stride, no format is required since the offset is format-independent.

Source

pub fn colorimetry(&self) -> Option<Colorimetry>

Colorimetry metadata (None = undefined; never auto-filled).

Source

pub fn set_colorimetry(&mut self, c: Option<Colorimetry>)

Attach/clear colorimetry metadata.

Source

pub fn with_colorimetry(self, c: Colorimetry) -> Self

Builder-style colorimetry attach.

Source

pub fn batch(&self, n: usize) -> Result<Tensor<T>>

Borrow batch element n of a batched tensor as a zero-copy view.

A batched tensor prepends N as the leading dimension over the per-element image layout ([N, H, W, C] packed or [N, C, H, W] planar) — N is over the whole per-element block regardless of HWC/CHW. batch(n) returns element n: the contiguous per-element region at byte offset n * element_size, sharing the parent’s BufferIdentity and inheriting its format / row stride / colorimetry. batch(0) on a tensor with N == 1 is equivalent to the whole tensor.

§Errors

Error::BatchIndexOutOfBounds if n >= N.
Error::InvalidShape if the tensor is not batched (a formatted tensor whose rank lacks the leading N, or an empty shape).

Source

pub fn view(&self, region: Region) -> Result<Tensor<T>>

Borrow a rectangular spatial sub-region of an image tensor as a zero-copy view — the destination/source crop primitive.

region is in pixels of the image’s leading frame. The returned view shares the parent’s BufferIdentity and addresses the sub-rectangle by offset + the parent’s row pitch (so each row lands at the correct columns). convert(src, &mut dst.view(rect), …) renders into that sub-rectangle of dst; a letterbox fit then clears the view and renders the aspect-preserved content into its inner region. view/batch/the whole tensor are the one coherent destination model — there is no separate dst_rect.

§Errors

Error::RegionOutOfBounds if region exceeds the image bounds.
Error::InvalidOperation if the tensor is not a packed-format image (planar/semi-planar spatial sub-rects are not a single strided window; use batch for batched planar tensors).

Source

pub fn as_pbo(&self) -> Option<&PboTensor<T>>

Downcast to PBO tensor reference (for GL backends).

Source

pub fn as_dma(&self) -> Option<&DmaTensor<T>>

Downcast to DMA tensor reference (for EGL import, G2D).

Source

pub fn dmabuf(&self) -> Result<BorrowedFd<'_>>

Borrow the DMA-BUF file descriptor backing this tensor.

§Returns

A borrowed reference to the DMA-BUF file descriptor, tied to self’s lifetime.

§Errors

Returns Error::NotImplemented if the tensor is not DMA-backed.

Source

pub fn from_pbo(pbo: PboTensor<T>) -> Self

Construct a Tensor from a PBO tensor (for GL backends that allocate PBOs).

Source

pub fn cuda(&self) -> Option<&CudaHandle>

The CUDA registration for this tensor, if any (set at creation on CUDA devices).

Source

pub fn set_cuda_handle(&mut self, h: CudaHandle)

Attach a CUDA handle (called by ImageProcessor::create_image after registering a PBO).

Source

pub fn cuda_map(&self) -> Option<CudaMap<'_>>

Fast-fail CUDA map: None (no GL routing) when no handle; else map (PBO routes to the GL worker).

Returns a scoped CudaMap guard holding the raw CUDA device pointer for the duration of the mapping. For GL-buffer-backed tensors the unmap is deferred until the guard drops, freeing the PBO for the next convert() call. When no CUDA handle is attached (the common case for plain Mem/DMA tensors without CUDA registration), returns None immediately — no GL routing, no allocation.

§Example — zero-copy CUDA input with host fallback

use edgefirst_tensor::{Tensor, TensorMemory, TensorTrait};
// Try the zero-copy CUDA device pointer first.
if let Some(cuda) = t.cuda_map() {
    feed_tensorrt(cuda.device_ptr(), cuda.len());
    // `cuda` (a CudaMap guard) unmaps when it goes out of scope, freeing
    // the GPU buffer for the next convert().
} else {
    // Fall back to the host mapping when no CUDA handle is attached.
    let _host = t.map().expect("host map fallback must succeed");
    // `_host` is a TensorMap<f32> that derefs to &[f32].
}

Source

pub fn try_init_dma_cuda(&mut self)

Attempt to attach a CUDA ExternalMemory handle for DMA-backed tensors.

On a CUDA-capable host, imports the DMA-BUF fd via cudaImportExternalMemory(OpaqueFd) and maps it to a device pointer. Sets self.cuda to a persistent ExternalMem handle on success. No-op if CUDA is unavailable, the tensor is not DMA-backed, or a handle is already set. Import failure is silently ignored — the tensor remains usable without a CUDA handle.

§RUNTIME-UNVALIDATED

No test platform has both /dev/dma_heap and a CUDA device. ABI is layout-asserted vs. CUDA 12.6 driver_types.h; the mechanism is proven by gpu-probe O5 on Orin. Best-effort: tensor creation never fails here.

Source §

impl<T> Tensor<T>
where T: IntegerType + Num + Clone + Debug + Send + Sync,

Source

pub fn quantization(&self) -> Option<&Quantization>

Quantization metadata for this tensor, if set.

Source

pub fn set_quantization(&mut self, q: Quantization) -> Result<()>

Attach quantization metadata to this tensor. Validates against the tensor’s shape — returns Error::QuantizationInvalid on any inconsistency (mismatched scale/zp lengths, out-of-range axis, etc.).

Source

pub fn with_quantization(self, q: Quantization) -> Result<Self>

Builder-style variant of Self::set_quantization. Consumes self and returns Result<Self> — on success yields the tensor with the attached quantization; on validation failure returns Error::QuantizationInvalid and drops self (the tensor is not returned in the error arm).

Source

pub fn clear_quantization(&mut self)

Clear any quantization metadata on this tensor.

Trait Implementations§

Source §

impl<T> Debug for Tensor<T>
where T: Num + Clone + Debug + Send + Sync + Debug,

Source §

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Source §