# edgefirst-tensor Testing
## Test Layout
```
crates/tensor/
├── src/
│ ├── lib.rs # Doc-tests on Tensor<T> public API
│ ├── dma.rs # DMA backend unit tests (Linux-only)
│ ├── shm.rs # POSIX shared-memory backend unit tests
│ ├── mem.rs # Heap backend unit tests
│ ├── pbo.rs # PboTensor + WeakSender behavior
│ ├── iosurface.rs # IOSurface backend (macOS/iOS) — runs on macOS hosts
│ ├── ahardwarebuffer.rs # AHardwareBuffer backend (Android) — FFI, on-device only
│ ├── ahardwarebuffer_layout.rs # Pure Android layout/policy logic — host-tested everywhere
│ ├── cuda.rs # Mock CudaGlOps map/unmap guards, degradation without libcudart, ABI asserts
│ ├── colorimetry.rs # V4L2 → Colorimetry axis mapping, JFIF = BT.601 full, BT luma constants
│ ├── tensor_dyn.rs # TensorDyn metadata, multi-plane, format/stride checks
│ ├── format.rs # PixelFormat + DType compatibility
│ └── error.rs # Error Display strings (incl. InsufficientCapacity message)
└── benches/
└── tensor_benchmark.rs # Allocation + map/unmap throughput
```
Per-backend unit tests live in the source files alongside the implementation. `tests/` holds the integration binaries, which reach the crate only through its public API:
| `protocol.rs`, `protocol_roundtrip.rs` | Descriptor export/import, including a Shm fd relabelled as `kind::DMABUF` so the import path runs without a heap |
| `pin.rs`, `identity.rs`, `map_access.rs`, `logical_shape.rs` | Host pins, `BufferIdentity`, CPU-access declarations, logical-shape narrowing |
| `image_geometry.rs` | Formatted `[N, H, W, C]` geometry: `width()`/`height()` describe one element, `batch()` pitch |
| `unplanned_access.rs` | `unplanned_cpu_access_count()`, in its own binary because the counter is process-global |
| `scenarios.rs` | Raw-handle aliasing scenarios for `TensorDyn`, run under Miri by `scripts/miri.sh` |
| `vocabulary.rs` | `ef_vocabulary!` and the `DType`/`PixelFormat`/`TensorMemory` codes it pins |
| `no_global_state.rs` | The crate carries no process-wide state that affects behaviour |
| `dma_require_policy.rs`, `cuda_require_policy.rs` | The `HAL_TEST_REQUIRE_DMA` / `HAL_TEST_REQUIRE_CUDA` decision functions in `support/` |
| `cuda_runtime_loader.rs` | libcudart loading; under `HAL_TEST_REQUIRE_CUDA=1` a loaded runtime must also create and synchronize a stream |
| `d3d11_tensor.rs` | Windows D3D11 texture tensors (Windows only) |
| `dynamic_primitives.rs` | The `dynamic` backend (`--no-default-features --features dynamic`) |
## Running Tests
All `cargo test` invocations below pass `-- --test-threads=1` per the
workspace single-threaded rule
([root TESTING.md § Single-threaded execution](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md#single-threaded-execution)).
```bash
# All tensor tests on the host
cargo test -p edgefirst-tensor -- --test-threads=1
# Heap-only (skips DMA tests when DMA-heap permissions are missing)
EDGEFIRST_TENSOR_FORCE_MEM=1 cargo test -p edgefirst-tensor -- --test-threads=1
# Doc-tests only
cargo test -p edgefirst-tensor --doc -- --test-threads=1
# With ndarray feature exercised explicitly
cargo test -p edgefirst-tensor --features ndarray -- --test-threads=1
```
## Special Requirements
- **DMA tests** require Linux **and** read+write access to a DMA-heap device
(typically `/dev/dma_heap/linux,cma` or `/dev/dma_heap/system`). On a workstation
this usually means being in the `video` group. If permissions are missing,
DMA tests are skipped at runtime and the suite falls back to heap-only
coverage; nothing fails outright. Set `EDGEFIRST_TENSOR_FORCE_MEM=1` to
skip even the probe.
**A green run therefore says nothing about the DMA paths.** This is how the
`from_fd` minor-number misclassification reached a release: the only test
covering the import path skipped on every unprivileged host, and asserted
fd counts rather than the backend it got back. When touching DMA import or
allocation, confirm the tests actually ran — build unprivileged and run the
binary under `sudo` if the heap device is root-only:
```bash
cargo test -p edgefirst-tensor --lib --no-run sudo ./target/debug/deps/edgefirst_tensor-<hash> --test-threads=1
```
Or set `HAL_TEST_REQUIRE_DMA=1`, which turns every such skip into a failure; see [DMA require gate](#dma-require-gate) below.
- **Import classification tests** (`test_from_fd_dma_imports_as_dma`,
`test_from_fd_shm_imports_as_shm`,
`test_from_fd_rejects_unknown_filesystem`) pin the `from_fd` contract:
a DMA-BUF fd must come back as `TensorMemory::DmaBuf`, a tmpfs fd as
`Shm`, and anything else must be rejected with `UnknownBufferType`
rather than assumed to be shared memory. The unknown case uses a pipe —
`pipefs` is another `get_anon_bdev()` pseudo-filesystem, so it shares
major 0 with the buffer types we do support and is distinguishable only
by magic. Assert on `memory()`, never merely on "the import succeeded":
the wrong branch mmaps fine and looks healthy (a pipe even imports as a
zero-length tensor). See ARCHITECTURE.md § Import classification.
- **PBO tests** live in
[`crates/tensor/src/pbo.rs`](https://github.com/EdgeFirstAI/hal/blob/main/crates/tensor/src/pbo.rs)
and exercise `PboTensor` against a mock `PboOps` implementation. They
are not gated on any feature — `cargo test -p edgefirst-tensor` runs
them on every host. End-to-end PBO behavior (real GL thread, EGL
context) is additionally exercised by the image crate's tests.
- **IOSurface tests** (`iosurface.rs`) are macOS-gated and run on the
macOS CI leg: allocation, wrap-by-`IOSurfaceID`, stride/geometry, and
the read-only lock round-trip (`map_read` locks with
`kIOSurfaceLockReadOnly`; write through a read map asserts).
- **Android logic is host-tested; Android FFI is device-tested.** Every
pure decision the AHardwareBuffer backend makes lives in
`ahardwarebuffer_layout.rs` — the CpuAccess→lock-usage mapping,
descriptor geometry and gralloc stride handling, packed RGB u8/i8
RGBA8888 mapping, the `ro.hardware.egl` compression-scheme classifier
and format-eligibility table, and the `BufferIdentity` intern policy —
and runs as ordinary host tests on every CI lane, so drift is caught
without a device. The FFI shell (`ahardwarebuffer.rs`) executes only
on-device via the internal hal-mobile Device Farm harness (root
TESTING.md § Android On-Device Validation).
- **CpuAccess map-contract tests** are host-portable: the declared-vs-
requested map-mode table, the `writable` assert on read-only maps, and
the `unplanned_cpu_access_count()` warn-once telemetry.
- **No LFS testdata.** All shapes and inputs are synthesized in-test.
`tensor_benchmark` does not use `edgefirst_bench::testdata`; it
generates its inputs in memory.
- **No feature flags required by default**; `ndarray` is on by default and
gates `TensorMapTrait::view()` / `view_mut()`, which return ndarray
`ArrayView` / `ArrayViewMut` over a mapped tensor. Building with
`--no-default-features` compiles those methods out, so any test that touches
them goes with it. Note these are unrelated to `Tensor::view(region)`, the
zero-copy sub-region primitive, which is always available.
## DMA require gate
`crates/tensor/tests/support/dma_require.rs` is the guard, mirroring `tests/support/cuda_require.rs`: under `HAL_TEST_REQUIRE_DMA=1` a DMA test that would skip for want of a usable heap fails instead, naming the test and the reason. Without the variable it skips as before and writes `SKIPPED: <test> - <reason>` straight to stderr, so libtest's capture cannot hide it.
- Unit tests in `src/` call `crate::test_support::dma_or_skip("<test name>")`, which `#[path]`-includes the same support module and feeds it `is_dma_available()`.
- Integration tests in `tests/` include `support/dma_require.rs` with `#[path]` and call `dma_require::available_or_skip(edgefirst_tensor::is_dma_available(), "<test name>")`.
- Its pure decision core, `decide`, is covered on every platform by `tests/dma_require_policy.rs`, which includes the same file rather than copying it.
A test that needs a DMA heap should return early through one of these, never through a hand-rolled `if !is_dma_available() { return; }`.
CI arms the gate with `.github/scripts/dma-heap-setup.sh` only where a probe allocation from the heap succeeds (the Linux mutation legs, and the hosted Linux lanes and board pre-command through `ci-setup.sh`); see root [TESTING.md § CI guard against silent DMA skips](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md#ci-guard-against-silent-dma-skips). macOS and Windows have no DMA heap and leave it unset.
To confirm the gate fails loudly on a workstation that does have a heap, hide the node in a private mount namespace:
```bash
unshare -rm sh -c 'mount --bind /dev/null /dev/dma_heap/system &&
HAL_TEST_REQUIRE_DMA=1 cargo test -p edgefirst-tensor -- --test-threads=1'
```
`EDGEFIRST_TENSOR_FORCE_MEM=1` also makes every DMA test skip, so do not combine it with the gate.
## Mutation testing
The nightly mutation lane runs this crate on all four legs (Linux x86_64 and arm64, macOS, Windows); see root [TESTING.md § Mutation Testing](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md#mutation-testing) for the legs, how survivors are judged and how to mark an equivalent mutant. To mutate one file locally:
```bash
cargo mutants -p edgefirst-tensor --file crates/tensor/src/<file>.rs --in-place -- -- --test-threads=1
```
A survivor in a DMA path only counts as a test gap if the tests actually ran against a heap, so run it with `HAL_TEST_REQUIRE_DMA=1` on a host that has one. `ahardwarebuffer.rs` is excluded from the corpus in `.cargo/mutants.toml` (no Android runner); its pure logic is in `ahardwarebuffer_layout.rs`, which is mutated everywhere.
## Benchmarks
```bash
# Native-host benchmark
cargo bench -p edgefirst-tensor
# Cross-compile for a target (see the project Makefile / BENCHMARKS.md
# for the full deploy workflow)
cargo-zigbuild zigbuild --target aarch64-unknown-linux-gnu --release \
-p edgefirst-tensor --bench tensor_benchmark
```
| `tensor_benchmark` | [`benches/tensor_benchmark.rs`](https://github.com/EdgeFirstAI/hal/blob/main/crates/tensor/benches/tensor_benchmark.rs) | Allocation cost across DMA/SHM/Mem backends; map/unmap throughput; clone_fd cost |
## Coverage Notes
- The crate participates in workspace `cargo llvm-cov`; lcov output
appears under `crates/tensor/`.
- `cfg(target_os = "linux")`-gated code (`dma.rs`, `dmabuf.rs`) is only
exercised on the Linux host runners and on the i.MX 8M Plus hardware
runner. The macOS CI leg covers the heap and IOSurface paths.
`cfg(target_os = "android")` code (`ahardwarebuffer.rs`) is excluded
from Sonar coverage (no CI lane can execute it) and is covered
on-device by the hal-mobile harness; its pure logic is host-covered
via `ahardwarebuffer_layout.rs`. There is no Windows
test job today; SHM coverage comes from the Linux runners (where
`shm.rs` is `#[cfg(unix)]`-gated and active).
- The `procfs` dev-dependency on Linux is used to read open-fd counts
for the current process via `procfs::process::Process::myself()` —
the leak-regression tests assert that fd count returns to its
baseline after DMA / SHM tensors are dropped. The check is fd-count
based, not a `/proc/self/maps` parse.
## CUDA Surface
The CUDA zero-copy surface (`cuda_map()`, `is_cuda_available()`, `CudaMap`,
`CudaHandle`) is fully testable without a GPU or `libcudart`.
### GPU-independent tests (run anywhere)
| Mock `CudaGlOps` — map/unmap/unregister call sequence | `crates/tensor/src/cuda.rs` `#[cfg(test)]` | `CudaMap` construction calls `map`; `Drop` calls `unmap`; handle drop calls `unregister` in the correct order (before PBO storage drops) |
| `Debug` impl on `CudaMap` / `CudaHandle` | Same | No panics, no format-string UB |
| Primitive degradation — absent `libcudart` | Same | `is_cuda_available()` returns `false`; `cuda_map()` returns `None`; no crash |
| ABI layout asserts | `crates/tensor/src/cuda.rs` (`mod ext_mem_layout`) | `sizeof` / `alignof` of HAL's CUDA type wrappers match CUDA 12.6 `driver_types.h` — verified as static compile-time assertions |
Run all CUDA tests on the host (no hardware required):
```bash
cargo test -p edgefirst-tensor -- --test-threads=1
```
To exercise just the CUDA tests by name:
```bash
cargo test -p edgefirst-tensor cuda -- --test-threads=1
```
### Windows D3D11 CUDA interop tests
`crates/tensor/tests/d3d11_tensor.rs` (Windows-only, `feature = "static"`)
has three tests exercising CUDA's external-memory import of a D3D11 texture
tensor (`cuda_map_matches_the_cpu_map_and_map_mut_writes_back`,
`cuda_map_covers_a_texture_whose_allocation_the_driver_padded`,
`cuda_map_reads_the_nv12_combined_plane`). Each self-skips on the WARP
adapter (`cudaD3D11GetDevice` has no ordinal for it — that skip is correct by
design), and then when `edgefirst_tensor::is_cuda_available()` is false. The
order matters: the WARP skip is a property of the adapter that no runtime can
satisfy, so checking it first keeps it a skip even under the gate below.
`crates/tensor/tests/support/cuda_require.rs` is the guard, mirroring
`crates/image/src/gl/cuda_policy.rs`: under `HAL_TEST_REQUIRE_CUDA=1` (set by
`scripts/test-windows.ps1 -RequireCuda`, which also arms it from NVIDIA host
detection unless WARP is the selected adapter) a "no CUDA runtime" skip fails
instead of passing, naming the test and the reason. Its pure decision core,
`decide`, is covered platform-generically — so on this box and every CI lane,
not just Windows — by `crates/tensor/tests/cuda_require_policy.rs`, which is
not Windows-gated and which `#[path]`-includes this same support module rather
than copying it, so the tests pin the function `d3d11_tensor.rs` actually
calls.
### On-target validation
On-target tests require a CUDA-capable device with `libcudart.so` present.
Validated platforms: Jetson Orin-nano (O5/O6/O8) on L4T R36.4 / CUDA 12.6 /
TRT 10.3; desktop RTX 3090 on CUDA 12.6.
Cross-compile and deploy:
```bash
cargo-zigbuild test --target aarch64-unknown-linux-gnu --release --no-run \
-p edgefirst-tensor
scp target/aarch64-unknown-linux-gnu/release/deps/edgefirst_tensor-* \
jetson-orin-nano:/tmp/hal-tests/
ssh jetson-orin-nano '/tmp/hal-tests/edgefirst_tensor-<hash> --test-threads=1'
```
The on-target suite includes an end-to-end test that:
1. Creates a float PBO tensor via `ImageProcessor::create_image()`.
2. Runs `convert()` on a synthetic frame.
3. Calls `cuda_map()` and asserts `device_ptr()` is non-null.
4. Validates numeric max-error against a host-side reference: observed
max\_err 0.00024 on Orin-nano at F16 precision.
5. Drops the `CudaMap` guard and asserts the PBO is usable by a subsequent
`convert()` call (aliasing rule enforcement).
### covguard (imx8mp coverage flush)
On the imx8mp lane the Vivante EGL driver may call `abort()` during shutdown.
The `edgefirst_tensor::covguard` module — compiled only under
`-Cinstrument-coverage` — installs a `SIGABRT` handler that flushes the LLVM
profile to disk before re-raising the signal. This is transparent to normal
builds; contributors do not need to do anything special to benefit from it.
See [`TESTING.md § CUDA tensor mapping`](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md#cuda-tensor-mapping)
for the cross-workspace view, including the fallback-contract test and the
CI matrix notes.
## Cross-References
- Project testing patterns: [../../TESTING.md](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md)
- Validating optimizations: [TESTING.md#validating-optimizations](https://github.com/EdgeFirstAI/hal/blob/main/TESTING.md#validating-optimizations)
- Image-side PBO testing: [../image/TESTING.md](https://github.com/EdgeFirstAI/hal/blob/main/crates/image/TESTING.md)
- C API DMA tests: [../tensor-capi/](https://github.com/EdgeFirstAI/hal/blob/main/crates/tensor-capi/)