Expand description
§video-interop
§The problem
A Rust producer and consumer in a zero-copy Linux GPU pipeline exchange DMA-BUF file descriptors instead of copying frame pixels through CPU memory. Each handoff also requires image layout, format, geometry, colorimetry, synchronization, file-descriptor ownership, and producer-buffer lifetime.
An integer file descriptor does not encode those properties and does not own the underlying allocation. Rust code must duplicate borrowed descriptors before retaining them beyond the current native call, then release each producer lease exactly once after GPU use finishes.
§The solution
The video-interop crate provides the Rust types and lifecycle primitives for
the VideoInterop contract. It describes owned binary frames, borrowed Linux
DMA-BUF frames, sync-file acquire fences, frame geometry, colorimetry, and stream
formats. Prepared and claimed frame types establish the ownership boundary
between Elixir and native queues.
The default Rustler integration maps these types to the video_interop Hex
package. This shared contract connects producer and consumer elements written in
Rust to pipeline control written in Elixir. Optional EGL and Vulkan modules
implement synchronization and import operations without taking ownership of the
caller’s graphics runtime.
The crate does not allocate producer buffers, create a renderer, or select a streaming framework.
§Status
The core, Rustler, EGL, and Vulkan APIs are part of the supported 0.1 contract.
The crate supports Rust 1.91 and newer.
§Installation
Rustler support is enabled by default:
[dependencies]
video-interop = "0.1"Disable the default feature for a native program that only uses the frame types:
[dependencies]
video-interop = { version = "0.1", default-features = false }Optional graphics helpers are separate features:
video-interop = { version = "0.1", features = ["egl"] }
video-interop = { version = "0.1", features = ["vulkan"] }§Binary storage
BinaryStorage owns its bytes and plane metadata. Cloning or decoding it copies
the bytes, so it needs neither file-descriptor duplication nor a release lease.
Binary formats include RGBA8888, RGB888, Gray8, Gray2, and BW1.
§DMA-BUF ownership
Descriptor contains borrowed integer file descriptors. Validate it before use.
Call Descriptor::duplicate_cloexec before keeping it after the current call.
The returned OwnedDescriptor closes those duplicates when dropped.
dmabuf_allocation_size reads the complete allocation size exposed by a
DMA-BUF fd. The reported size includes any exporter alignment and driver
padding outside the bytes addressed by the image planes. Producers must store
this size in Object::size. Vulkan imports compare the published size with the
fd before using it.
A sync-file descriptor follows the same rule: it is borrowed until duplicated into an owned frame.
§Rustler frames and leases
The default feature maps both storage kinds to the structs in the
video_interop Hex package. It also provides prepared and claimed frames for
moving either storage kind into native queues.
Frame::prepare_cloexec validates a frame. It copies owned binary bytes without
acquiring a lease client and duplicates borrowed descriptors with CLOEXEC. If
a prepared borrowed frame is dropped, the descriptors close but lease ownership
stays with the Elixir caller.
PreparedVideoFrame::claim moves any lease release responsibility to native
code. Owned binary claims carry no lease. Dropping a borrowed
ClaimedVideoFrame or ClaimedLease sends the release through a
ReleaseDispatcher.
The NIF lifecycle owner must call ReleaseDispatcher::close_and_join from a
dirty I/O NIF after all claims have drained. Dispatcher and guard destructors do
not join worker threads or send through OwnedEnv.
§EGL
The egl feature loads native-fence function pointers supplied by the caller.
It supports:
- native sync-file fence creation and import;
- fence duplication with
FD_CLOEXEC; - server waits and bounded client waits;
- a bounded
pollfallback when EGL import is unavailable.
The caller must keep the EGL display, context, function pointers, and calling thread valid. The module has no link-time EGL or GL dependency.
§Vulkan
The vulkan feature accepts a caller-owned ash device and provides DMA-BUF
capability checks, memory import, sync-file waits, queue-family ownership
transfer, and release-fence handling.
Imported images stay in a bounded source cache. With planar preference, non-linear NV12 is imported as a transfer-source image and copied plane-for-plane into ordinary renderer-owned Y and UV images; linear NV12 uses an imported transfer-source buffer. Packed RGBA and BGRA use a compute copy when the device does not support sampling the producer-linear image. Copy regions cover image bytes only; allocation padding remains mapped but is not copied.
The caller remains responsible for:
- physical-device and queue selection;
- serializing host access to the Vulkan queue;
- choosing a supported import strategy;
- renderer waits and release submission;
- presentation and device-loss recovery.
Platform-specific qualification is tracked separately from the Vulkan API’s supported status.
§Features
| Feature | Default | Provides |
|---|---|---|
rustler | yes | Elixir schema codecs, prepared/claimed frames, release dispatcher |
egl | no | Dynamically loaded EGL native-fence helpers |
vulkan | no | DMA-BUF import and synchronization helpers |
test-support | no | Rustler helpers used by schema tests |
§License
Apache-2.0. See the license.
§Optional Vulkan GPU timing
Consumers can override VulkanDeviceContext::video_gpu_timing_enabled to omit
query-pool allocation and sample_video_gpu_timing to select an entire staged
acquire-to-release bracket. Defaults retain existing instrumentation. Keep the enable
policy fixed for sync-owner lifetimes and make sampling independent of cached slot
indices. report_video_gpu_timing receives resource status; timing is not a capability
or completion authority. ImportedImageSync::has_gpu_timing_resources supports diagnostics.
Off/unsampled paths preserve copies, ownership transfers, waits, release submissions and fences, including release command buffers with no timestamp commands. Results remain optional and are collected only after release completion. Unsupported timestamps and query-pool OOM omit instrumentation; device loss is never optional success.
Existing consumers, including Emerge’s Ganesh/headless renderer, do not need to implement the timing hooks: defaults enable timing for every supported staged frame and ignore status notifications. Direct imports remain untimed. This API has no Graphite dependency. Driver-free legacy-consumer tests cover these defaults, release-fence gating, and repeated sync-owner reuse.
Modules§
Structs§
- Abandonment
Guard - Fallback resource attached to one lease holder.
- Binary
Format - Rust representation of
VideoInterop.Binary.Format. - Binary
Plane - Binary
Storage - Claimed
Lease - Claimed
Video Frame - Colorimetry
- Rust representation of
VideoInterop.Colorimetry. - Descriptor
- Dispatcher
Error - Dispatcher
Probe - DmaBuf
Format - Rust representation of
VideoInterop.DMABuf.Format. - Format
- Rust representation of the
VideoInterop.Formatstream schema. - Frame
- Frame
Descriptor - Layer
- Lease
- Object
- Owned
Descriptor - Owned
Frame - Owned
Object - Plane
- Prepared
Lease - Prepared
Video Frame - Rect
- Release
Dispatcher - Lifecycle-owned dispatcher for normal and fallback lease messages.
- Sync
File
Enums§
- Acquire
Sync - Acquire
Sync Policy - Synchronization policy declared by
VideoInterop.Format. - Alpha
Mode - Binary
Pixel Format - Bw1Polarity
- Chroma
Location - Color
Range - Dispatcher
Health - DmaBuf
Allocation Size Error - Duplicate
Error - Format
Validation Error - Numeric validation errors for a stream format.
- Interlace
Mode - Matrix
- Modifier
- Modifier
Policy - DMA-BUF modifier policy declared by
VideoInterop.DMABuf.Format. - Owned
Acquire Sync - Owned
Storage - Prepare
Error - Primaries
- Storage
- Storage
Format - Transfer
- Validation
Error
Constants§
Functions§
- dmabuf_
allocation_ size - Returns the complete allocation size exposed by a DMA-BUF fd.
- is_
abandonment_ guard_ resource - Creates the resource used by a thin producer-NIF guard constructor.
- new_
abandonment_ guard
Type Aliases§
- Rational
- Positive rational represented as the Elixir
{numerator, denominator}tuple. - Release
Worker Error - Stream
Acquire Sync Policy - Alias matching the stream-policy terminology used by native consumers.
- Stream
Modifier Policy - Alias matching the stream-policy terminology used by native consumers.