Module video 

Hardware video encoding into, and decoding of, H.264 elementary streams.

VideoEncoder compresses image tensors with the platform’s video encoder, and VideoDecoder turns access units back into image tensors. On Linux both are V4L2 stateful memory-to-memory devices (i.MX 8M Plus vsi_v4l2, i.MX 95 Wave6, …), discovered by capability. Source tensors in DMA-BUF memory are imported without a copy, and decoded pictures are returned in the DMA-BUF buffers the device wrote; only the compressed bitstream is copied.

use edgefirst_codec::video::{EncoderConfig, FrameOptions, VideoEncoder};
use edgefirst_tensor::{CpuAccess, PixelFormat, TensorDyn, TensorMemory};

let mut encoder = VideoEncoder::new(EncoderConfig::h264(1920, 1080, PixelFormat::Nv12, 30.0))?;
let frame = TensorDyn::image(1920, 1080, PixelFormat::Nv12, edgefirst_tensor::DType::U8,
                             Some(TensorMemory::DmaBuf), CpuAccess::Write)?;
let mut stream = Vec::new();
for pts in 0..30u64 {
    let opts = FrameOptions { pts, ..FrameOptions::default() };
    if let Some(au) = encoder.encode(&frame, &opts)? {
        stream.extend_from_slice(&au.data);
    }
}
for au in encoder.flush()? {
    stream.extend_from_slice(&au.data);
}

Decoding takes one access unit at a time; access_units splits a byte stream into them. Each call returns everything the decoder has ready.

use edgefirst_codec::video::{access_units, DecodeEvent, DecoderConfig, VideoDecoder};

let stream = std::fs::read("clip.h264")?;
let mut decoder = VideoDecoder::new(DecoderConfig::h264())?;
let mut show = |event| match event {
    DecodeEvent::Frame(frame) => println!("{} {:?}", frame.pts(), frame.tensor().shape()),
    DecodeEvent::Reconfigured(info) => println!("{}x{}", info.width, info.height),
};
for (pts, au) in access_units(&stream).enumerate() {
    decoder.decode(au, pts as u64)?.into_iter().for_each(&mut show);
}
decoder.flush()?.into_iter().for_each(&mut show);

Structs

AccessUnits

Iterator over the access units of a byte stream, from access_units.

DecodedFrame

A decoded picture, lent from the decoder’s pool.

DecoderConfig

Configuration of a VideoDecoder.

EncodedFrame

One encoded access unit.

EncoderConfig

Configuration of a VideoEncoder.

FrameOptions

Per-frame options for VideoEncoder::encode.

StreamInfo

Geometry and format of the decoded pictures, known once the decoder has parsed the stream’s headers.

VideoDecoder

A hardware video decoder.

VideoEncoder

A hardware video encoder.

Enums

Bitrate

Target bitrate of an encoder.

DecodeEvent

What VideoDecoder::decode and VideoDecoder::flush produce, in order.

H264Level

H.264 level (ITU-T H.264 Table A-1).

H264Profile

H.264 profile (ITU-T H.264 Annex A).

OutputMemory

Whose memory a decoder writes its pictures to. Either way the frames are DMA-BUF tensors and reach the GPU without a copy.

OutputPool

Where a VideoDecoder writes decoded pictures.

VideoCodec

A compressed video format.

Functions

access_units

Splits an H.264 Annex B byte stream into access units, the input VideoDecoder::decode takes.