ff-render

GPU compositing pipeline for real-time video preview, built on [wgpu]. Apply per-frame visual effects — colour grading, blending, masking, chroma key, YUV upload — in a linear render graph wired directly to ff-preview's PlayerRunner.

Project status (as of 2026-04-20): This crate is in an early phase. The high-level API is designed and reviewed by hand; AI is used as an accelerator to implement FFmpeg bindings efficiently. Code contributions are not expected at this time — questions, bug reports, and feature requests are welcome. See the main repository for full context.

Installation

[dependencies]
ff-render = "0.14"

# Enable GPU processing (requires wgpu-compatible hardware)
ff-render = { version = "0.14", features = ["wgpu"] }

Feature Flags

Without wgpu only the CPU fallback path is available via RenderGraph::process_cpu. The CPU path is suitable for unit tests, CI, and software-only environments.

CPU Path (no wgpu required)

All built-in nodes implement RenderNodeCpu, which processes raw RGBA bytes without any GPU dependency.

use ff_render::{RenderGraph, ColorGradeNode, BlendMode, BlendModeNode};

// Build a pipeline: boost brightness then multiply-blend with an overlay.
let overlay_rgba: Vec<u8> = /* ... */ vec![0u8; 4 * 4 * 4];
let graph = RenderGraph::new_cpu()
    .push_cpu(ColorGradeNode::new(0.2, 1.0, 1.0, 0.0, 0.0))
    .push_cpu(BlendModeNode::new(BlendMode::Multiply, 0.8, overlay_rgba, 4, 4));

let input_rgba: Vec<u8> = /* decoded frame */ vec![128u8; 4 * 4 * 4];
let output: Vec<u8> = graph.process_cpu(&input_rgba, 4, 4);

GPU Path (wgpu feature)

When the wgpu feature is enabled, nodes run on the GPU via RenderGraph::process_gpu. The same nodes implement both RenderNode (GPU) and RenderNodeCpu (CPU fallback).

#[cfg(feature = "wgpu")]
use std::sync::Arc;
#[cfg(feature = "wgpu")]
use ff_render::{RenderContext, RenderGraph, ColorGradeNode};

#[cfg(feature = "wgpu")]
async fn example() -> Result<(), ff_render::RenderError> {
    let ctx = Arc::new(RenderContext::init().await?);
    let graph = RenderGraph::new(Arc::clone(&ctx))
        .push(ColorGradeNode::new(0.1, 1.2, 1.0, 0.0, 0.0));

    let input_rgba = vec![128u8; 1920 * 1080 * 4];
    let output = graph.process_gpu(&input_rgba, 1920, 1080).await?;
    Ok(())
}

Integration with ff-preview

GpuFrameSink implements ff_preview::FrameSink, wiring the render graph directly into a PlayerRunner pipeline.

#[cfg(feature = "wgpu")]
use std::sync::Arc;
#[cfg(feature = "wgpu")]
use ff_preview::{PreviewPlayer, RgbaSink};
#[cfg(feature = "wgpu")]
use ff_render::{RenderContext, RenderGraph, ColorGradeNode, GpuFrameSink};

#[cfg(feature = "wgpu")]
async fn with_preview() -> Result<(), Box<dyn std::error::Error>> {
    let ctx = Arc::new(RenderContext::init().await?);

    let graph = RenderGraph::new(Arc::clone(&ctx))
        .push(ColorGradeNode::new(0.1, 1.0, 1.0, 0.0, 0.0));

    let downstream = RgbaSink::new();
    let handle = downstream.frame_handle();

    let (mut runner, _player_handle) = PreviewPlayer::open("clip.mp4")?.split();
    runner.set_sink(Box::new(GpuFrameSink::new(graph, Box::new(downstream))));

    std::thread::spawn(move || runner.run());

    // Retrieve the latest processed frame from any thread.
    if let Some(frame) = handle.lock().unwrap().as_ref() {
        println!("frame: {}×{} pts={:?}", frame.width, frame.height, frame.pts);
    }
    Ok(())
}

Multi-Layer Compositor (wgpu feature)

Compositor is a stateful high-level compositor that accepts a Vec<FrameLayer>, sorts layers by z_order, applies per-layer transforms and blend modes, and returns the composited wgpu::Texture.

#[cfg(feature = "wgpu")]
use std::sync::Arc;
#[cfg(feature = "wgpu")]
use ff_render::{
    RenderContext, Compositor, FrameLayer, LayerTransform, BlendMode,
};

#[cfg(feature = "wgpu")]
async fn compositor_example() -> Result<(), ff_render::RenderError> {
    let ctx = Arc::new(RenderContext::init().await?);
    let mut comp = Compositor::new(Arc::clone(&ctx), 1920, 1080);

    let mut layers = vec![
        FrameLayer {
            frame:      background_frame,
            transform:  LayerTransform::default(),   // identity
            blend_mode: BlendMode::Normal,
            opacity:    1.0,
            z_order:    0,
        },
        FrameLayer {
            frame:      overlay_frame,
            transform:  LayerTransform { x: 0.1, scale_x: 0.5, scale_y: 0.5, ..Default::default() },
            blend_mode: BlendMode::Screen,
            opacity:    0.8,
            z_order:    1,
        },
    ];

    let texture: wgpu::Texture = comp.composite(&mut layers)?;
    Ok(())
}

Built-in Nodes

Blend Modes

BlendModeNode supports the following modes via BlendMode:

Normal · Multiply · Screen · Overlay · Darken · Lighten · ColorDodge · ColorBurn · HardLight · SoftLight · Difference · Exclusion

YUV Upload

YuvUploadNode accepts planar YUV data directly, bypassing sws_scale:

use ff_render::{RenderGraph, YuvUploadNode, YuvFormat};

let mut node = YuvUploadNode::new(YuvFormat::Yuv420p, 1920, 1080);
node.set_planes(y_plane, cb_plane, cr_plane);

let graph = RenderGraph::new_cpu().push_cpu(node);
let rgba = graph.process_cpu(&vec![0u8; 1920 * 1080 * 4], 1920, 1080);

Supported formats: Yuv420p, Yuv422p, Yuv444p.

Error Handling

All fallible operations return RenderError:

use ff_render::RenderError;

match result {
    Err(RenderError::Ffmpeg { code, message }) => { /* wgpu / FFmpeg error */ }
    Err(RenderError::UnsupportedFormat)        => { /* pixel format not supported */ }
    Err(RenderError::Composite { message })    => { /* compositor error */ }
    Ok(output) => { /* process output */ }
}

Crate stack

ff-sys → ff-common → ff-format → ff-preview → ff-render

ff-render depends on ff-preview for the FrameSink trait and VideoFrame type. It has no direct dependency on ff-decode or ff-filter — frames can come from any source.