videotoolbox

Safe Rust bindings for Apple's VideoToolbox framework — hardware-accelerated encode/decode, pixel transfer/rotation, multipass helpers, HDR metadata, motion estimation, RAW processing, and VTFrameProcessor pipelines on macOS.

Status: experimental, but the crate now covers the main public VideoToolbox surfaces used by the doom-fish stack. Objective-C-only APIs use a small Swift bridge behind the frame_processor feature, and executor-agnostic encode/decode futures live in videotoolbox::async_api behind the async feature.

Features

• Hardware-accelerated encoding + decoding — H.264, HEVC, and ProRes 422/4444
• Pixel transfer / rotation / utilities — VTPixelTransferSession, VTPixelRotationSession, VTCreateCGImageFromCVPixelBuffer
• Multipass + HDR helpers — VTFrameSilo, VTMultiPassStorage, VTHDRPerFrameMetadataGenerationSession
• Advanced processing — VTFrameProcessor, VTMotionEstimationSession, VTRAWProcessingSession
• Direct IOSurface input/output — zero-copy composition with apple-cf::iosurface
• Builder pattern — fluent encoder configuration for bitrate, frame rate, keyframe interval, real-time mode, and profile level
• Executor-agnostic async module — videotoolbox::async_api::{AsyncCompressionSession, AsyncDecompressionSession} bridges one-shot frame callbacks to Futures via doom-fish-utils::completion
• Mostly pure C bindings — optional Swift bridge only for Objective-C-only APIs
• Minimal dependencies — apple-cf, plus optional apple-metal for VTFrameProcessor command-buffer integration

Why not bindgen?

The full VideoToolbox SDK surface is large, but the useful set for real macOS media pipelines is still small enough to hand-audit. Hand-writing those declarations gives us:

• No build-time dependency on clang
• Type-safe Rust enums for codec types (instead of raw u32 four-character codes)
• Builder APIs that map ergonomically to VT's CFDictionary property bag

Requirements

• macOS 13.0+
• Apple Silicon or Intel Mac with hardware video encoder

Quick start

use videotoolbox::prelude::*;
use apple_cf::iosurface::{IOSurface, IOSurfaceLockOptions};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Allocate a 1920×1080 BGRA IOSurface.
    let surface = IOSurface::create(1920, 1080, u32::from_be_bytes(*b"BGRA"), 4)
        .ok_or("failed to allocate")?;

    // Build a real-time H.264 encoder.
    let encoder = CompressionSession::builder(1920, 1080, Codec::H264)
        .with_real_time(true)
        .with_average_bit_rate(8_000_000)
        .with_expected_frame_rate(60.0)
        .with_max_keyframe_interval(120)
        .build()?;

    // Encode one frame and inspect the resulting CMSampleBuffer.
    let encoded = encoder.encode(&surface, (0, 60))?;
    println!("Got {} bytes of H.264", encoded.data.len());

    if let Some(sb) = encoded.cm_sample_buffer() {
        // Hand `sb` straight to avassetwriter::Writer::append_sample for
        // zero-copy muxing — no raw pointer hand-off needed.
        let _ = sb.is_valid();
    }

    Ok(())
}

Composes with the rest of the doom-fish stack

screencapturekit-rs ──► IOSurface ──► videotoolbox-rs ──► H.264 bytes
                                              ↓
                                        avassetwriter-rs (future)
                                              ↓
                                          .mp4 file

Roadmap

• VTCompressionSession (encoder)
• VTDecompressionSession (decoder)
• VTPixelTransferSession (pixel format / colour space conversion)
• VTPixelRotationSession
• VTMultiPassStorage + VTFrameSilo (two-pass encoding)
• VTHDRPerFrameMetadataGenerationSession (Dolby Vision metadata)
• VTFrameProcessor capability queries (super-resolution / optical flow detection)
• VTFrameProcessor pipeline (super-resolution + motion blur + temporal noise + frame-rate conversion + optical flow + 2 low-latency variants)
• VTMotionEstimationSession
• VTRAWProcessingSession (with parameter introspection)
• VTProfessionalVideoWorkflow decoder/encoder registration
• VTCreateCGImageFromCVPixelBuffer
• HEVC profile-level helpers
• Executor-agnostic async encode/decode module behind the async feature

License

Licensed under either of Apache-2.0 or MIT at your option.