Crate videotoolbox

Expand description

§videotoolbox

Safe, zero-runtime-dependency Rust bindings for Apple’s VideoToolbox framework — hardware-accelerated H.264, HEVC, and ProRes codecs on macOS.

Status: experimental. Encoder is functional; decoder, pixel transfer, and multi-pass support are planned.

§Features

Hardware-accelerated encoding — H.264, HEVC, and ProRes 422/4444
Direct IOSurface input — encode zero-copy from screencapturekit / camera output via apple-cf::iosurface
Builder pattern — fluent configuration of bitrate, frame rate, keyframe interval, real-time mode
Pure C bindings — no Swift bridge, no bindgen, no procedural macros
Single dependency — only apple-cf for shared types

§Why not bindgen?

The full VideoToolbox C surface is ~200 symbols, but the useful set for an encoder is closer to 15. Hand-writing those declarations gives us:

No build-time dependency on clang
Type-safe Rust enums for codec types (instead of raw u32 four-character codes)
Builder APIs that map ergonomically to VT’s CFDictionary property bag

§Requirements

macOS 13.0+
Apple Silicon or Intel Mac with hardware video encoder

§Quick start

use videotoolbox::prelude::*;
use apple_cf::iosurface::{IOSurface, IOSurfaceLockOptions};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Allocate a 1920×1080 BGRA IOSurface.
    let surface = IOSurface::create(1920, 1080, u32::from_be_bytes(*b"BGRA"), 4)
        .ok_or("failed to allocate")?;

    // Build a real-time H.264 encoder.
    let encoder = CompressionSession::builder(1920, 1080, Codec::H264)
        .with_real_time(true)
        .with_average_bit_rate(8_000_000)
        .with_expected_frame_rate(60.0)
        .with_max_keyframe_interval(120)
        .build()?;

    // Encode one frame and inspect the resulting CMSampleBuffer.
    let encoded = encoder.encode(&surface, (0, 60))?;
    println!("Got {} bytes of H.264", encoded.data.len());

    if let Some(sb) = encoded.cm_sample_buffer() {
        // Hand `sb` straight to avassetwriter::Writer::append_sample for
        // zero-copy muxing — no raw pointer hand-off needed.
        let _ = sb.is_valid();
    }

    Ok(())
}

§Composes with the rest of the doom-fish stack

screencapturekit-rs ──► IOSurface ──► videotoolbox-rs ──► H.264 bytes
                                              ↓
                                        avassetwriter-rs (future)
                                              ↓
                                          .mp4 file

§Roadmap

VTCompressionSession (encoder)
VTDecompressionSession (decoder)
VTPixelTransferSession (pixel format / colour space conversion)
VTPixelRotationSession
VTMultiPassStorage + VTFrameSilo (two-pass encoding)
VTHDRPerFrameMetadataGenerationSession (Dolby Vision metadata)
VTFrameProcessor capability queries (super-resolution / optical flow detection)
VTFrameProcessor pipeline (super-resolution + motion blur + temporal noise + frame-rate conversion + optical flow + 2 low-latency variants)
VTMotionEstimationSession
VTRAWProcessingSession (with parameter introspection)
VTProfessionalVideoWorkflow decoder/encoder registration
VTCreateCGImageFromCVPixelBuffer
HEVC profile-level helpers
Async encode API via VTCompressionSessionEncodeFrameWithOutputHandler

§License

Licensed under either of Apache-2.0 or MIT at your option.

§API Documentation

Safe, zero-runtime-dependency Rust bindings for Apple’s VideoToolbox framework — hardware-accelerated H.264, HEVC, and ProRes codecs on macOS.

Unlike the rest of the doom-fish suite, VideoToolbox is a pure C framework, so this crate does not ship a Swift bridge — all bindings are direct extern "C" declarations against the system framework.

§Quick start

use videotoolbox::prelude::*;
use apple_cf::iosurface::IOSurface;

let surface = IOSurface::create(1920, 1080, u32::from_be_bytes(*b"BGRA"), 4)
    .ok_or("failed to allocate IOSurface")?;

let encoder = CompressionSession::builder(1920, 1080, Codec::H264)
    .with_real_time(true)
    .with_average_bit_rate(8_000_000)
    .with_expected_frame_rate(60.0)
    .build()?;

let encoded = encoder.encode(&surface, (0, 60))?;
println!("Got {} bytes of H.264", encoded.data.len());

Re-exports§

pub use error::VTError;
pub use encoder_list::available_video_encoders;
pub use encoder_list::VideoEncoder;
pub use hdr_metadata::HdrMetadataSession;
pub use multipass::FrameSilo;
pub use multipass::MultiPassStorage;
pub use session::Codec;
pub use transfer::PixelRotationSession;
pub use transfer::PixelTransferSession;
pub use transfer::Rotation;
pub use utilities::create_cg_image_from_pixel_buffer;
pub use utilities::register_professional_workflow_decoders;
pub use utilities::register_professional_workflow_encoders;
pub use frame_processor::frame_processor_capabilities;frame_processor
pub use frame_processor::super_resolution_supported_scale_factors;frame_processor
pub use frame_processor::FrameProcessor;frame_processor
pub use frame_processor::FrameProcessorCapabilities;frame_processor
pub use motion_estimation::MotionEstimationSession;frame_processor
pub use raw_processing::RawProcessingParameter;frame_processor
pub use raw_processing::RawProcessingSession;frame_processor
pub use compression::CompressionSession;compression
pub use compression::CompressionSessionBuilder;compression
pub use compression::EncodedFrame;compression

Modules§

compressioncompression: CompressionSession — hardware H.264/HEVC/ProRes encoder.
decompression: DecompressionSession — hardware H.264/HEVC/ProRes decoder.
encoder_list: available_video_encoders — enumerate Apple’s installed video encoders. Wraps VTCopyVideoEncoderList.
error: Errors produced by VideoToolbox APIs.
ffi: Raw extern "C" declarations against <VideoToolbox/VideoToolbox.h>, <CoreMedia/CoreMedia.h>, <CoreVideo/CVPixelBuffer.h>, and <CoreFoundation/CoreFoundation.h>.
frame_processorframe_processor: VTFrameProcessor capability queries — isSupported + supported scale factors via the new frame_processor Swift bridge.
hdr_metadata: VTHDRPerFrameMetadataGenerationSession — generate Dolby Vision per-frame HDR metadata (macOS 15+).
motion_estimationframe_processor: VTMotionEstimationSession — between-frame motion-vector estimation backed by the Apple Neural Engine (macOS 26+).
multipass: VTFrameSilo + VTMultiPassStorage — multi-pass video encoding storage.
prelude: Common imports for users of this crate.
raw_processingframe_processor: VTRAWProcessingSession — ProRes RAW / CinemaDNG decoder with per-frame parameter controls (macOS 15+).
session: Video codecs supported by VideoToolbox.
transfer: PixelTransferSession and PixelRotationSession — Apple’s zero-copy pixel-format conversion / scaling / rotation engines.
utilities: VTUtilities + VTProfessionalVideoWorkflow helpers.