Crate oximedia_align

Source

Expand description

Video alignment and registration tools for multi-camera synchronization in OxiMedia.

This crate provides comprehensive tools for aligning and registering video from multiple cameras:

§Temporal Alignment

The temporal module provides time-based synchronization:

Audio Cross-Correlation - Sync cameras using audio tracks
Timecode Synchronization - LTC/VITC-based alignment
Visual Markers - Clapper detection and flash-based sync
Sub-frame Accuracy - Precise timing down to microseconds

§Spatial Registration

The spatial module provides geometric alignment:

Homography Estimation - Planar perspective transformation
Perspective Correction - Remove keystone distortion
Feature Matching - Correspond points between views
RANSAC - Robust outlier rejection

§Feature Detection

The features module provides patent-free feature detection and matching:

FAST Corners - High-speed corner detection
BRIEF Descriptors - Binary robust independent elementary features
ORB Features - Oriented FAST and Rotated BRIEF
Brute-Force Matching - Hamming distance matching

§Lens Distortion

The distortion module corrects lens aberrations:

Brown-Conrady Model - Radial and tangential distortion
Fisheye Model - Wide-angle lens correction
Calibration - Camera intrinsic parameter estimation
Undistortion - Real-time image correction

§Color Matching

The color module matches color across cameras:

Color Transfer - Match color distributions
Histogram Matching - Equalize color histograms
White Balance - Illuminant estimation
Color Calibration - ColorChecker-based calibration

§Sync Markers

The markers module detects synchronization markers:

Clapper Detection - Automatic slate detection
Flash Detection - Bright flash sync
LED Markers - Coded light patterns
Audio Spike - Sharp transient detection

§Rolling Shutter

The rolling_shutter module corrects rolling shutter artifacts:

Motion Estimation - Per-scanline motion vectors
Correction - Remove wobble and skew
Global Shutter Simulation - Temporal interpolation

§Example: Audio-Based Sync

use oximedia_align::temporal::{AudioSync, SyncConfig};
use oximedia_align::AlignResult;

// Configure audio synchronization
let config = SyncConfig {
    sample_rate: 48000,
    window_size: 480000, // 10 seconds
    max_offset: 240000,  // ±5 seconds
};

// Create audio sync analyzer
let sync = AudioSync::new(config);

// Find offset between two audio tracks
// let offset = sync.find_offset(&audio1, &audio2)?;

§Example: Homography Estimation

use oximedia_align::spatial::{HomographyEstimator, RansacConfig};
use oximedia_align::features::{FeatureMatcher, MatchPair};

// Configure RANSAC for robust estimation
let config = RansacConfig {
    threshold: 3.0,
    max_iterations: 1000,
    min_inliers: 8,
};

let estimator = HomographyEstimator::new(config);

// Estimate homography from matched points
// let (homography, inliers) = estimator.estimate(&matches)?;

Modules§

affine: Affine transformation for image alignment.
align_report: Alignment quality reporting and diagnostics.
audio_align: Audio-to-video alignment utilities.
beat_align: Beat-grid alignment for music and rhythmic media synchronisation.
color: Color matching and calibration for multi-camera alignment.
confidence_map: Alignment confidence and quality mapping.
distortion: Lens distortion correction.
drift_correct: Clock drift correction using PLL-based techniques.
drift_correction: Long-form timing drift correction.
elastic_align: Elastic (non-rigid) alignment for deformable media registration.
farneback_flow: Dense optical flow using the Farneback polynomial expansion method.
features: Patent-free feature detection and matching.
frame_matcher: Frame-level descriptor matching for video alignment in OxiMedia.
frequency_align: Frequency-domain alignment for audio and video streams.
gradient_flow: Gradient-based optical flow for video alignment.
icp: Iterative Closest Point (ICP) algorithm for point cloud / contour alignment.
klt_tracker: Kanade-Lucas-Tomasi (KLT) sparse optical flow tracker.
lip_sync: Lip sync alignment for audio/video synchronization.
markers: Synchronization marker detection.
motion_compensate: Motion compensation for temporal video alignment.
multi_stream: Multi-stream alignment for synchronizing groups of audio/video streams.
multicam_sync: Multi-camera synchronisation using various sync methods.
multitrack_align: Multi-track alignment for synchronizing multiple audio/video tracks.
optical_flow: Optical flow computation for frame-to-frame alignment.
phase_correlate: Phase correlation for sub-pixel image alignment.
projective_warp: Projective (perspective) warp transformations for image alignment.
prosac: PROSAC (PROgressive SAmple Consensus) for robust model estimation.
rigid_transform: Rigid body (rotation + translation) transformations for alignment.
rolling_shutter: Rolling shutter correction.
spatial: Spatial registration and geometric alignment.
stabilize: Video stabilization pipeline.
stereo_depth: Depth estimation from a rectified stereo image pair.
stereo_rectify: Stereo image pair rectification for epipolar geometry alignment.
subframe_interp: Sub-frame interpolation for precision alignment.
sync_score: Sync quality scoring for multi-stream alignment in OxiMedia.
tempo_align: Tempo-based audio alignment for music synchronization.
temporal: Temporal synchronization for multi-camera alignment.
temporal_align: Temporal stream alignment utilities for OxiMedia.
transform: Geometric transformations for image alignment.
warp: Time warp / time-stretching alignment using Dynamic Time Warping (DTW).

Structs§

Point2D: 2D point
TimeOffset: Time offset between two streams

Enums§

AlignError: Errors that can occur during alignment operations

Type Aliases§

AlignResult: Result type for alignment operations