oxigaf-render
Differentiable 3D Gaussian Splatting rasterizer using wgpu compute shaders.
Overview
This crate implements a GPU-accelerated, differentiable 3D Gaussian Splatting (3DGS) rasterizer for real-time rendering and gradient-based optimization:
- Forward pass: Project 3D Gaussians → Sort by depth → Tile-based alpha blending
- Backward pass: Compute gradients for positions, rotations, scales, opacities, and colors
- FLAME binding: Anchor Gaussians to a parametric head model for expression control
- wgpu backend: Cross-platform GPU compute (Vulkan, Metal, DX12, WebGPU)
- Tile-based rendering: Efficient parallel rasterization with minimal memory overhead
v0.1.2 — what's included:
- Two gradient-correctness fixes in the backward shaders — a wrong-Gaussian gradient accumulation bug in the backward tile kernel, and a missing position gradient through view-dependent SH color for
sh_degree >= 1models. Both affected every 0.1.1 training run — see "Upgrading from 0.1.1" below before you retrain anything - glTF export — new
gltfmodule (write_gltf/GltfError), a single spec-conformant glTF 2.0 writer consolidating what were three independently-written, mutually-incompatible glTF emitters in the workspace - GPU timestamp profiler —
profiler::GpuTimestampProfiler+Rasterizer::enable_gpu_timestamps(), pass-level GPU-side timing (viawgpu::Features::TIMESTAMP_QUERY) alongside the existing CPU-sidePassProfiler Rasterizer::from_devicenow validates a caller-supplied device's limits up front (rasterizer::rasterizer_device_limits) instead of failing later as an opaque pipeline-validation error- 2,903 tests, all passing (2,887 run by default + 16 GPU-hardware
#[ignore]d tests verified separately)
Upgrading from 0.1.1
- Retrain any
sh_degree >= 1model.preprocess_bwd.wgslonly differentiated the projection path from Gaussian position to screen space, omitting the position gradient through view-dependent SH color — so every 0.1.1 model withsh_degree >= 1trained against a systematically incomplete position gradient. There is no way to recover the missing signal from an already-trained model; retraining is the only remedy. - The backward tile kernel could attribute a tile's gradient sum to the wrong Gaussian.
rasterize_bwd.wgsl's reverse-traversal loop bound was read per-pixel, so threads in the same 16×16 workgroup ran a different number of loop iterations aroundworkgroupBarrier()calls — non-uniform control flow, which WGSL requires to be uniform. Fixed by computing a single workgroup-uniform loop bound. This also affected every 0.1.1 training run. - Re-export any
.plywithsh_degree >= 1written byGaussianModel::save_plybefore this release. Thef_rest_*property order changed to channel-major (matching the reference 3DGS Python convention); older files load with correctly-valued but permuted higher-order SH coefficients. - API renames to watch for:
hdr_tone_mapping::lottes_approx→generalized_reinhard(a real, non-approximatinglottes()was added alongside it);MbStats.mean_samples_used→estimated_sample_utilization;RadixSorter::sortno longer takesdevice/keys/values(that setup moved to a newprepare()step);Rasterizer::from_devicenow errors ontile_size != 16instead of silently accepting it.
See CHANGELOG.md for the complete list of changes, including two more minor backward-pass fixes (a missing background-alpha gradient contribution, and NaN gradients for culled Gaussians).
Installation
[]
= "0.1"
Features
| Feature | Description |
|---|---|
default |
Minimal configuration (production-ready) |
gpu_debug |
GPU validation layers with detailed error messages (10-100× slower) |
Feature Details
gpu_debug: Enables comprehensive GPU debugging- Vulkan validation layers (Linux/Windows)
- Metal API validation (macOS)
- DirectX debug layer (Windows)
- Enhanced error messages and warnings
- Performance profiling with detailed traces
- Warning: Adds significant runtime overhead (10-100× slower)
- Recommended for development/debugging only
Example Usage
# Production use (fast, minimal validation)
= "0.1"
# Development/debugging (slow, extensive validation)
= { = "0.1", = ["gpu_debug"] }
Usage
All the examples below use pollster to block on the async GPU setup calls
from a plain fn main; add pollster = "1" to your own [dependencies]
to run them as-is (any other async executor works too).
Basic Gaussian Rendering
use ;
// `Rasterizer::new` is async because wgpu device creation is async.
async
Differentiable Rendering with Gradients
use ;
async
// --- Helper functions (fill in for your training loop) ---
/// L1 loss gradient: sign(rendered - target).
Custom Camera Trajectories
use ;
use FRAC_PI_4;
async
High-Quality Rendering Configuration
use ;
Buffer Pool for Memory Efficiency
use BufferPool;
Exporting to glTF
use write_gltf;
use ;
use Path;
Building a Rasterizer from a Custom Device
Rasterizer::new creates its own wgpu instance/adapter/device. To reuse an
existing wgpu::Device (e.g. one shared with a windowing/UI layer), raise
its limits with rasterizer_device_limits before requesting it, then call
Rasterizer::from_device. This needs wgpu directly in your own
[dependencies] — add wgpu = "30", matching the version this crate
depends on, since from_device takes a wgpu::Device and a version
mismatch there is a type error, not a warning:
use rasterizer_device_limits;
use ;
async
Architecture
3D Gaussian Splatting
Each Gaussian primitive is defined by:
- Position (3 floats): 3D center point (x, y, z)
- Rotation (4 floats): Quaternion (x, y, z, w) defining orientation
- Scale (3 floats): Log-scale values (sx, sy, sz) — exponentiated before use
- Opacity (1 float): Inverse-sigmoid opacity — passed through sigmoid(x)
- SH Coefficients (N floats): Spherical harmonics for view-dependent color
- Degree 0: 3 coeffs (1 band × 3 RGB)
- Degree 1: 12 coeffs (4 bands × 3 RGB)
- Degree 2: 27 coeffs (9 bands × 3 RGB)
- Degree 3: 48 coeffs (16 bands × 3 RGB)
Rendering Pipeline
- Preprocess: Project Gaussians to screen space, compute covariance matrices
- Sort: Sort Gaussians by depth (front-to-back for alpha blending)
- Rasterize: Tile-based parallel alpha blending
- Output: RGB image + depth map + alpha mask
Shader Specialization
The rasterizer includes specialized shaders for different SH degrees:
preprocess_sh0.wgsl— DC component only (fastest)preprocess_sh1.wgsl— Degree 1 (4 bands)preprocess_sh2.wgsl— Degree 2 (9 bands)preprocess_sh3.wgsl— Degree 3 (16 bands)
Specialization provides 10× speedup over generic shader.
Performance
Rendering performance on various hardware (512×512 resolution):
| Hardware | Gaussians | FPS | Memory |
|---|---|---|---|
| NVIDIA RTX 4090 | 100K | 240+ | 1.2 GB |
| NVIDIA RTX 3080 | 100K | 120+ | 1.5 GB |
| Apple M2 Max | 100K | 60+ | 1.8 GB |
| Apple M1 | 50K | 60+ | 1.2 GB |
Differentiability
Backward pass — all parameters have verified gradients (two gradient-correctness bugs were fixed in 0.1.2 — see "Upgrading from 0.1.1" above):
- ∂L/∂color, ∂L/∂alpha, ∂L/∂conic, ∂L/∂mean2D (rasterize_bwd), including the background's contribution to ∂L/∂alpha through final transmittance
- ∂L/∂SH, ∂L/∂scale, ∂L/∂rotation, ∂L/∂mean3D (preprocess_bwd), including the position gradient through view-dependent SH color for
sh_degree >= 1 - ∂L/∂local_offset for FLAME mesh-bound Gaussians (binding backward pass)
Gradient verification — 78 tests (see tests/gpu_gradient_verify.rs + tests/gpu_gradient_verify/sh.rs and tests/gradient_verification/): finite-difference parameter checks plus the harness's own NaN/empty-input guard tests, checked with a median-error metric robust to per-pixel outliers: ≤5e-2 median relative error for most parameters, relaxed to ≤2.5e-1 for position (the tiled rasterizer's forward pass has tile-boundary discontinuities that finite differences can't model, so position gradients need a wider tolerance). These tests skip themselves at runtime (not via #[ignore]) when no GPU adapter is available, so a headless run without a GPU still reports them as passing without actually exercising the shaders — a real GPU adapter was present and exercised when the numbers below were last measured.
Statistics
- Tests: 2,903
#[test]-attributed tests in source (src/+tests/), all passing as of this release:- 2,887 run by default:
cargo nextest run -p oxigaf-render --all-features - 16 more are
#[ignore]d because they need real GPU hardware (4deform, 5multi_view, 4cpu_gpu_compare, 1pipeline, 1sort, plus 1 slow 100-Gaussian position-gradient check); run them with--run-ignored ignored-only— all 16 pass when a GPU adapter is available - 23 doc-tests passing, 3
#[ignore]d - 78 of the above are in the gradient-verification suite (
gpu_gradient_verify.rs+tests/gpu_gradient_verify/sh.rsandtests/gradient_verification/) — see "Differentiability" above
- 2,887 run by default:
- Shaders (17
.wgslfiles):preprocessand its degree-specialized variantspreprocess_{sh0,sh1,sh2,sh3},preprocess_bwd,rasterize_{fwd,bwd},deform_gaussians,flame_binding_bwd,tile_assign,tile_ranges,prefix_sum{,_add},radix_{histogram,scatter},atomic_to_f32.cov2d_bwddocuments the 2D-covariance backward math but is not itself compiled/dispatched — the live GPU path is inline inpreprocess_bwd.
Run benchmarks with:
GPU Backend Selection
wgpu automatically selects the best available backend:
| Platform | Primary Backend | Fallback |
|---|---|---|
| Linux | Vulkan | — |
| Windows | Vulkan | DX12 |
| macOS | Metal | — |
| Web | WebGPU | WebGL 2 |
Documentation
License
Licensed under the Apache License, Version 2.0 (LICENSE)