๐ฆ Apex Solver
A high-performance Rust-based nonlinear least squares optimization library designed for computer vision applications including bundle adjustment, SLAM, and pose graph optimization. Built with focus on zero-cost abstractions, memory safety, and mathematical correctness.
Apex Solver is a comprehensive optimization library that bridges the gap between theoretical robotics and practical implementation. It provides manifold-aware optimization for Lie groups commonly used in computer vision, multiple optimization algorithms with unified interfaces, flexible linear algebra backends supporting both sparse Cholesky and QR decompositions, and industry-standard file format support for seamless integration with existing workflows.
Key Features (v1.2.1)
- ๐ท Bundle Adjustment with Camera Intrinsic Optimization: Simultaneous optimization of camera poses, 3D landmarks, and camera intrinsics (8 camera models via apex-camera-models crate) apex-camera-models
- ๐ง Explicit & Implicit Schur Complement Solvers: Memory-efficient matrix-free PCG for large-scale problems (10,000+ cameras) alongside traditional explicit formulation
- ๐ก๏ธ 15 Robust Loss Functions: Comprehensive outlier rejection (Huber, Cauchy, Tukey, Welsch, Barron, and more)
- ๐ Manifold-Aware: Full Lie group support (SE2, SE3, SO2, SO3) with analytic Jacobians apex-manifolds
- ๐ Three Optimization Algorithms: Levenberg-Marquardt, Gauss-Newton, and Dog Leg with unified interface
- ๐ Prior Factors & Fixed Variables: Anchor poses with known values and constrain specific parameter indices
- ๐ Uncertainty Quantification: Covariance estimation for both Cholesky and QR solvers
- ๐จ Real-time Visualization: Integrated Rerun support for live debugging of optimization progress
- ๐ I/O: Read and write G2O, Toro, BAL format files for seamless integration with SLAM ecosystems apex-io
- โก High Performance: Sparse linear algebra with persistent symbolic factorization
- โ Production-Grade: Comprehensive error handling, structured tracing, integration test suite
๐ Quick Start
use HashMap;
use Problem;
use ;
use ;
use ;
use dvector;
Result:
Status: CostToleranceReached
Initial cost: 1.280e+05
Final cost: 2.130e+01
Iterations: 5
๐๏ธ Architecture
The workspace root is the apex-solver crate. Sub-crates for manifolds, I/O, and camera models live in crates/:
apex-solver/ # workspace root = apex-solver crate
โโโ src/
โ โโโ core/ # Problem formulation, factors, residuals
โ โโโ factors/ # Factor implementations (projection, between, prior)
โ โโโ optimizer/ # LM, GN, Dog Leg algorithms
โ โโโ linalg/ # Cholesky, QR, Explicit/Implicit Schur
โ โโโ observers/ # Optimization observers and callbacks
โโโ bin/ # Executable binaries
โโโ benches/ # Benchmarks
โโโ examples/ # Example programs
โโโ tests/ # Integration tests
โโโ doc/ # Extended documentation
โโโ crates/
โโโ apex-manifolds/ # Lie groups: SE2, SE3, SO2, SO3, Rn
โโโ apex-io/ # File I/O: G2O, TORO, BAL formats
โโโ apex-camera-models/ # 8 camera projection models
Core Modules (in src/):
core/: Optimization problem definitions, residual blocks, robust loss functions, and variable managementoptimizer/: Three optimization algorithms (Levenberg-Marquardt with adaptive damping, Gauss-Newton, Dog Leg trust region) with real-time visualization supportlinalg/: Linear algebra backends including sparse Cholesky decomposition, QR factorization, explicit Schur complement, and implicit Schur complement (matrix-free PCG)observers/: Optimization observers and callbacks (Rerun visualization, custom hooks)
Workspace Sub-crates (in crates/):
apex-manifolds: Lie group implementations (SE2/SE3 for rigid transformations, SO2/SO3 for rotations, Rn for Euclidean space) with analytic Jacobiansapex-io: File format parsers for G2O, TORO, and BAL formatsapex-camera-models: Camera projection models with analytic Jacobians (8 models)
Low-level Dependencies:
faer/nalgebra: High-performance linear algebra backends
๐ Data Files (Git LFS)
The benchmark datasets in data/ are stored with Git LFS.
After cloning, pull the actual data files:
# Install Git LFS (once per machine)
# Download all data files
Without this step, the .g2o files will be empty LFS pointer stubs, causing
Graph Statistics: Vertices: 0, Edges: 0 errors when running the binaries.
Available datasets:
- SE2 (2D):
intel,M3500,mit,ring - SE3 (3D):
sphere2500,parking-garage,torus3D,cubicle
๐ฆ Workspace Crates
Apex Solver is organized as a Cargo workspace with specialized sub-crates that can be used independently:
| Crate | Description | Docs |
|---|---|---|
| apex-manifolds | Lie group manifolds (SE2, SE3, SO2, SO3, Rn) with analytic Jacobians | README |
| apex-camera-models | 8 camera projection models for bundle adjustment and SLAM | README |
| apex-io | File I/O utilities for G2O, TORO, and BAL formats | README |
Using sub-crates independently:
[]
= "0.1.0"
[]
= "0.1.0"
[]
= "0.1.0"
๐ Performance Benchmarks
Hardware: Apple Mac Mini M4, 64GB RAM
Methodology: Average over multiple runs
Pose Graph Optimization
Performance comparison across 6 optimization libraries on standard pose graph datasets. All benchmarks use Levenberg-Marquardt algorithm with consistent parameters (max_iterations=100, cost_tolerance=1e-4).
Metrics: Wall-clock time (ms), iterations, initial/final cost, convergence status
2D Datasets (SE2)
| Dataset | Solver | Lang | Time (ms) | Iters | Init Cost | Final Cost | Improve % | Conv |
|---|---|---|---|---|---|---|---|---|
| intel (1228 vertices, 1483 edges) | ||||||||
| apex-solver | Rust | 28.5 | 12 | 3.68e4 | 3.89e-1 | 100.00 | โ | |
| factrs | Rust | 2.9 | - | 3.68e4 | 8.65e3 | 76.47 | โ | |
| tiny-solver | Rust | 87.9 | - | 1.97e4 | 4.56e3 | 76.91 | โ | |
| Ceres | C++ | 9.0 | 13 | 3.68e4 | 2.34e2 | 99.36 | โ | |
| g2o | C++ | 74.0 | 100 | 3.68e4 | 3.15e0 | 99.99 | โ | |
| GTSAM | C++ | 39.0 | 11 | 3.68e4 | 3.89e-1 | 100.00 | โ | |
| mit (808 vertices, 827 edges) | ||||||||
| apex-solver | Rust | 140.7 | 107 | 1.63e5 | 1.10e2 | 99.93 | โ | |
| factrs | Rust | 3.5 | - | 1.63e5 | 1.48e4 | 90.91 | โ | |
| tiny-solver | Rust | 5.7 | - | 5.78e4 | 1.19e4 | 79.34 | โ | |
| Ceres | C++ | 11.0 | 29 | 1.63e5 | 3.49e2 | 99.79 | โ | |
| g2o | C++ | 46.0 | 100 | 1.63e5 | 1.26e3 | 99.23 | โ | |
| GTSAM | C++ | 39.0 | 4 | 1.63e5 | 8.33e4 | 48.94 | โ | |
| M3500 (3500 vertices, 5453 edges) | ||||||||
| apex-solver | Rust | 103.5 | 10 | 2.86e4 | 1.51e0 | 99.99 | โ | |
| factrs | Rust | 62.6 | - | 2.86e4 | 1.52e0 | 99.99 | โ | |
| tiny-solver | Rust | 200.1 | - | 3.65e4 | 2.86e4 | 21.67 | โ | |
| Ceres | C++ | 77.0 | 18 | 2.86e4 | 4.54e3 | 84.14 | โ | |
| g2o | C++ | 108.0 | 33 | 2.86e4 | 1.51e0 | 99.99 | โ | |
| GTSAM | C++ | 67.0 | 6 | 2.86e4 | 1.51e0 | 99.99 | โ | |
| ring (434 vertices, 459 edges) | ||||||||
| apex-solver | Rust | 8.5 | 10 | 1.02e4 | 2.22e-2 | 100.00 | โ | |
| factrs | Rust | 4.8 | - | 1.02e4 | 3.02e-2 | 100.00 | โ | |
| tiny-solver | Rust | 21.0 | - | 3.17e3 | 9.87e2 | 68.81 | โ | |
| Ceres | C++ | 3.0 | 14 | 1.02e4 | 2.22e-2 | 100.00 | โ | |
| g2o | C++ | 6.0 | 34 | 1.02e4 | 2.22e-2 | 100.00 | โ | |
| GTSAM | C++ | 10.0 | 6 | 1.02e4 | 2.22e-2 | 100.00 | โ |
3D Datasets (SE3)
| Dataset | Solver | Lang | Time (ms) | Iters | Init Cost | Final Cost | Improve % | Conv |
|---|---|---|---|---|---|---|---|---|
| sphere2500 (2500 vertices, 4949 edges) | ||||||||
| apex-solver | Rust | 176.3 | 5 | 1.28e5 | 2.13e1 | 99.98 | โ | |
| factrs | Rust | 334.8 | - | 1.28e5 | 3.49e1 | 99.97 | โ | |
| tiny-solver | Rust | 2020.3 | - | 4.08e4 | 4.06e4 | 0.48 | โ | |
| Ceres | C++ | 1447.0 | 101 | 8.26e7 | 8.25e5 | 99.00 | โ | |
| g2o | C++ | 10919.0 | 84 | 8.26e7 | 3.89e3 | 100.00 | โ | |
| GTSAM | C++ | 138.0 | 7 | 8.26e7 | 1.01e4 | 99.99 | โ | |
| parking-garage (1661 vertices, 6275 edges) | ||||||||
| apex-solver | Rust | 153.1 | 6 | 8.36e3 | 6.24e-1 | 99.99 | โ | |
| factrs | Rust | 453.1 | - | 8.36e3 | 6.28e-1 | 99.99 | โ | |
| tiny-solver | Rust | 849.2 | - | 1.21e5 | 1.21e5 | -0.05 | โ | |
| Ceres | C++ | 344.0 | 36 | 1.22e8 | 4.84e5 | 99.60 | โ | |
| g2o | C++ | 635.0 | 56 | 1.22e8 | 2.82e6 | 97.70 | โ | |
| GTSAM | C++ | 31.0 | 3 | 1.22e8 | 4.79e6 | 96.08 | โ | |
| torus3D (5000 vertices, 9048 edges) | ||||||||
| apex-solver | Rust | 1780.5 | 27 | 1.91e4 | 1.20e2 | 99.37 | โ | |
| factrs | Rust | - | - | - | - | - | โ | |
| tiny-solver | Rust | - | - | - | - | - | โ | |
| Ceres | C++ | 1063.0 | 34 | 2.30e5 | 3.85e4 | 83.25 | โ | |
| g2o | C++ | 31279.0 | 96 | 2.30e5 | 1.52e5 | 34.04 | โ | |
| GTSAM | C++ | 647.0 | 12 | 2.30e5 | 3.10e5 | -34.88 | โ | |
| cubicle (5750 vertices, 16869 edges) | ||||||||
| apex-solver | Rust | 512.0 | 5 | 3.19e4 | 5.38e0 | 99.98 | โ | |
| factrs | Rust | - | - | - | - | - | โ | |
| tiny-solver | Rust | 1975.8 | - | 1.14e4 | 9.92e3 | 12.62 | โ | |
| Ceres | C++ | 1457.0 | 36 | 8.41e6 | 1.95e4 | 99.77 | โ | |
| g2o | C++ | 8533.0 | 47 | 8.41e6 | 2.17e5 | 97.42 | โ | |
| GTSAM | C++ | 558.0 | 5 | 8.41e6 | 7.52e5 | 91.05 | โ |
Key Observations:
- apex-solver: 100% convergence rate (8/8 datasets), most reliable Rust solver
- Ceres/g2o: 100% convergence but often slower (especially g2o)
- GTSAM: Fast when it converges, but diverged on torus3D (87.5% rate)
- factrs: Fast on 2D but panics on large 3D problems (62.5% rate)
- tiny-solver: Convergence issues on several datasets (75% rate)
Bundle Adjustment (Self-Calibration)
Large-scale bundle adjustment benchmarks optimizing camera poses, 3D landmarks, and camera intrinsics simultaneously. Tests self-calibration capability on real-world structure-from-motion datasets from the Bundle Adjustment in the Large (BAL) collection.
| Dataset | Solver | Lang | Cameras | Landmarks | Observations | Init RMSE | Final RMSE | Time (s) | Iters | Status |
|---|---|---|---|---|---|---|---|---|---|---|
| Dubrovnik | ||||||||||
| Apex-Iterative | Rust | 356 | 226,730 | 1,255,268 | 2.043 | 0.533 | 47.16 | 9 | โ | |
| Ceres | C++ | 356 | 226,730 | 1,255,268 | 12.975 | 1.004 | 2879.23 | 101 | โ | |
| GTSAM | C++ | 356 | 226,730 | 1,255,268 | 2.812 | 0.562 | 196.72 | 31 | โ | |
| g2o | C++ | 356 | 226,730 | 1,255,268 | 12.975 | 12.168 | 34.67 | 20 | โ | |
| Ladybug | ||||||||||
| Apex-Iterative | Rust | 1,723 | 156,502 | 678,718 | 1.382 | 0.537 | 146.69 | 30 | โ | |
| Ceres | C++ | 1,723 | 156,502 | 678,718 | 13.518 | 1.168 | 17.53 | 101 | โ | |
| GTSAM | C++ | 1,723 | 156,502 | 678,718 | 1.857 | 0.981 | 95.46 | 2 | โ | |
| g2o | C++ | 1,723 | 156,502 | 678,718 | 13.518 | 13.507 | 150.46 | 20 | โ | |
| Trafalgar | ||||||||||
| Apex-Iterative | Rust | 257 | 65,132 | 225,911 | 2.033 | 0.679 | 10.39 | 14 | โ | |
| Ceres | C++ | 257 | 65,132 | 225,911 | 14.753 | 1.320 | 44.14 | 101 | โ | |
| GTSAM | C++ | 257 | 65,132 | 225,911 | 2.798 | 0.626 | 77.64 | 100 | โ | |
| g2o | C++ | 257 | 65,132 | 225,911 | 14.753 | 8.151 | 16.11 | 20 | โ | |
| Venice (Largest) | ||||||||||
| Apex-Iterative | Rust | 1,778 | 993,923 | 5,001,946 | 1.676 | 0.458 | 83.17 | 2 | โ | |
| Ceres | C++ | 1,778 | 993,923 | 5,001,946 | - | - | TIMEOUT | - | โ | |
| GTSAM | C++ | 1,778 | 993,923 | 5,001,946 | - | - | TIMEOUT | - | โ | |
| g2o | C++ | 1,778 | 993,923 | 5,001,946 | 10.128 | 10.126 | 252.17 | 20 | โ |
Key Results:
- Apex-Iterative: 100% convergence rate (4/4 datasets), handles up to 5M observations efficiently
- Superior scalability: Only solver alongside g2o to complete Venice dataset; Ceres and GTSAM timeout after 10 minutes
- Best accuracy on largest dataset: Achieves 0.458 RMSE on Venice (5M observations) in only 2 iterations
- Speed advantage: 61x faster than Ceres on Dubrovnik, 4x faster on Trafalgar (where Ceres converged)
๐ Examples
Example 1: Basic Pose Graph Optimization
use HashMap;
use Problem;
use BetweenFactor;
use ;
use ;
use dvector;
// Load pose graph
let graph = load?;
// Build optimization problem
let mut problem = new;
let mut initial_values = new;
// Add SE3 poses as variables
for in &graph.vertices_se3
// Add between factors
for edge in &graph.edges_se3
// Configure and solve
let config = new
.with_max_iterations
.with_cost_tolerance;
let mut solver = with_config;
let result = solver.optimize?;
println!;
Example 2: Custom Factor Implementation
Create custom factors by implementing the Factor trait:
use Factor;
use ;
// Use in optimization
problem.add_residual_block;
Example 3: Self-Calibration Bundle Adjustment
Optimize camera poses, 3D landmarks, AND camera intrinsics simultaneously. See the apex-camera-models crate for detailed camera model documentation.
use HashMap;
use Problem;
use ProjectionFactor;
use HuberLoss;
use ;
// Use any camera model from apex-camera-models crate
Optimization Types (compile-time configuration):
SelfCalibration: Optimize pose + landmarks + intrinsicsBundleAdjustment: Optimize pose + landmarks (fixed intrinsics)OnlyPose: Visual odometry (fixed landmarks and intrinsics)OnlyLandmarks: Triangulation (known poses)OnlyIntrinsics: Camera calibration (known structure)
See apex-camera-models documentation for complete camera model reference and advanced examples.
๐งฎ Technical Implementation
Manifold Operations
Apex Solver implements mathematically rigorous Lie group operations following the manif library conventions. All manifold types provide plus() (retraction), minus() (inverse retraction), compose() (group composition), inverse() (group inverse), and analytic Jacobians for efficient optimization.
Supported Manifolds:
| Manifold | Description | DOF | Use Case |
|---|---|---|---|
| SE(3) | 3D rigid transformations | 6 | 3D SLAM, visual odometry |
| SO(3) | 3D rotations | 3 | Orientation tracking |
| SE(2) | 2D rigid transformations | 3 | 2D SLAM, mobile robots |
| SO(2) | 2D rotations | 1 | 2D orientation |
| R^n | Euclidean space | n | Landmarks, camera parameters |
See apex-manifolds documentation for detailed API reference, mathematical background, and usage examples.
Robust Loss Functions
15 robust loss functions for handling outliers in optimization:
- L2Loss: Standard least squares (no outliers)
- L1Loss: Linear growth (light outliers)
- HuberLoss: Quadratic near zero, linear after threshold (moderate outliers)
- CauchyLoss: Logarithmic growth (heavy outliers)
- FairLoss, GemanMcClureLoss, WelschLoss, TukeyBiweightLoss, AndrewsWaveLoss: Various robustness profiles
- RamsayEaLoss: Asymmetric outliers
- TrimmedMeanLoss: Ignores worst residuals
- LpNormLoss: Generalized Lp norm
- BarronGeneralLoss, AdaptiveBarronLoss: Adaptive robustness
- TDistributionLoss: Statistical outliers
Usage:
use HuberLoss;
let loss = new; // 95% efficiency threshold
problem.add_residual_block;
Optimization Algorithms
Levenberg-Marquardt (Recommended)
- Adaptive damping between gradient descent and Gauss-Newton
- Robust convergence from poor initial estimates
- Supports covariance estimation for uncertainty quantification
- 9 comprehensive termination criteria (gradient norm, cost change, trust region radius, etc.)
Gauss-Newton
- Fast convergence near solution
- Minimal memory requirements
- Best for well-initialized problems
Dog Leg Trust Region
- Combines steepest descent and Gauss-Newton
- Global convergence guarantees
- Adaptive trust region management
Linear Algebra Backends
Four sparse linear solvers for different use cases:
- Sparse Cholesky: Direct factorization of J^T J + ฮปI - fast, moderate memory, best for well-conditioned problems
- Sparse QR: QR factorization of Jacobian - robust for rank-deficient systems, slightly slower
- Explicit Schur Complement: Constructs reduced camera matrix S = B - E Cโปยน Eแต explicitly in memory - most accurate for bundle adjustment, moderate memory usage
- Implicit Schur Complement: Matrix-free PCG solver computing only Sยทx products - memory-efficient for large-scale problems (10,000+ cameras), highly scalable
Configure via LinearSolverType in optimizer config:
config.with_linear_solver_type // For bundle adjustment
config.with_linear_solver_type // For very large BA
๐จ Interactive Visualization
Real-time optimization debugging with integrated Rerun visualization using the observer pattern:
use ;
let config = new
.with_max_iterations;
let mut solver = with_config;
// Add Rerun visualization observer (requires `visualization` feature)
let result = solver.optimize?;
Visualized Metrics:
- Time series: Cost, gradient norm, damping (ฮป), step quality (ฯ), step norm
- Matrix visualizations: Hessian heat map, gradient vector
- 3D poses: SE3 camera frusta, SE2 2D points
Run Examples:
# Enable visualization feature and run
Note: The data files (e.g.,
sphere2500.g2o) need to be pulled using Git LFS. See ๐ Data Files (Git LFS) for instructions.
Zero overhead when disabled (feature-gated).
๐ง Learning Resources
Computer Vision Background
- Multiple View Geometry (Hartley & Zisserman) - Mathematical foundations
- Visual SLAM algorithms (Durrant-Whyte & Bailey) - Probabilistic robotics
- g2o documentation - Reference C++ implementation
Lie Group Theory
- A micro Lie theory (Solร et al.) - Practical introduction
- manif library - C++ reference we follow
- State Estimation for Robotics (Barfoot) - SO(3) and SE(3)
Optimization Theory
- Numerical Optimization (Nocedal & Wright) - Standard reference
- Trust Region Methods - Dog Leg theory
- Ceres Solver Tutorial - Practical guide
๐ Acknowledgments
Apex Solver draws inspiration and reference implementations from:
- Ceres Solver - Google's C++ optimization library
- g2o - General framework for graph optimization
- GTSAM - Georgia Tech Smoothing and Mapping library
- tiny-solver - Lightweight nonlinear least squares solver
- factrs - Rust factor graph optimization library
- faer - High-performance linear algebra library for Rust
- manif - C++ Lie theory library (for manifold conventions)
- nalgebra - Geometry and linear algebra primitives
๐ License
Licensed under the Apache License, Version 2.0. See LICENSE for details.