Expand description
Multi-view 3D reconstruction on top of pixelmap.
Given three or more photos of a static scene taken with one camera, this crate computes
dense correspondence between every pair of them with pixelmap, recovers where each
photo was taken from, and fuses the result into a single mesh. pixelmap_model_3d
lifts one mapping between two photos into a surface; this crate registers every view
into one frame so that their surfaces agree.
§Quick start
Hand over the photos, say how much effort to spend, and follow along:
use std::sync::Arc;
use pixelmap::{Photo, Quality};
use pixelmap_multiview::{Event, Flow, Focal, Options};
let photos = photos();
let options = Options::new()
.quality(Quality::Medium)
.focal(Focal::Equivalent35mm(28.0));
let model = pixelmap_multiview::run(&photos, &options, &mut |event: Event| {
println!("{}: {}", event.stage(), event.message());
Flow::Continue(())
})?;
println!("{} triangles", model.mesh.triangles.len());With the threads feature, Job runs that on a worker thread and hands back a
channel of events, a Status to poll, and a way to stop it.
§Stages
In order: pairwise correspondence with pixelmap, the relative pose of each pair, points
followed across views, every camera placed in one frame, bundle adjustment, a dense
depth map per view, fusion into one Mesh, and colour from the photos. Stage
names them; every Event says which one it came from.
§Feedback while it runs
One pixelmap solve per pair of photos is the bulk of the work, so a run is minutes long
for a handful of photos and the caller hears about it throughout. Event carries
data rather than prose — which pair is being mapped, how far through it, why one was
rejected — so that showing progress never means parsing a sentence. The event that
finishes a pair carries that pair’s PairMap, the raw correspondence grid, for a
caller that wants to show the match as it is found.
Returning Flow::Break stops the run, which then
returns Error::Cancelled. A stop takes effect at the next checkpoint: within
pairwise correspondence that is one pixelmap schedule step, elsewhere it is prompt.
§Design rules
- No I/O, no platform. Photos come in as RGBA buffers and a mesh goes out.
Decoding, EXIF and resizing are the caller’s job, which keeps the crate buildable for
wasm32-unknown-unknown. - One type per coordinate space. See
PhotoPx,Norm,World. - Deterministic. Every random choice draws from a generator derived from a single seed, as pixelmap’s own solver does, so any run can be replayed exactly.
- Fail loudly. Each stage has exit criteria. A run that cannot produce a correct
model returns an
Errornaming the stage and the reason, rather than a plausible-looking wrong model.
§Dependencies in the API
Positions, normals and poses are nalgebra types, and photos, quality and
correspondences are pixelmap types. Both crates are re-exported, so
pixelmap_multiview::nalgebra and pixelmap_multiview::pixelmap are always the versions
this crate was built against. A new major version of either is a breaking change here.
§Feature flags
parallel(default) — passed through to pixelmap. Turn it off for wasm.threads(default) —Job, which runs a reconstruction on a worker thread. Turn it off for wasm, which has no threads to hand out;runworks there and is whatJobdrives underneath.
Re-exports§
pub use pipeline::reconstruct;pub use pipeline::run;pub use pipeline::Diagnostics;pub use pipeline::Model;pub use pipeline::PairReport;pub use job::Cancel;threadspub use job::Job;threadspub use nalgebra;pub use pixelmap;
Modules§
- export
- Writing results out for inspection in other tools.
- job
threads - Running a reconstruction on a worker thread.
- pipeline
- The whole reconstruction, from photos to a textured mesh, in one call.
Structs§
- Adjustment
Report - What bundle adjustment did.
- Depth
Counts - What became of a group of depth samples.
- Depth
Map - Depths on a regular grid over one view’s photo.
- Depth
Stats - Where one view’s depth samples went, split by whether a sample mapped into one other view or into several.
- Intrinsics
- A pinhole camera with square pixels and no skew.
- Mesh
- Triangles over shared vertices, in the reconstruction’s frame.
- Norm
- Normalized camera coordinates: a pixel with K⁻¹ applied, so that
(x, y, 1)is the direction of its ray in the camera’s own frame. - Options
- Settings for a reconstruction.
- Pair
Graph - One mapping per pair of views.
- PairId
- An unordered pair of distinct views, stored with the lower id first.
- PairMap
- The dense correspondence found between one pair of photos.
- PhotoPx
- A position in one of the photos handed to the pipeline, in pixels.
- Pose
- Where a camera is and which way it faces, stored as the transform from world
coordinates into the camera’s frame:
x_cam = R · x_world + t. - Registration
- How a view was added by PnP.
- Sparse
Model - Cameras in one frame, and the points triangulated from them.
- Sparse
Point - A triangulated track.
- Status
- Where a run has got to, for a caller that would rather ask than follow every event.
- Texture
- Colour for a mesh.
- Track
- One scene point, observed in two or more views.
- ViewId
- The index of a photo in the input set.
- World
- A point in the reconstruction’s frame.
Enums§
- Degeneracy
- Why a pair cannot anchor a registration.
- Drop
Reason - Why a photo was left out of a reconstruction.
- Error
- Why a reconstruction could not continue.
- Event
- Something that happened while reconstructing.
- Focal
- What is known about the focal length the photos were taken at.
- Focal
Source - Where a focal length came from.
- Level
- How much a message matters.
- Registration
Warning - Something worth telling the user that did not stop the reconstruction.
- Sample
Fate - What became of one depth sample.
- Stage
- The pipeline’s stages, in the order they run.
Constants§
- DEFAULT_
MAX_ TEXTURE_ SIZE - The largest texture atlas a reconstruction produces by default, in pixels on a side.
- MIN_
PAIR_ COVERAGE - The coverage below which a pair is not trusted to connect its two views.
- MIN_
VIEWS - The fewest photos a reconstruction accepts.
Traits§
- Pair
Lookup - A dense mapping between the two photos of a pair, in
PhotoPxcoordinates.
Type Aliases§
- Flow
- What a progress callback returns.
ControlFlow::Break(())asks the pipeline to stop, and the run then returnscrate::Error::Cancelled.