Skip to main content

Crate pixelmap_multiview

Crate pixelmap_multiview 

Source
Expand description

Multi-view 3D reconstruction on top of pixelmap.

Given three or more photos of a static scene taken with one camera, this crate computes dense correspondence between every pair of them with pixelmap, recovers where each photo was taken from, and fuses the result into a single mesh. pixelmap_model_3d lifts one mapping between two photos into a surface; this crate registers every view into one frame so that their surfaces agree.

§Quick start

Hand over the photos, say how much effort to spend, and follow along:

use std::sync::Arc;
use pixelmap::{Photo, Quality};
use pixelmap_multiview::{Event, Flow, Focal, Options};

let photos = photos();
let options = Options::new()
    .quality(Quality::Medium)
    .focal(Focal::Equivalent35mm(28.0));

let model = pixelmap_multiview::run(&photos, &options, &mut |event: Event| {
    println!("{}: {}", event.stage(), event.message());
    Flow::Continue(())
})?;

println!("{} triangles", model.mesh.triangles.len());

With the threads feature, Job runs that on a worker thread and hands back a channel of events, a Status to poll, and a way to stop it.

§Stages

In order: pairwise correspondence with pixelmap, the relative pose of each pair, points followed across views, every camera placed in one frame, bundle adjustment, a dense depth map per view, fusion into one Mesh, and colour from the photos. Stage names them; every Event says which one it came from.

§Feedback while it runs

One pixelmap solve per pair of photos is the bulk of the work, so a run is minutes long for a handful of photos and the caller hears about it throughout. Event carries data rather than prose — which pair is being mapped, how far through it, why one was rejected — so that showing progress never means parsing a sentence. The event that finishes a pair carries that pair’s PairMap, the raw correspondence grid, for a caller that wants to show the match as it is found.

Returning Flow::Break stops the run, which then returns Error::Cancelled. A stop takes effect at the next checkpoint: within pairwise correspondence that is one pixelmap schedule step, elsewhere it is prompt.

§Design rules

  • No I/O, no platform. Photos come in as RGBA buffers and a mesh goes out. Decoding, EXIF and resizing are the caller’s job, which keeps the crate buildable for wasm32-unknown-unknown.
  • One type per coordinate space. See PhotoPx, Norm, World.
  • Deterministic. Every random choice draws from a generator derived from a single seed, as pixelmap’s own solver does, so any run can be replayed exactly.
  • Fail loudly. Each stage has exit criteria. A run that cannot produce a correct model returns an Error naming the stage and the reason, rather than a plausible-looking wrong model.

§Dependencies in the API

Positions, normals and poses are nalgebra types, and photos, quality and correspondences are pixelmap types. Both crates are re-exported, so pixelmap_multiview::nalgebra and pixelmap_multiview::pixelmap are always the versions this crate was built against. A new major version of either is a breaking change here.

§Feature flags

  • parallel (default) — passed through to pixelmap. Turn it off for wasm.
  • threads (default) — Job, which runs a reconstruction on a worker thread. Turn it off for wasm, which has no threads to hand out; run works there and is what Job drives underneath.

Re-exports§

pub use pipeline::reconstruct;
pub use pipeline::run;
pub use pipeline::Diagnostics;
pub use pipeline::Model;
pub use pipeline::PairReport;
pub use job::Cancel;threads
pub use job::Job;threads
pub use nalgebra;
pub use pixelmap;

Modules§

export
Writing results out for inspection in other tools.
jobthreads
Running a reconstruction on a worker thread.
pipeline
The whole reconstruction, from photos to a textured mesh, in one call.

Structs§

AdjustmentReport
What bundle adjustment did.
DepthCounts
What became of a group of depth samples.
DepthMap
Depths on a regular grid over one view’s photo.
DepthStats
Where one view’s depth samples went, split by whether a sample mapped into one other view or into several.
Intrinsics
A pinhole camera with square pixels and no skew.
Mesh
Triangles over shared vertices, in the reconstruction’s frame.
Norm
Normalized camera coordinates: a pixel with K⁻¹ applied, so that (x, y, 1) is the direction of its ray in the camera’s own frame.
Options
Settings for a reconstruction.
PairGraph
One mapping per pair of views.
PairId
An unordered pair of distinct views, stored with the lower id first.
PairMap
The dense correspondence found between one pair of photos.
PhotoPx
A position in one of the photos handed to the pipeline, in pixels.
Pose
Where a camera is and which way it faces, stored as the transform from world coordinates into the camera’s frame: x_cam = R · x_world + t.
Registration
How a view was added by PnP.
SparseModel
Cameras in one frame, and the points triangulated from them.
SparsePoint
A triangulated track.
Status
Where a run has got to, for a caller that would rather ask than follow every event.
Texture
Colour for a mesh.
Track
One scene point, observed in two or more views.
ViewId
The index of a photo in the input set.
World
A point in the reconstruction’s frame.

Enums§

Degeneracy
Why a pair cannot anchor a registration.
DropReason
Why a photo was left out of a reconstruction.
Error
Why a reconstruction could not continue.
Event
Something that happened while reconstructing.
Focal
What is known about the focal length the photos were taken at.
FocalSource
Where a focal length came from.
Level
How much a message matters.
RegistrationWarning
Something worth telling the user that did not stop the reconstruction.
SampleFate
What became of one depth sample.
Stage
The pipeline’s stages, in the order they run.

Constants§

DEFAULT_MAX_TEXTURE_SIZE
The largest texture atlas a reconstruction produces by default, in pixels on a side.
MIN_PAIR_COVERAGE
The coverage below which a pair is not trusted to connect its two views.
MIN_VIEWS
The fewest photos a reconstruction accepts.

Traits§

PairLookup
A dense mapping between the two photos of a pair, in PhotoPx coordinates.

Type Aliases§

Flow
What a progress callback returns. ControlFlow::Break(()) asks the pipeline to stop, and the run then returns crate::Error::Cancelled.