Expand description
One reconstruction run as a state machine over the kernels.
The engine does no I/O and reads no clock. A host drives it one step at a time: each pass (scan, solve, render, reveal, framing, pyramid) is a session that takes one frame (plus the rows the host stored for that frame earlier) and returns what that step produced — rows to write or delete, structured diagnostics, how far it got. The host stores the rows, formats and shows the diagnostics, and checks for pause and stop between steps. The web page’s TypeScript loop is one such host (through WebAssembly); the driver at the crate root is another. See docs/ARCHITECTURE.md §十三 for the boundary.
Diagnostics carry a code and parameters, never text. catalog (feature catalog) turns them into the same zh
or en sentence the web page shows, from the catalogues exported out of src/i18n; the web page’s WebAssembly
formats them with its own i18n and leaves the feature off.
This is an implementation layer, used by the driver and the web page’s WebAssembly; most users want the driver at the crate root, which drives every pass in order. Its API follows what those hosts need and may change in minor versions.
Run the scan pass by hand over a few frames of a page scrolling up by 6 pixels a frame:
use long_screen::engine::scan::{FrameMeta, ScanSession};
let (width, height) = (320, 240);
let page = |x: usize, y: usize| ((x * 37 + y * 101 + (x * y) % 23) % 251) as u8;
let mut scan = ScanSession::new(width, height, 1);
for index in 0..4u32 {
let offset = index as usize * 6;
for (i, pixel) in scan.frame_buffer().chunks_exact_mut(4).enumerate() {
let v = page(i % width, i / width + offset);
pixel.copy_from_slice(&[v, v, v, 255]);
}
let step = scan
.push(FrameMeta { index, time: index as f64 / 30.0, duration: 1.0 / 30.0 }, None)
.expect("a frame was written");
for diagnostic in &step.diagnostics {
println!("frame {index}: {}", diagnostic.code);
}
}
let regions = scan.finish(&[]);
println!("{} region(s)", regions.regions.len());Modules§
- attachments
- Fragment attachments: a fragment canvas that later evidence tied rigidly onto another canvas. The solve pass
records them (
attach/<id>rows) and follows the chains to compare a keyframe’s raw canvas and pose with the canvas it is observed at today; the render pass walks the same chains. - catalog
- The zh and en diagnostic and progress messages, and the formatter that fills in their
{{name}}placeholders; also the display names of regions and canvases (display_name()). - compositor
- Compositing one placed region into the tile cache: the per-tile paint, temporal-conflict resolution against the per-canvas temporal index, and the diagnostics both raise (docs/ARCHITECTURE.md §七, §十 “合成与瞬态像素”).
- diagnostic
- Structured diagnostics: what a pass found, as a code plus numbers, never text. The host formats the message from
its catalogue (the page’s
t(), orcrate::engine::catalog::formatnatively) underdiag.<code>.*with these parameters, so the zh text it persists stays byte-identical to what the page wrote before the pass moved here. A diagnostic may also name the canvas it concerns, a confidence and a rectangle, and carry adetailpayload the journal row stores as it is. - framing
- The framed presentation canvas of one reconstructed canvas (docs/ARCHITECTURE.md §十 “呈现画布”): the reference frame’s native chrome around the reconstructed content, with flat background bands where the content is taller or wider than the pane it was seen through. No toolbar is translated into world coordinates, no sidebar icon is stretched or repeated, and background extensions never count as observed evidence.
- keyframes
- The solve pass’s keyframe index: which earlier keyframes a frame might be revisiting, by visual words.
- layers
- Region learning over a whole recording: the core’s per-frame evidence accumulators
(
crate::kernels::layers::Learner), the last informative native frame kept for appearance evidence, and the one-shot region construction at the end — the automatic layout (crate::kernels::regions::finish) or a hand-placed one (manual regions, plus the “unassigned” region when they leave pixels uncovered). - pose_
graph - The solve pass’s pose graph as a state object: keyframe nodes, odometry and loop edges, the loop-closure error spread along a chain, the resumable relaxation, and the render pass’s per-frame correction.
- pyramid
- The preview pyramid of one canvas (docs/ARCHITECTURE.md §八): each level halves the one below it, four child tiles per parent tile, alpha-weighted so unobserved pixels never darken observed ones. Level 0 stays at source resolution and is never rewritten.
- render
- The render pass (docs/ARCHITECTURE.md §七): every solved placement of every frame resolved to its final canvas
and pose, checked against the frames either side of it, and painted into the tile cache by the
Compositor. - reveal
- The reveal stage (docs/ARCHITECTURE.md §七 “揭示”, §十 “揭示”): after the compositor has written every canvas, four
more frame-major passes over the recording replace what overlays burned in — a composer bar, a floating button, a
scrollbar thumb, a cursor, a fade veil — with the cleanest observation of the same page content, and give content
that really changed over time a single moment. The algorithm is
crate::kernels::reveal;RevealSessionis the run around it: which layers and canvases take part, which decisions of the solve count, the memory check that skips the stage, feeding the compositor’s tiles in, the order of the passes, and copying the chosen 16×16 blocks into the tiles. - scan
- The scan pass (pass 1): per-frame motion evidence and region learning.
- solve
- The solve pass as a session the host drives frame by frame: per-region tracking (odometry, anchor re-acquisition, drift control, relocalization, fragments), keyframes and revisits, fragment attachment, the pose graph’s nodes and edges, displacement-spread consistency voting, and each frame’s placements.
- tiles
- The tile cache: every canvas tile the render, reveal and presentation passes paint, held in engine memory under a working-memory budget with least-recently-used eviction (docs/ARCHITECTURE.md §十 “瓦片缓存”).