lumen-format 1.0.0

Reference encoder, decoder and validator for the LUMEN resin print file format (v1.0)
Documentation

lumen-format

A reference implementation of the LUMEN print file format, version 1.0, in Rust: encoder, decoder, and validator.

LUMEN is the Open Resin Alliance's resin (MSLA) print format. Its normative specification lives beside this crate in the LumenFormat repository, under spec/, with the conformance corpus under test-vectors/.

[dependencies]

lumen-format = "1.0"

The package is lumen-format and the library target is lumen, so code reads use lumen::….

Why this crate exists

LUMEN has more than one consumer - the DragonFruit slicer writes it, the Odyssey firmware prints it - and every divergence between their implementations is a compatibility bug that only shows up on a printer. Three readings of a specification are three chances to disagree about REE canonicalization, dictionary-ID agreement, or the per-block AAD that binds a sealed frame to its chunk and index. So the format has one implementation, and each consumer is an adapter over it:

  • DragonFruit maps its own settings into META/PROF and hands this crate layer masks.
  • Odyssey implements its PrintFile trait over [LumenFile], whose layer is already the "give me layer i" call that trait wants.

What stays in the consumer is what is genuinely theirs: the mapping from their own model into META, and hardware-side concerns the specification deliberately excludes (calibration, firmware-side mirroring, §1.1).

Reading

use lumen::{reader::LumenFile, validate::Level};

let bytes = std::fs::read("print.lumen")?;
let file = LumenFile::open(&bytes, Level::Loose)?;

println!("{} layers of {}x{}", file.layer_count(),
         file.head().display_width_px, file.head().display_height_px);
let meta = file.meta();
println!("normal exposure {} ms", meta.timing.normal_exposure_ms.unwrap_or(0));

let layer = file.layer(0)?;            // decompresses one block, not the file
println!("{:?} {}", layer.tag, layer.pixels.len());

// Timing, with the bottom/transition blend and LROV overrides already applied.
let timing = file.timing_for(0, 0)?;
println!("layer 0: {} ms at PWM {}", timing.exposure_ms, timing.light_pwm);
# Ok::<(), lumen::Error>(())

layer decompresses only the block that holds the layer it was asked for, so a firmware reader streams a 2,000-layer print through a bounded buffer instead of materializing it. See [LumenFile::block] for the block-level interface and [LumenFile::verify_layer] for bounded integrity verification.

On size. [LumenFile] borrows the container's bytes. Opening validates, which walks the layer data once - one block at a time, per section 4.10's design note for memory-constrained readers - and after that each [LumenFile::layer] call touches only the block that holds it. The peak is therefore one block rather than the file. Measured on a 7.3 MB print of eight 1 MB blocks (tests/memory.rs, which asserts the bound because it is easy to lose and invisible in a small test): decompressing one block peaks at 2.1 MB and a full validation at 2.2 MB, against 8.3 MB of decompressed layer stream.

For a print too large to keep resident, memory-map it and hand over the slice: because the directory sits at the end and LTBL carries per-layer ranges, only the blocks a caller asks for are faulted in, and the pages the validation pass already walked are evictable. Reading from an arbitrary Read + Seek source would need a different API here; the format already permits it, so in that case the interface is the limitation rather than the container.

Encrypted files are opened with a key, a password, or a recipient private key:

# use lumen::reader::LumenFile;
# use lumen::validate::Level;
# let bytes: Vec<u8> = Vec::new();
let file = LumenFile::open_with_password(&bytes, Level::Loose, "hunter2")?;
# Ok::<(), lumen::Error>(())

Writing

use lumen::chunks::head::Head;
use lumen::json::Meta;
use lumen::writer::Encoder;
# let (head, meta): (Head, Meta) = todo!();
# let layers: Vec<Vec<u8>> = Vec::new();

let mut encoder = Encoder::new(head, meta);
encoder.set_layers_per_chunk(64);
encoder.set_zstd_level(6);
for mask in &layers {
    encoder.push_layer(mask)?;
}
let bytes = encoder.finish()?;
# Ok::<(), lumen::Error>(())

A layer's tag is the encoder's choice (§5.7). By default the writer encodes each layer group both ways - with the tag EncodeMode::Auto picks and with EncodeMode::Attached (tag 0x03) - compresses both exactly as the file will frame them, and keeps the smaller; Encoder::set_tag_probe(false) turns that off, and Encoder::push_layer_with_mode states a tag instead of choosing one. The comparison is what makes the attached encoding safe to have on by default: on a print whose layers barely change its raw bytes are the smaller ones and its compressed bytes are 23% larger, so an encoder that picked by byte count would write the bigger file. A layer that arrives already encoded (push_encoded_layer) carries one encoding, so it is stored as it is.

The encoder is deterministic: identical input and settings produce identical bytes, which is what makes re-slicing a scene produce the same file and the same LHAS hashes (§5.7). Framing each LAYR chunk and hashing each layer are per-item passes, so finish runs them on worker threads - one per core unless Encoder::set_worker_threads says otherwise - and that count is not one of the settings: it moves the wall clock and not a byte. Encrypted output is of course not deterministic - it draws a fresh session key, salts and nonces from the OS CSPRNG, as §9.4 requires.

Validating

use lumen::validate::{validate, Level};

# let bytes: Vec<u8> = Vec::new();
validate(&bytes, Level::Strict)?;      // first failure, or Ok
# Ok::<(), lumen::Error>(())

A failure carries the check that failed, named exactly as the corpus names it:

# use lumen::validate::{validate, Level};
# let bytes: Vec<u8> = Vec::new();
if let Err(error) = validate(&bytes, Level::Strict) {
    eprintln!("{}: {}", error.check_name(), error.detail());  // e.g. "ree.run_lengths: ..."
}

Level::Loose is for printing: structurally valid files are accepted, unknown chunks and fields are skipped, and non-canonical encodings are tolerated. Level::Strict is for verification tools: every check in §11 runs, including the strict-only ones, and a loop count or host could be pinned by them. Validation returns the first failure in the specification's own order, which is what the corpus's expected_failure contract asserts.

Conformance

The corpus in test-vectors/ is the contract this crate is written against, and tests/conformance.rs checks three separate claims:

  1. Validator agreement. Every valid vector passes at both levels; every invalid vector fails the check its manifest entry names, and fails it first; the eight strict_only vectors pass a loose read and fail a strict one.
  2. Decoder agreement. The golden values in manifest.json - chunk layout, the layer table, each LAYR chunk's frame and version, per-layer decompressed bytes, leaf hashes and Merkle root - are recomputed from this crate's own parse of the same files.
  3. Pipeline agreement. The settings a conforming reader must resolve for a sample of (layer, sector) points - base values, the bottom and transition blend, the absent-field defaults, the LROV overrides that §4.5 requires a reader to apply - are compared against the manifest's resolved_timing for every valid vector. This crate did not write those expectations, so agreeing with them is evidence about the pipeline rather than a restatement of it.

The corpus itself is generated by make_vectors (in rust/corpus-gen) and checked by verify_vectors (in rust/corpus), which share no code with each other and depend on neither this crate nor one another - and cross_check points that second one at files this crate wrote. Agreement between all three is evidence about the specification, not self-consistency, and CI runs all of it on every change: the generator reproducing the committed corpus byte for byte, the corpus under the specification's own reader, this crate's output under that reader, this crate's suite over the corpus, plus formatting, lints, the declared minimum Rust version, the wasm32 target and the packaging path a release takes.

tests/roundtrip.rs covers the other direction: files this crate writes validate, decode back to the exact pixels that went in, verify their own integrity tree, and round trip through password and machine-binding encryption. tests/memory.rs measures what validating a file actually allocates, since "one block at a time" is a claim that a small test cannot show.

The make_test_file example writes four sample files - a full-featured single-sector print, the same without the optional integrity tree, the same sealed with a password, and a two-resin print with per-layer settings. Appendix A of the specification documents what is in each of them, with the numbers the command just printed.

cargo test                                   # unit, conformance and round trip

cargo test --test conformance                # the corpus only

The conformance tests read the corpus from beside the specification, so they run from a checkout of the repository. Unpacking the published .crate on its own and running cargo test fails with that explanation rather than passing without having checked anything — conformance is not a property a crate can assert about itself in isolation.

Scope

Implemented: the container (§3), every chunk type (§4), binary, grayscale and split REE (§5), the zstd block and dictionary strategy (§6), the sector model (§7), the layer timing pipeline (§8), authenticated encryption with Argon2id password wrapping and X25519 machine binding (§9), and the checks of §11.

Deliberately not implemented, with reasons:

  • VOXL parsing. A VOXL payload is opaque to LUMEN (§4.11) and is carried through byte for byte. A strict read checks only that it is recognizably VOXL.
  • Extension payloads. EXTD is vendor-defined and copied through, never interpreted (§4.12). No extension is implemented, so any extension claiming critical is refused, as the specification requires.
  • Signatures. EXTD/SIGN is optional in v1 and not part of this crate.

Specification revision

Written against LUMEN v1.0, the published revision tagged v1.0 in the LumenFormat repository, which is where the text of that revision is fixed. A published revision is immutable (§10.3): a correction ships as a new revision with its own number, and it is this crate's conformance test that reports the difference, since the crate names a revision rather than tracking the working tree. That is why the crate's version is the revision it was written against, 1.0.

The corpus is the shared contract. A file this crate writes is checked against the specification's own encoder by its own validator, in both directions, and that agreement is what "conforming" means here rather than any claim about this crate's internals.

Requirements

Rust 1.85 or later, edition 2021. The floor is set by a transitive dependency of the RustCrypto stack (zeroize_derive, which uses edition 2024) rather than by anything this crate does, and CI runs the whole suite on 1.85 with the committed lock to keep the claim honest.

Hosted targets build with no extra tooling. A wasm32-unknown-unknown build - the DragonFruit webview target - needs clang on the build host, because the zstd dependency compiles C; without it the failure is cc-rs: failed to find tool "clang", which is a toolchain gap rather than a crate one. The getrandom js backend is selected for wasm32, since that target has no other entropy source.

There is no no_std support: a print file is a bulk data format and the block/dictionary machinery wants an allocator.

License

MIT OR Apache-2.0, matching the specification's MIT license while remaining usable by GPL-3.0 consumers such as Odyssey. See LICENSE-MIT and LICENSE-APACHE.