Skip to main content

Module container

Module container 

Source
Expand description

Container awareness for ZIP-based formats (DOCX/XLSX/PPTX/EPUB/JAR/ODT/plain ZIP).

The container is parsed with a small, strict ZIP framing parser that keeps the exact bytes of every local header, central-directory record, data descriptor and the end-of-central-directory block. Member payloads are handled by kind:

  • stored — the payload is the member’s plaintext;
  • deflate — the raw deflate stream is inverted with preflate (Microsoft, Apache-2.0), which yields the plaintext plus a small “corrections” blob that recreates the original stream bit-exact whatever encoder produced it (zlib, zlib-ng, miniz, .NET, …);
  • raw — anything else (unknown methods, encrypted members, streams preflate cannot model): the compressed bytes themselves are kept.

The plaintext/raw payloads are what T-saur chunks, deduplicates and compresses. Rebuilding is deterministic and is verified against the original hash at pack time; if the framing has gaps, ZIP64 records or anything else we do not model, the whole file is stored raw instead.

Constants§

KIND_DEFLATE
KIND_RAW
KIND_STORED
LEPTON_MAX_INPUT
JPEG streams above this size are kept literal (bounded memory/time).
PREFLATE_PLAINTEXT_LIMIT
Deflate streams above this plaintext size are kept raw (bounded memory, anti-bomb).
SEG_JPEG
JPEG payload recompressed losslessly with Lepton (the chunked payload is the Lepton stream).
SEG_LITERAL
SEG_ZLIB
STREAM_EXTENSIONS
ZIP_EXTENSIONS

Functions§

explode
Parse the ZIP framing and invert the member payloads. Returns None for anything we do not model exactly (the caller then stores the file raw).
explode_streams
Split a PDF (or similar) into literal segments, inverted zlib streams and, when jpeg is set and the build has Lepton, recompressed JPEG streams. A file that is itself a JPEG becomes a single JPEG segment (or, with jpeg off, no recipe at all: it is stored as it is, which every build can read).
is_stream_container
is_zip_like
jpeg_to_lepton
Lepton-encode a JPEG; the library verifies the round trip itself. Single-threaded so the output is deterministic whatever the machine.
kinds
Statistics helper for reports.
lepton_to_jpeg
rebuild
Rebuild the container from the recipe and the member payloads (same order as recipe.members).
rebuild_streams
try_bitexact
Explode and prove that the rebuild reproduces the exact input bytes.
try_bitexact_streams