pub struct Lzma;Expand description
Compression type 1: liblzma’s raw LZMA1 decoder, to the end of the input.
Implementations§
Source§impl Lzma
impl Lzma
Sourcepub const DECODER_STATE: u64
pub const DECODER_STATE: u64
What one liblzma raw LZMA1 decoder holds beside its dictionary, in
glibc chunks (heap_cost): liblzma’s own state, 32,784 bytes (the
stream’s lzma_internal, common/common.c:183; the LZ decoder’s
coder, lz/lz_decoder.c:229; the LZMA1 decoder and its probability
tables, lzma/lzma_decoder.c:1122, of the xz 5.8.4 that
liblzma-sys 0.4.9 builds), and the 256 bytes the liblzma crate’s
filter chain takes on the Rust heap while it is built (a one-entry
array grown to four, 16 bytes then 64, both held while it grows, and
a 128-byte options node: src/stream.rs:709-735 and :1128-1131 of
liblzma 0.4.9). Each attempt charges it beside the dictionary
(Self::decoder_charge; review round 12: only the dictionary was
charged, so about 32 KiB a decode reached no meter).
tests/lzma_decoder_heap.rs measures both on glibc, in a child
process, and fails unless liblzma’s allocations and the filter chain
are exactly Self::decoder_charge, and one decode’s charge covers
the heap it holds with at most 32 bytes to spare (the filter array’s
first chunk, freed once it grows); cargo test -p unityrs-core --test lzma_decoder_heap -- --nocapture prints what it measured.
Sourcepub const fn decoder_charge(dict: u32) -> u64
pub const fn decoder_charge(dict: u32) -> u64
What one decode attempt with a dictionary of dict bytes charges
before liblzma allocates anything: the dictionary buffer liblzma
allocates (lz/lz_decoder.c:253-289: at least 4 KiB, rounded up to
a multiple of 16, and 2 * LZ_DICT_REPEAT_MAX, 576 bytes, more; its
LZ_DICT_EXTRA is 0 in liblzma-sys’s build, whose config.h
defines no HAVE_IMMINTRIN_H, lz/lz_decoder.h:22-46), in a glibc
chunk, and Self::DECODER_STATE.
A const fn, so a bound sized from it is checked when it compiles
(unityrs-tpk’s TpkProvider::MAX_DECODER_MEMORY).
Trait Implementations§
Source§impl BlockCodec for Lzma
impl BlockCodec for Lzma
Source§fn decode(
&self,
input: &Bytes,
uncompressed: u32,
out: &mut DecodeBuf,
) -> Result<Decoded, Error>
fn decode( &self, input: &Bytes, uncompressed: u32, out: &mut DecodeBuf, ) -> Result<Decoded, Error>
decompress_lzma (CompressionHelper.py:17-44): the properties byte
and the little-endian dictionary size, then a raw LZMA1
LZMADecompressor’s decompress(data[5:]), whose loop is
Python’s decompress_buf (Modules/_lzmamodule.c): run until the
stream ends, the input is used up with room left in the output, or an
error; the declared size plays no part.
The dictionary. liblzma allocates the dictionary the properties
name (up to 4 GiB) on the C heap. The decoder only uses it to
refuse a match distance at or beyond min(output so far, dictionary),
so a dictionary at least as large as the output decodes exactly what
the named one does. The first attempt uses the declared size, between
liblzma’s 4 KiB minimum and 8 MiB, capped by the named size (a
well-formed block fills exactly its declared size, so it decodes
once); an attempt that fails on data
after it outgrew its dictionary is decoded again with a larger one
(at least twice the output so far, capped by the named size), so the
dictionary stays within a constant plus twice the block’s charged
output. The two caps keep a match farther back than the named
dictionary UnityPy’s LZMAError, whatever the declared size
(tests/codecs.rs
lzma_a_match_past_the_named_dictionary_raises_whatever_the_declared_size).
Each attempt’s dictionary buffer and the decoder’s state
(Lzma::DECODER_STATE) are charged to the input’s meter
(DecodeBuf::meter) before liblzma allocates them, and returned
when the attempt ends, so the meter sees them beside the output (plan
section 3.15); a refusal is the meter’s, like a refused growth. The
output’s room is out’s: it grows geometrically, never past the
block’s cap, nor past its declared size while the block is short of it
(its windows stop there, DecodeBuf::begin_block), and a retry
keeps the room the failed attempt grew, so an attempt holds its room
(and, while the room grows, the block it replaces) beside its
dictionary. A caller that must bound that sum by
the output’s cap alone allocates the room whole first, as unityrs-tpk
does (TpkProvider::MAX_DECODER_MEMORY; review round 13).