Skip to main content

Module detect

Module detect 

Source
Expand description

ImportHelper.check_file_type and parse_file’s naming rule (helpers/ImportHelper.py:44-156), quirks included: every rule of docs/CANONICAL_DUMP.md section 4.2 (l0) and section 7 (L0 detection).

Notable rules: eight U+00FA characters as a signature (the UTF-8 bytes C3 BA x 8, never FA x 8); a zip only when byte 4 is NUL; a 20-byte file with no NUL raises from bytes, and from a path, whose stream reader drops the last byte it read, is a resource (check_file_type_with_reader); a gzip file must decompress in full, every member and trailer checked (the P0 spike inflated only 20 bytes, which this fixes). That is UnityPy’s own behaviour, not a unityrs policy: its streaming read_string_to_null tests Position != Length per byte (streams/EndianBinaryReader.py:136-144), and Length seeks the GzipFile to its end (streams/EndianBinaryReader.py:307-312), which inflates every member. The output is streamed through a scratch buffer and only its first 20 bytes and its length are kept, so detection charges nothing: the scratch (at most 64 KiB) is part of plan section 3.15’s constant, as are the inflater’s state and 32 KiB window, one member’s at a time. Also: brotli at 0x20; the SerializedFile header read big-endian.

The inflater is zlib’s. flate2 runs zlib-rs (unityrs-core’s manifest), a port of zlib that keeps its rules, so a stream inflates or raises as under CPython’s zlib, which gzip._GzipReader runs. One rule differs in flate2’s default backend, miniz_oxide: a match reaching before its member’s first output byte is zlib’s “invalid distance too far back”, where miniz_oxide copies zeros from its window (round 9: a corrupt stream detected as ResourceFile or WebFile, and a bundle UnityPy fails loaded). tests/fixtures/c/detect.json pins it.

Work. One stream inflates at most Limits::max_inflate bytes. A container detects each of its entries (check_entry_type), and many directory rows can name one small stream that inflates to the bound, so there every inflated byte is also counted as the open’s work, the allowance block decodes draw on (Limits::max_decode_work, Meter::count_decode): the open’s detections together inflate no more than that, and the one that passes it is refused (DetectError::Work, which the container reports as the open’s LimitKind::DecodeWork limit), as is every one after it. The bytes of the stream a detection reads count too, inflated or not: a header’s file name or comment searched for its NUL, deflate input that inflates to nothing (empty blocks), each member’s header and trailer, and the zero padding after it, all of which each row naming the stream reads again (round 4’s sweep: a stream of one tiny member and a gigabyte of padding counted a few bytes per row). A detection that fails counts what it read and inflated before the failure, and returns the stream’s own error (round 5: a malformed deflate stream returned before its consumed input was counted, so every row read it again uncounted).

Owner: package C.

Enums§

DetectError
Why detection raised (the dump’s "detect" error kind).
FileType
UnityPy’s FileType verdict.
ParsedAs
What UnityPy builds from a detected file (l0.parsed_as).

Functions§

check_entry_type
check_file_type on a container’s entry, with every byte a gzip stream inflates counted as work of the open meter belongs to (module docs, “Work”): the verdict is the same until the open’s allowance runs out, and DetectError::Work from then on.
check_file_type
check_file_type on the bytes (UnityPy’s memory reader).
check_file_type_with_reader
check_file_type by the rules of the reader UnityPy reads data with: the stream reader for a file loaded from a path, whose read_string_to_null(20) drops the last byte it read where no NUL is in reach and never raises from the start of a file, so a 20-byte file with no NUL is a ResourceFile (or a WebFile by its first 19 bytes) where the memory reader raises (DetectError::Unterminated).
parsed_as
parse_file’s choice. An AssetsFile whose name ends in .resS, .resource, .config, .xml or .dat (a case-sensitive byte suffix) stays raw; a zip is Zip only for an input (is_entry false), else raw.