Expand description
Python string semantics on raw bytes, as UnityPy applies them to names.
UnityPy compares CAB and resource names after ntpath.basename and
str.lower() (environment.py:357-363, classes/PPtr.py:66-77,
helpers/ResourceReader.py:11-18). Python lower-cases a surrogate-escaped
string: valid UTF-8 runs with Unicode full case mapping (Python 3.11 has
Unicode 14.0), invalid bytes passed through. Rust’s tables are newer, so a
generated exception table (crate::tables::lower_exception) pins the code
points whose mapping changed after 14.0 (tools/gen/, package A; plan risk R7).
The ntpath functions behave as Python 3.11’s do, which
tools/fixtures/env.py records (tests/fixtures/d/pystr.json).
Which bytes are characters. A surrogate-escaped string is UTF-8
decoded with surrogateescape: every valid UTF-8 sequence is one
character, and every byte of an invalid sequence is a lone surrogate
(U+DC80-U+DCFF), which lower() keeps and the encoder turns back into
the byte. Python’s decoder and std::str::Utf8Chunks split a byte string
into the same valid runs (both reject surrogates, overlong forms and code
points past U+10FFFF), so a run of std::str::Utf8Chunks is lowered as a
str and its invalid bytes are copied.
Final sigma. str.lower() maps U+03A3 to U+03C2 in the
Final_Sigma context (handle_capital_sigma in Python’s
unicodeobject.c): a cased character before it and none after it,
skipping case-ignorable ones. The two classes come from Python 3.11’s
own str.lower() (crate::tables::is_sigma_cased,
crate::tables::is_case_ignorable, Unicode 14.0), not from Rust’s newer
tables, which differ for characters Unicode assigned after 14.0. A lone
surrogate (an invalid byte) is neither cased nor case-ignorable to
Python, so the context never crosses a run’s edge and a run lowered on
its own gets Python’s answer.
Memory. Each function that returns a string computes it from an
input’s name (a directory path, an external path, a resource path, up to
the input’s size) and charges it to that input’s meter before it is
allocated (RawStr::charged’s cost): a refusal allocates nothing. A
result equal to its input (a name already lower-case and without a
directory) is returned as the input’s own bytes, uncopied, and a result
that is a part of its input (a basename already lower-case) as a slice of
them. Comparisons that only need to know whether two names are equal
(lower_eq) compare without building the lowered string at all, and
nothing else is allocated: a capital sigma’s context is read in place.
Owner: package D.
Functions§
- clean_
external_ path PPtr.deref’s external path cleaning: strip a leadingarchive:/, then a leadingassets/, keep what follows the last/, lower-case it; charged tometer.- is_
split_ path - Whether UnityPy’s
load_filereads a path as the parts of a split file:reSplit.match(path)(environment.py:25,:111-116), withreSplit(.*?([^\/\\]+?))\.split\d+, a match at the start that need not reach the end. So.splitand a digit must follow at least one character that is no/or\, and no\n(which.does not match) may come before the last separator ahead of it. Python’s\don astrpattern isPy_UNICODE_ISDECIMAL(sre’sSRE_UNI_IS_DIGIT): a character with a decimal digit value, which Unicode gives exactly the categoryNd, read from Python 3.11’s own table (crate::tables::nd_digit, Unicode 14.0), so a name such asx.split²(No) orx.splitⅠ(Nl) is one file, as UnityPy reads it (tests/fixtures/d/pystr.jsonrecords\dover every code point). Linear in the path. - lower_
eq - Whether
a.lower() == lowered, compared piece by piece without buildinga.lower()(classes/PPtr.py:75-76compares eachfileskey so); it allocates nothing. - ntpath_
basename ntpath.basename: aftersplitdrive, the part after the last/or\.- ntpath_
splitext ntpath.splitext:(root, ext), leading dots not counting as an extension.- py_
lower str.lower()on surrogate-escaped bytes, charged tometer.- simplify_
name simplify_name:ntpath.basename(name).lower(), the CAB registry’s key, charged tometer.- simplify_
name_ vec - A transient CAB lookup key. Keeps computed bytes in a fallibly allocated,
charged vector instead of constructing a
Bytesshared owner (CR10).