Skip to main content

Module env

Module env 

Source
Expand description

UnityPy’s Environment (environment.py:28-363): loaded files, the CAB registry, dependency loading, pointer dereference, resource reads and the container.

Two load modes (plan section 3.13):

  • LoadMode::Eager is UnityPy: a bundle decodes every block, then parses every entry in directory order and registers each SerializedFile and raw entry as a CAB by crate::pystr::simplify_name as it parses it (files/File.py:76-90). A failing entry fails the load, and load_file raises before the bundle enters files (environment.py:154-157), but the CABs of the entries parsed before it stay registered and resolvable (partial registration: probed on UnityPy 1.25.2 with a bundle [good CAB, its .resS, bad CAB], after which env.files is empty and env.cabs holds the first two).
  • LoadMode::Lazy is the batch engine’s: a bundle registers its entries’ names at open and parses an entry the first time it is asked for (computed with no lock held and published once: two threads that ask at once may both parse, and the first to publish wins).

On a scene whose blocks all decode, whose entries all parse and none of which is a container, the two modes give identical results, the container pass’s RuntimeError included. Elsewhere they differ in exactly four ways (plan section 3.13, PARITY.md section 19.3, each pinned by a test in tests/env/modes.rs): (a) a failing entry fails the load under Eager (with the partial registration above) but only the get that parses it under Lazy (an entry shadowed by a later one of its path is then never parsed; crate::files::BundleFile::load_all reproduces Eager’s error); (b) a nested container’s own name is a key under Lazy, whose slot answers None, where Eager registers no such key (and keeps an earlier CAB of that path under it); its entries are registered when it is reached, before any lookup that resolves its slot, starts from a file inside it, names it as the asking file’s sibling or resource candidate, or walks the container, and before a registration drops its slot (an evicted dependency’s before the next container pass), with the position and the value an eager load gives them, so only a lookup by name alone of one of its entries can miss before then (registry’s module docs state the invariant); (c) a block that fails to decode, or trips a limit, after the last byte any read needs fails the load under Eager (UnityPy decodes every block before any entry, files/BundleFile.py:173-186) and nothing under Lazy; (d) a directory whose spans nest or overlap makes Lazy decode and keep the shared blocks once per span, so it can stop at the input’s meter (crate::LimitKind::InputMemory) or decode-work bound (crate::LimitKind::DecodeWork) where Eager’s one run fits.

Threads (plan section 3.16, item 6). Each key’s registration, each files entry and each path_dict entry is published whole, so no call sees one half made. A load’s or a reach’s set of keys is not published at once: another thread may find one key of a load, or of a nested container being reached, and miss a later one (review round 6: these docs said concurrent calls act as if made one after another, which that does not). A call that reaches a nested container returns only after its whole subtree is registered, and a load only after all its keys and its files entry are. A dependency’s load under Lazy (load_dependency, or a resolver’s load made as a dependency, its files entry under keep_resolver_loads included) publishes its first key only once every one is charged, so one that fails publishes none (registry’s module docs, “A failed dependency load”); a caller’s load, and any load under Eager, publishes key by key, and one that fails keeps those it published. A call’s result depends on what was done before it, by its thread or another, as a Python dict’s does: a load, a resolver load, and under Lazy the first call that reaches a nested container. A container pass’s preload walk that a load on another thread meets gives, for each key the load changes, what UnityPy’s walk gives with that change made at the point of the walk at which it was made, between two of its steps (watch‘s module docs, W1; review round 14); a key the load adds does not fail the pass, where UnityPy’s raises. What a set of calls leaves does not depend on the order of the ones that are not loads: the registry orders entries by rank (registry), files by the loads’ numbers the same way (Environment::files), and each path_dict entry keeps the least stamp (crate::container). So once racing loads have returned, the registry and files hold what the same loads make one after another in the order of their numbers (review round 16: files kept the order the threads listed in, so a load that took the lower number but listed last won files and lost the registry, for good).

Memory. Each load makes its input’s meter (crate::limits::Meter::for_input): crate::budget::Reservation::Input for what a caller loads, crate::budget::Reservation::Dependency for a dependency. Every Source is bounded by crate::limits::Limits::max_input_bytes before its bytes are detected (LimitKind::InputBytes): a Source::Path is read by read_path, refused from its length before anything is read and charged to that meter before its bytes exist; a Source::Fetch is refused from the size its fetcher states (Fetch::size_hint) before it is fetched, and from the length of what it returns; Source::Bytes from their length. Each registry slot, its key and its share of the registry’s tables are charged to the meter of the file that registered it (registry), and each files entry, its share of the list and its lowered key to the loaded file’s (Environment::files); the LRU’s evictions allocate nothing (registry). Listings (Environment::objects, Environment::container, Registry::keys) are iterators: they hold a handle per nesting level and allocate nothing per record.

Dependencies (files loaded to resolve a pointer or a resource, or by Environment::load_dependency) are kept in an LRU bounded by EnvOptions::dep_cache_bytes (the sum of their meters). The registry holds no strong handle on a dependency, only what reopens it (registry), so evicting one drops the LRU’s handle and with it the dependency’s runs and parsed entries, and the bytes of a source that can be read again (Source::Path, Source::Fetch) with their charge, once nothing else holds them; the next read opens it again from its source on crate::limits::Meter::reopen of its meter: the same memory account, and the new open’s own work allowance, so eviction churn never exhausts a valid dependency’s allowance, and a handle of the old open that someone still holds (a nested bundle a resource read kept) goes on counting against the old open’s, which nothing resets. A Source::Bytes dependency keeps its bytes, which are the caller’s. An ordinary CAB lookup after reopening a dependency can see changed bytes, and it follows the indices registered at the first open, so a source whose bundle directory was reordered can resolve a CAB name to a different entry. Retained-object reuse via Environment::deref_held_reusing assumes stable bytes across reopens. If the source changes after eviction, reuse may still return the previously retained object. The files a caller loaded are never evicted. Every call that reads or registers through a dependency’s meter (a load, a lookup, a dereference, a resource read, an object’s tree with the script hook, the container pass), when the pool refuses it, evicts the others and retries once, by one rule however the calls nest (registry’s with_room; tests/env/calls.rs lists every public call and how it meets the pool); the refusal is never cached (crate::once::get_or_publish_result), so a later read with room in the pool succeeds. A read the caller makes itself on a dependency’s handle (an object’s value) is no call of the Environment’s, and its refusal is the caller’s.

Resolver loads in files (probed on UnityPy 1.25.2, plan section 3.13). UnityPy keeps every file it loads, those it loads to resolve a pointer or a resource included, in env.files, for good:

  • a resource read’s fallback (SerializedFile.load_dependencies: the file’s externals and the four candidates) calls load_file(name, True), whose second positional parameter is parent, so those files are not dependencies: keyed by the name asked, their objects are in env.objects and their CABs’ rows in env.container, and a second fallback loads the file again and replaces the entry in place;
  • find_file loads a file as a dependency (its is_dependency defaults to true, and a dereference takes the default), keyed by the path it resolved (env.path joined with the name: load_file keys a path by itself); env.objects and env.container skip it, and since env.container walks env.cabs, where a dependency’s CAB replaces a CAB of the same key, the replaced file’s rows leave the container while its objects stay in env.objects, which walks env.files. With is_dependency=False the file is a caller’s load under that key. A raw reader carries no flag (parse_file returns the reader itself), so it is no dependency to getattr(f, "is_dependency", False).

EnvOptions::keep_resolver_loads (true in EnvOptions::unitypy) does exactly that: Environment::files lists each resolver load as a FileEntry, is_dependency as UnityPy sets it, and none is evicted. With it false (EnvOptions::batch) resolver loads are LRU dependencies, evictable, and in neither files, objects nor container, whatever find_file’s is_dependency: PARITY.md section 19.3’s row “resolver loads under keep_resolver_loads false” (find_file’s and a dereference’s; review round 12: no row listed them) and section 19.2’s SerializedFile.load_dependencies row (a resource fallback’s). An evicted dependency is opened again from its source by the next read that needs it (section 19.3’s row “a dependency the LRU evicts”). Environment::load_dependency, which UnityPy has no call for, never adds to files.

Locks. The registry, the files list, the LRU and a file’s path_dict are locked only to look up or publish; a file is loaded, an entry parsed and a resolver, fetcher or provider called with no lock held (crate::once; tests/env/locks.rs runs loads on another thread while a resolver or a fetch waits for them). The one caller code that may run under them is a crate::budget::MemoryBudget or crate::budget::PermitHome (a growth charged while a table is published, a replaced value’s charge returned), whose contract forbids blocking and calling the core (plan section 3.16).

Where UnityPy reaches the filesystem (find_file’s walk of env.path, load_file’s relative paths), the core asks a CabResolver: the core itself never opens a path it was not given.

Owner: package D. Option constructors are implemented by the scaffold.

Re-exports§

pub use pptr::DerefError;
pub use pptr::PPtr;
pub use registry::Cab;
pub use registry::HeldCab;
pub use registry::Keys;
pub use registry::RawCab;
pub use registry::Registry;
pub use resource::Candidate;
pub use resource::SUFFIXES;
pub use resource::resource_candidates;

Modules§

pptr
PPtr and its dereference (classes/PPtr.py:15-107).
registry
The CAB registry: UnityPy’s env.cabs dict (environment.py:252-280).
resource
Resource lookup (helpers/ResourceReader.py:10-38).

Structs§

ContainerItem
A container row with the object it points at, as env.container yields.
ContainerItems
The iterator Environment::container returns, CABs in registry order.
DependencyFile
Reopens a registered dependency without pinning its files in the cache.
EnvOptions
Environment options.
EnvStats
Counters, for the batch report and tests.
Environment
UnityPy’s Environment. A cheap handle: clones share one environment.
FileEntries
The iterator Environment::files returns: a cursor at the position of the last entry it yielded, which entries listed meanwhile do not disturb.
FileEntry
One entry of Environment::files: UnityPy’s env.files[name], with the file’s is_dependency flag.
FileOccurrence
A serialized file’s occurrence, comparable before its data is loaded. Dependency occurrences are stable across cache reopens.
Objects
The iterator Environment::objects returns: a handle per nesting level (the position of the last files entry it stepped into, the bundle, the entry, the object index).

Enums§

LoadMode
When a container’s entries are parsed.
MonoBehaviourError
Failure of the explicit MonoBehaviour generator operation. Only ValueError failures are swallowed by automatic fallback; other failures remain visible.
Source
Something to load. Every kind is bounded by Limits::max_input_bytes: bytes longer than it, however they came, are refused (LimitKind::InputBytes) before anything detects, parses or keeps them, so a refused fetch’s bytes drop at once, with any charge they carry.

Traits§

CabResolver
Finds a file the registry does not hold, where UnityPy looks in env.path: find_file’s fallback after get_cab (environment.py:310-354), and each load_file(name, True) of a resource read’s fallback (files/SerializedFile.py:330-349).
Fetch
Bytes the caller fetches on demand (a batch engine’s mmap or object store).

Functions§

read_path
Reads path whole for Source::Path. A file whose length is above limits.max_input_bytes is refused before anything is read (LimitKind::InputBytes). The buffer is allocated once at the stat’d length, charged to meter first, and filled; the end of the file is then checked with a read into a fixed stack buffer, so a file that ends where its length says costs exactly its buffer (heap_cost(len) and the owner record, crate::budget::OWNER_COST), never a growth past it. Only a file that outlasts its stat’d length (one that grows while it is read, a pipe, a device) grows the buffer, charged as it grows, and the read stops at the bound. The returned bytes carry the charge.