unityrs_core/env/mod.rs
1// Ported from UnityPy 1.25.2 (MIT): UnityPy/environment.py, UnityPy/files/SerializedFile.py, UnityPy/files/BundleFile.py, UnityPy/files/File.py, UnityPy/helpers/ResourceReader.py. See THIRD_PARTY_LICENSES.
2//! UnityPy's `Environment` (`environment.py:28-363`): loaded files, the CAB
3//! registry, dependency loading, pointer dereference, resource reads and the
4//! container.
5//!
6//! Two load modes (plan section 3.13):
7//! - [`LoadMode::Eager`] is UnityPy: a bundle decodes every block, then parses
8//! every entry in directory order and registers each `SerializedFile` and
9//! raw entry as a CAB by [`crate::pystr::simplify_name`] *as it parses it*
10//! (`files/File.py:76-90`). A failing entry fails the load, and
11//! `load_file` raises before the bundle enters `files`
12//! (`environment.py:154-157`), but the CABs of the entries parsed before it
13//! stay registered and resolvable (partial registration: probed on
14//! UnityPy 1.25.2 with a bundle `[good CAB, its .resS, bad CAB]`, after
15//! which `env.files` is empty and `env.cabs` holds the first two).
16//! - [`LoadMode::Lazy`] is the batch engine's: a bundle registers its entries'
17//! names at open and parses an entry the first time it is asked for
18//! (computed with no lock held and published once: two threads that ask
19//! at once may both parse, and the first to publish wins).
20//!
21//! On a scene whose blocks all decode, whose entries all parse and none of
22//! which is a container, the two modes give identical results, the
23//! container pass's `RuntimeError` included. Elsewhere they differ in
24//! exactly four ways (plan section 3.13, `PARITY.md` section 19.3, each
25//! pinned by a test in `tests/env/modes.rs`): (a) a failing entry fails the load
26//! under `Eager` (with the partial registration above) but only the `get`
27//! that parses it under `Lazy` (an entry shadowed by a later one of its path
28//! is then never parsed; [`crate::files::BundleFile::load_all`] reproduces
29//! `Eager`'s error); (b) a nested container's own name is a key under
30//! `Lazy`, whose slot answers `None`, where `Eager` registers no such key
31//! (and keeps an earlier CAB of that path under it); its entries are
32//! registered when it is *reached*, before any lookup that resolves its
33//! slot, starts from a file inside it, names it as the asking file's
34//! sibling or resource candidate, or walks the container, and before a
35//! registration drops its slot (an evicted dependency's before the next
36//! container pass), with the position and the value an eager load gives
37//! them, so only a lookup by name alone of one of its entries can miss
38//! before then ([`registry`]'s module docs state the invariant);
39//! (c) a block that fails to decode, or trips a limit,
40//! after the last byte any read needs fails the load under `Eager` (UnityPy
41//! decodes every block before any entry, `files/BundleFile.py:173-186`) and
42//! nothing under `Lazy`; (d) a directory whose spans nest or overlap makes
43//! `Lazy` decode and keep the shared blocks once per span, so it can stop
44//! at the input's meter ([`crate::LimitKind::InputMemory`]) or decode-work
45//! bound ([`crate::LimitKind::DecodeWork`]) where `Eager`'s one run fits.
46//!
47//! **Threads** (plan section 3.16, item 6). Each key's registration, each
48//! `files` entry and each `path_dict` entry is published whole, so no call
49//! sees one half made. A load's or a reach's set of keys is not published at
50//! once: another thread may find one key of a load, or of a nested
51//! container being reached, and miss a later one (review round 6: these
52//! docs said concurrent calls act as if made one after another, which that
53//! does not). A call that reaches a nested container returns only after its
54//! whole subtree is registered, and a load only after all its keys and its
55//! `files` entry are. A dependency's load under `Lazy` (`load_dependency`,
56//! or a resolver's load made as a dependency, its `files` entry under
57//! `keep_resolver_loads` included) publishes its first key only once every
58//! one is charged, so one that fails publishes none ([`registry`]'s module
59//! docs, "A failed dependency load"); a caller's load, and any load under
60//! `Eager`, publishes key by key, and one that fails keeps those it
61//! published. A call's result depends on what was done before it,
62//! by its thread or another, as a Python dict's does: a load, a resolver
63//! load, and under `Lazy` the first call that reaches a nested container.
64//! A container pass's preload walk that a load on another thread meets
65//! gives, for each key the load changes, what UnityPy's walk gives with
66//! that change made at the point of the walk at which it was made, between
67//! two of its steps (`watch`'s module docs, W1; review round 14); a key
68//! the load adds does not fail the pass, where UnityPy's raises.
69//! What a set of calls leaves does not depend on the order of the ones that
70//! are not loads: the registry orders entries by rank ([`registry`]),
71//! `files` by the loads' numbers the same way ([`Environment::files`]),
72//! and each `path_dict` entry keeps the least stamp ([`crate::container`]).
73//! So once racing loads have returned, the registry and `files` hold what
74//! the same loads make one after another in the order of their numbers
75//! (review round 16: `files` kept the order the threads listed in, so a
76//! load that took the lower number but listed last won `files` and lost
77//! the registry, for good).
78//!
79//! **Memory.** Each load makes its input's meter
80//! ([`crate::limits::Meter::for_input`]): [`crate::budget::Reservation::Input`]
81//! for what a caller loads, [`crate::budget::Reservation::Dependency`] for a
82//! dependency. Every [`Source`] is bounded by
83//! [`crate::limits::Limits::max_input_bytes`] before its bytes are detected
84//! ([`LimitKind::InputBytes`]): a [`Source::Path`] is read by [`read_path`],
85//! refused from its length before anything is read and charged to that
86//! meter before its bytes exist; a [`Source::Fetch`] is refused from the
87//! size its fetcher states ([`Fetch::size_hint`]) before it is fetched, and
88//! from the length of what it returns; [`Source::Bytes`] from their length.
89//! Each registry slot, its key and its share of the
90//! registry's tables are charged to the meter of the file that registered
91//! it ([`registry`]), and each `files` entry, its share of the list and its
92//! lowered key to the loaded file's ([`Environment::files`]); the LRU's
93//! evictions allocate nothing ([`registry`]). Listings ([`Environment::objects`],
94//! [`Environment::container`], [`Registry::keys`]) are iterators: they hold
95//! a handle per nesting level and allocate nothing per record.
96//!
97//! **Dependencies** (files loaded to resolve a pointer or a resource, or by
98//! [`Environment::load_dependency`]) are kept in an LRU bounded by
99//! [`EnvOptions::dep_cache_bytes`] (the sum of their meters). The registry
100//! holds no strong handle on a dependency, only what reopens it
101//! ([`registry`]), so evicting one drops the LRU's handle and with it the
102//! dependency's runs and parsed entries, and the bytes of a source that can
103//! be read again ([`Source::Path`], [`Source::Fetch`]) with their charge,
104//! once nothing else holds them; the next read opens it again from its
105//! source on [`crate::limits::Meter::reopen`] of its meter: the same memory
106//! account, and the new open's own work allowance, so eviction churn never
107//! exhausts a valid dependency's allowance, and a handle of the old open
108//! that someone still holds (a nested bundle a resource read kept) goes on
109//! counting against the old open's, which nothing resets. A
110//! [`Source::Bytes`] dependency keeps its bytes, which are the caller's.
111//! An ordinary CAB lookup after reopening a dependency can see changed bytes,
112//! and it follows the indices registered at the first open, so a source whose
113//! bundle directory was reordered can resolve a CAB name to a different entry.
114//! Retained-object reuse via [`Environment::deref_held_reusing`] assumes
115//! stable bytes across reopens. If the source changes after eviction, reuse
116//! may still return the previously retained object.
117//! The files a caller loaded are never evicted. Every call that reads or
118//! registers through a dependency's meter (a load, a lookup, a dereference,
119//! a resource read, an object's tree with the script hook, the container
120//! pass), when the pool refuses it, evicts the others and retries once, by
121//! one rule however the calls nest ([`registry`]'s `with_room`;
122//! `tests/env/calls.rs` lists every public call and how it meets the
123//! pool); the refusal is never cached
124//! ([`crate::once::get_or_publish_result`]), so a later read with room in
125//! the pool succeeds. A read the caller makes itself on a dependency's
126//! handle (an object's value) is no call of the Environment's, and its
127//! refusal is the caller's.
128//!
129//! **Resolver loads in `files`** (probed on UnityPy 1.25.2, plan section
130//! 3.13). UnityPy keeps every file it loads, those it loads to resolve a
131//! pointer or a resource included, in `env.files`, for good:
132//! - a resource read's fallback (`SerializedFile.load_dependencies`: the
133//! file's externals and the four candidates) calls
134//! `load_file(name, True)`, whose second positional parameter is
135//! `parent`, so those files are **not** dependencies: keyed by the name
136//! asked, their objects are in `env.objects` and their CABs' rows in
137//! `env.container`, and a second fallback loads the file again and
138//! replaces the entry in place;
139//! - `find_file` loads a file **as a dependency** (its `is_dependency`
140//! defaults to true, and a dereference takes the default), keyed by the
141//! path it resolved (`env.path` joined with the name: `load_file` keys a
142//! path by itself); `env.objects` and `env.container` skip it, and since
143//! `env.container` walks `env.cabs`, where a dependency's CAB replaces a
144//! CAB of the same key, the replaced file's rows leave the container while
145//! its objects stay in `env.objects`, which walks `env.files`. With
146//! `is_dependency=False` the file is a caller's load under that key. A raw
147//! reader carries no flag (`parse_file` returns the reader itself), so it
148//! is no dependency to `getattr(f, "is_dependency", False)`.
149//!
150//! [`EnvOptions::keep_resolver_loads`] (true in [`EnvOptions::unitypy`]) does
151//! exactly that: [`Environment::files`] lists each resolver load as a
152//! [`FileEntry`], `is_dependency` as UnityPy sets it, and none is evicted.
153//! With it false ([`EnvOptions::batch`]) resolver loads are LRU
154//! dependencies, evictable, and in neither `files`, `objects` nor
155//! `container`, whatever `find_file`'s `is_dependency`: `PARITY.md`
156//! section 19.3's row "resolver loads under `keep_resolver_loads` false"
157//! (`find_file`'s and a dereference's; review round 12: no row listed
158//! them) and section 19.2's `SerializedFile.load_dependencies` row (a
159//! resource fallback's). An evicted dependency is opened again from its
160//! source by the next read that needs it (section 19.3's row "a dependency
161//! the LRU evicts"). [`Environment::load_dependency`], which UnityPy has
162//! no call for, never adds to `files`.
163//!
164//! **Locks.** The registry, the `files` list, the LRU and a file's
165//! `path_dict` are locked only to look up or publish; a file is loaded, an
166//! entry parsed and a resolver, fetcher or provider called with no lock held
167//! ([`crate::once`]; `tests/env/locks.rs` runs loads on another thread while a
168//! resolver or a fetch waits for them). The one
169//! caller code that may run under them is a [`crate::budget::MemoryBudget`]
170//! or [`crate::budget::PermitHome`] (a growth charged while a table is
171//! published, a replaced value's charge returned), whose contract forbids
172//! blocking and calling the core (plan section 3.16).
173//!
174//! Where UnityPy reaches the filesystem (`find_file`'s walk of `env.path`,
175//! `load_file`'s relative paths), the core asks a [`CabResolver`]: the core
176//! itself never opens a path it was not given.
177//!
178//! Owner: package D. Option constructors are implemented by the scaffold.
179
180pub mod pptr;
181pub mod registry;
182pub mod resource;
183
184mod container_index;
185mod deref;
186#[cfg(test)]
187mod gate;
188#[cfg(test)]
189mod heap;
190mod held;
191mod listing;
192#[cfg(test)]
193mod listing_tests;
194mod load;
195mod object_tree;
196pub use object_tree::MonoBehaviourError;
197mod paths;
198#[cfg(test)]
199mod preload_tests;
200#[cfg(test)]
201mod race_tests;
202mod reader;
203mod resource_data;
204mod revisit;
205mod split;
206#[cfg(test)]
207mod threads_tests;
208mod watch;
209
210use std::fmt;
211use std::io::Read;
212use std::path::{Path, PathBuf};
213use std::sync::atomic::{AtomicU64, Ordering};
214use std::sync::{Arc, RwLock};
215
216use bytes::Bytes;
217
218pub use container_index::{ContainerItem, ContainerItems};
219pub use deref::FileOccurrence;
220pub use listing::{FileEntries, FileEntry, Objects};
221pub use load::DependencyFile;
222pub use pptr::{DerefError, PPtr};
223pub use registry::{Cab, HeldCab, Keys, RawCab, Registry};
224pub use resource::{Candidate, SUFFIXES, resource_candidates};
225
226use crate::budget::DecodeBuf;
227use crate::error::{Error, LimitError, LimitKind};
228use crate::files::File;
229use crate::limits::{Limits, Meter};
230use crate::options::FileContext;
231use crate::raw::RawStr;
232use listing::Files;
233use registry::DepCache;
234
235/// When a container's entries are parsed.
236#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)]
237pub enum LoadMode {
238 /// At load, every entry, as UnityPy does.
239 #[default]
240 Eager,
241 /// On first use, once.
242 Lazy,
243}
244
245/// Bytes the caller fetches on demand (a batch engine's mmap or object store).
246pub trait Fetch: Send + Sync + fmt::Debug {
247 /// The file's bytes. A fetcher of a load the Environment makes itself
248 /// (an [`Environment::load_dependency`], any load of a source the
249 /// resolver returned, whether it is listed as a dependency or not, or a
250 /// reopen after an eviction; review round 12: a resolver's load listed
251 /// as no dependency was not retried) that takes them from the
252 /// dependency pool may refuse with the pool's refusal ([`Error::Budget`] naming
253 /// [`crate::budget::Reservation::Dependency`]): the Environment then
254 /// evicts the other dependencies and fetches once more, as for a
255 /// refusal of its own (plan section 3.13), whatever the fetcher did
256 /// before it returned the refusal: a refusal it passes on from a call of
257 /// its own into the Environment, which that call already retried, is
258 /// fetched once more too, since the core cannot tell it from the
259 /// fetcher's own (`registry::from_callers`). A caller's
260 /// [`Environment::load`] is no dependency's, and returns such a refusal
261 /// as it is.
262 fn fetch(&self) -> Result<Bytes, Error>;
263 /// Fetch bytes together with their reader rules. The default is the
264 /// memory reader; descriptor-backed fetchers can preserve path semantics.
265 /// Joined split files must continue to use the memory reader.
266 fn fetch_with_reader(&self) -> Result<(Bytes, crate::stream::ReaderKind), Error> {
267 self.fetch()
268 .map(|bytes| (bytes, crate::stream::ReaderKind::Memory))
269 }
270
271 /// Its size, when known without fetching. A size above
272 /// [`Limits::max_input_bytes`] refuses the source before it is fetched
273 /// ([`LimitKind::InputBytes`], the size stated as its value), whatever
274 /// the fetch would return; the bytes a fetch returns are checked against
275 /// the bound whatever was stated (the [`Source`] docs).
276 fn size_hint(&self) -> Option<u64> {
277 None
278 }
279}
280
281/// Something to load. Every kind is bounded by [`Limits::max_input_bytes`]:
282/// bytes longer than it, however they came, are refused
283/// ([`LimitKind::InputBytes`]) before anything detects, parses or keeps
284/// them, so a refused fetch's bytes drop at once, with any charge they
285/// carry.
286#[derive(Clone, Debug)]
287pub enum Source {
288 /// Bytes in memory, under a name: refused above the bound from their
289 /// length.
290 Bytes {
291 /// The name UnityPy would key it by (`stream_name`).
292 name: RawStr,
293 /// The bytes.
294 data: Bytes,
295 },
296 /// A path the core reads whole into memory, with [`read_path`]: refused
297 /// above [`Limits::max_input_bytes`], and charged to the input's meter
298 /// before it is read. (Not mapped: a mapping of a file another process
299 /// truncates faults, and the core takes no new dependency for it; a
300 /// caller who wants a mapping passes [`Source::Bytes`] or
301 /// [`Source::Fetch`], whose bytes are the caller's.)
302 Path(PathBuf),
303 /// A local path whose physical file and split companions must remain
304 /// beneath `root`. The restriction is retained if a dependency reopens.
305 ConfinedPath {
306 /// The original path spelling, also used as its key.
307 path: PathBuf,
308 /// The directory allowed to contain every opened file.
309 root: PathBuf,
310 },
311 /// Bytes fetched on demand: refused above the bound from the size the
312 /// fetcher states ([`Fetch::size_hint`]) before it is fetched, and from
313 /// the length of the bytes it returns.
314 Fetch {
315 /// The name.
316 name: RawStr,
317 /// The fetcher.
318 fetch: Arc<dyn Fetch>,
319 },
320}
321
322/// Reads `path` whole for [`Source::Path`]. A file whose length is above
323/// `limits.max_input_bytes` is refused before anything is read
324/// ([`LimitKind::InputBytes`]). The buffer is allocated once at the stat'd
325/// length, charged to `meter` first, and filled; the end of the file is then
326/// checked with a read into a fixed stack buffer, so a file that ends where
327/// its length says costs exactly its buffer (`heap_cost(len)` and the owner
328/// record, [`crate::budget::OWNER_COST`]), never a growth past it. Only a
329/// file that outlasts its stat'd length (one that grows while it is read, a
330/// pipe, a device) grows the buffer, charged as it grows, and the read stops
331/// at the bound. The returned bytes carry the charge.
332pub fn read_path(path: &Path, limits: &Limits, meter: &Arc<Meter>) -> Result<Bytes, Error> {
333 let mut file = std::fs::File::open(path)?;
334 let max = limits.max_input_bytes;
335 let len = file.metadata()?.len();
336 within_input_bytes(len, limits)?;
337 let mut out = DecodeBuf::new(meter, len, max, LimitKind::InputBytes)?;
338 // The stat'd length, into the buffer allocated for it (no growth).
339 let stated = usize::try_from(len).unwrap_or(usize::MAX);
340 let dst = out.grow_zeroed(stated)?;
341 let mut filled = 0;
342 while let Some(rest) = dst.get_mut(filled..).filter(|r| !r.is_empty()) {
343 match file.read(rest) {
344 Ok(0) => break,
345 Ok(n) => filled += n,
346 Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {}
347 Err(e) => return Err(e.into()),
348 }
349 }
350 out.truncate(filled);
351 // Past it, a stack buffer: the buffer grows only for bytes that exist.
352 let mut probe = [0_u8; 8192];
353 loop {
354 match file.read(&mut probe) {
355 Ok(0) => break,
356 Ok(n) => out.extend_from_slice(probe.get(..n).unwrap_or_default())?,
357 Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {}
358 Err(e) => return Err(e.into()),
359 }
360 }
361 Ok(out.into_bytes()?)
362}
363
364/// `len`, an input's length or the length its fetcher states for it
365/// ([`Fetch::size_hint`]), within [`Limits::max_input_bytes`]; above it,
366/// the input's [`LimitKind::InputBytes`] refusal. Every [`Source`] is
367/// checked by it before its bytes are detected (`registry::open_source`).
368pub(crate) fn within_input_bytes(len: u64, limits: &Limits) -> Result<(), Error> {
369 let max = limits.max_input_bytes;
370 if len > max {
371 return Err(Error::Limit(LimitError::refused(
372 LimitKind::InputBytes,
373 len,
374 max,
375 )));
376 }
377 Ok(())
378}
379
380/// Finds a file the registry does not hold, where UnityPy looks in
381/// `env.path`: `find_file`'s fallback after `get_cab`
382/// (`environment.py:310-354`), and each `load_file(name, True)` of a
383/// resource read's fallback (`files/SerializedFile.py:330-349`).
384pub trait CabResolver: Send + Sync + fmt::Debug {
385 /// The source for `name`, or `None` when there is none. `name` is one of
386 /// three kinds, each as UnityPy passes it on (review round 9: these docs
387 /// named only the first and the third):
388 /// - `find_file`'s name ([`Environment::find_file`]): for a dereference,
389 /// the external's path cleaned as `PPtr.deref` cleans it
390 /// ([`crate::pystr::clean_external_path`]: a leading `archive:/`, then
391 /// `assets/`, dropped, the last `/` component kept, lower-cased); for
392 /// a caller's own call, the name the caller gave, as it is. UnityPy
393 /// opens `env.path` joined with it when that exists; else it takes,
394 /// from the files a walk of `env.path` lists, the first whose walked
395 /// path is the name, else the first whose walked path ends with it, a
396 /// plain suffix with no separator boundary (beside `sub/ab.assets`, a
397 /// root `xab.assets` answers `ab.assets`), else the first whose
398 /// walked path, its basename lower-cased, ends with the name's
399 /// simplified name, which it opens by that lower-cased path, so on a
400 /// case-sensitive file system only a basename all lower case on disk
401 /// is found that way; a miss raises `FileNotFoundError`. Its test of
402 /// the simplified name alone never matches (it compares a bare name
403 /// with the walk's full paths). `environment.py:326-354`; `PARITY.md`
404 /// section 3's `Environment.find_file` row has the whole account (the
405 /// walk built once, links, prefixes).
406 /// - In a resource read's fallback ([`Environment::resource`]), first
407 /// each of the reading file's externals' stored path, as it is, not
408 /// cleaned (`archive:/CAB-x/CAB-x`, `Assets/Level/Thing.asset`:
409 /// `files/SerializedFile.py:339-340`),
410 /// - then each of its four resource candidates ([`resource_candidates`]:
411 /// the resource path's basename and the three names made from its
412 /// stem, `helpers/ResourceReader.py:11-18`, `SerializedFile.py:342-349`).
413 ///
414 /// UnityPy's `load_file` opens a name of either of the last two kinds as
415 /// a path when one exists there; else, a relative one, joined to
416 /// `env.path`; else the case-insensitive match `find_sensitive_path`
417 /// finds under `env.path`; else nothing (`environment.py:110-137`).
418 fn resolve(&self, name: &[u8]) -> Option<Source>;
419
420 /// What `find_file` gets for `name` (the first kind above), where the
421 /// registry has no CAB of it: [`Self::resolve`]'s source by default, a
422 /// miss `None`. UnityPy's walk answers a miss two ways
423 /// (`environment.py:330-354`): `None` where `load_file` cannot open the
424 /// path it matched (a broken link, or a basename it lower-cased, which
425 /// on a case-sensitive file system names no file), and
426 /// `FileNotFoundError` where it matched nothing. A resolver that tells
427 /// them apart (the bindings') returns the second as
428 /// `Err(Error::NotFound(NotFound::Cab(name)))`
429 /// ([`crate::error::NotFound::Cab`]): [`Environment::find_file`] returns
430 /// it as it is, and a dereference, which raises `FileNotFoundError`
431 /// either way (`classes/PPtr.py:80-85`), fails with
432 /// [`DerefError::NotFound`], which a container preload skips as
433 /// UnityPy's skips every `FileNotFoundError`. Any other error fails the
434 /// call that asked. An error returned here is the caller's code's, a
435 /// pool refusal in it counted as it enters the core and retried once,
436 /// as a fetcher's is ([`Fetch::fetch`]): [`Environment::find_file`], a
437 /// dereference and the container pass's preload each evict the other
438 /// dependencies and ask once more (review round 12: only a
439 /// dereference did).
440 fn find_file(&self, name: &[u8]) -> Result<Option<Source>, Error> {
441 Ok(self.resolve(name))
442 }
443}
444
445/// Environment options.
446#[derive(Clone, Debug)]
447#[non_exhaustive]
448pub struct EnvOptions {
449 /// Shared by every file loaded.
450 pub context: Arc<FileContext>,
451 /// When container entries are parsed.
452 pub load: LoadMode,
453 /// The dependency LRU's bound, in bytes of file data. `u64::MAX` with
454 /// [`Self::keep_resolver_loads`] (as [`Self::unitypy`] sets them) evicts
455 /// nothing, not even for a pool refusal, which then fails the read; any
456 /// other setting may evict (a pool refusal evicts the others and retries
457 /// once), and [`Environment::container_path`] then refuses a
458 /// dependency's object.
459 pub dep_cache_bytes: u64,
460 /// The `find_file` fallback; `None` finds nothing.
461 pub resolver: Option<Arc<dyn CabResolver>>,
462 /// Whether the files the resolver loads (for `find_file` and a resource
463 /// read's fallback) are kept in [`Environment::files`] as UnityPy keeps
464 /// them, and never evicted (module docs). False keeps them in the
465 /// dependency LRU only, whatever `find_file`'s `is_dependency`, a
466 /// `PARITY.md` divergence (section 19.3, "resolver loads under
467 /// `keep_resolver_loads` false"; section 19.2,
468 /// `SerializedFile.load_dependencies`).
469 pub keep_resolver_loads: bool,
470}
471
472impl EnvOptions {
473 /// UnityPy's behaviour: eager, nothing evicted, no resolver, resolver
474 /// loads kept in `files` as UnityPy keeps them.
475 #[must_use]
476 pub fn unitypy() -> Self {
477 Self {
478 context: Arc::new(FileContext::unitypy()),
479 load: LoadMode::Eager,
480 dep_cache_bytes: u64::MAX,
481 resolver: None,
482 keep_resolver_loads: true,
483 }
484 }
485
486 /// The batch engine's: lazy, a 512 MiB dependency cache, resolver loads
487 /// in that cache only (evictable, not in `files`).
488 #[must_use]
489 pub fn batch(context: Arc<FileContext>) -> Self {
490 Self {
491 context,
492 load: LoadMode::Lazy,
493 dep_cache_bytes: 512 << 20,
494 resolver: None,
495 keep_resolver_loads: false,
496 }
497 }
498}
499
500impl Default for EnvOptions {
501 fn default() -> Self {
502 Self::unitypy()
503 }
504}
505
506/// Counters, for the batch report and tests.
507#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
508#[non_exhaustive]
509pub struct EnvStats {
510 /// Files loaded by the caller.
511 pub loaded: u64,
512 /// Entries parsed (eagerly or on first use).
513 pub entries_parsed: u64,
514 /// Dependencies loaded through the resolver.
515 pub dependencies_loaded: u64,
516 /// Dependencies evicted from the LRU.
517 pub dependencies_evicted: u64,
518 /// Bytes the LRU holds now.
519 pub dependency_bytes: u64,
520 /// Preload table pointers the container passes dereferenced: each index
521 /// of a file's table once per walk, however many rows' spans cover it,
522 /// and once more, at the next row whose span reaches it or later in the
523 /// row, each time a load (the walk's own or another thread's) has
524 /// changed a key its external's lookup read ([`Environment::container`]).
525 pub preload_dereferences: u64,
526 /// Of those, the ones that looked their external up: a walk keeps an
527 /// answer until a key its lookup read changes, by any thread
528 /// ([`Environment::container`]).
529 pub preload_lookups: u64,
530}
531
532struct EnvInner {
533 options: EnvOptions,
534 files: RwLock<Files>,
535 registry: Registry,
536 cache: Arc<DepCache>,
537 loaded: AtomicU64,
538 eager_parsed: AtomicU64,
539 preload_derefs: AtomicU64,
540 preload_lookups: AtomicU64,
541}
542
543/// UnityPy's `Environment`. A cheap handle: clones share one environment.
544#[derive(Clone)]
545pub struct Environment {
546 inner: Arc<EnvInner>,
547}
548
549impl fmt::Debug for Environment {
550 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
551 f.debug_struct("Environment")
552 .field("options", &self.inner.options)
553 .finish_non_exhaustive()
554 }
555}
556
557/// What a dereference reached: a retained object, a registered CAB, or a
558/// file (a sibling, a `files` entry, one the resolver loaded).
559#[derive(Clone)]
560enum Found {
561 Cab(Cab),
562 File(File),
563 Reused(crate::ObjectRef),
564}
565
566/// The name UnityPy keys a source by: the caller's name, or a path's string.
567fn source_name(source: &Source) -> RawStr {
568 match source {
569 Source::Bytes { name, .. } | Source::Fetch { name, .. } => name.clone(),
570 Source::Path(p) | Source::ConfinedPath { path: p, .. } => {
571 let name = p.as_os_str().as_encoded_bytes();
572 RawStr::from(split::base(name).unwrap_or(name))
573 }
574 }
575}
576
577impl Environment {
578 /// An empty environment.
579 #[must_use]
580 pub fn new(options: EnvOptions) -> Self {
581 let cache = Arc::new(DepCache::new(
582 options.dep_cache_bytes,
583 !never_evicts(&options),
584 ));
585 Self {
586 inner: Arc::new(EnvInner {
587 options,
588 files: RwLock::new(Files::default()),
589 registry: Registry::default(),
590 cache,
591 loaded: AtomicU64::new(0),
592 eager_parsed: AtomicU64::new(0),
593 preload_derefs: AtomicU64::new(0),
594 preload_lookups: AtomicU64::new(0),
595 }),
596 }
597 }
598
599 /// Its options.
600 #[must_use]
601 pub fn options(&self) -> &EnvOptions {
602 &self.inner.options
603 }
604
605 /// The CAB registry.
606 #[must_use]
607 pub fn registry(&self) -> &Registry {
608 &self.inner.registry
609 }
610
611 fn ctx(&self) -> &Arc<FileContext> {
612 &self.inner.options.context
613 }
614
615 /// Counters.
616 #[must_use]
617 pub fn stats(&self) -> EnvStats {
618 let cache = &self.inner.cache;
619 EnvStats {
620 loaded: self.inner.loaded.load(Ordering::Relaxed),
621 entries_parsed: self
622 .inner
623 .eager_parsed
624 .load(Ordering::Relaxed)
625 .saturating_add(self.inner.registry.parsed()),
626 dependencies_loaded: cache.loaded.load(Ordering::Relaxed),
627 dependencies_evicted: cache.evicted.load(Ordering::Relaxed),
628 dependency_bytes: cache.bytes(),
629 preload_dereferences: self.inner.preload_derefs.load(Ordering::Relaxed),
630 preload_lookups: self.inner.preload_lookups.load(Ordering::Relaxed),
631 }
632 }
633}
634
635impl Environment {
636 /// [`registry::with_room`] on this Environment's LRU for the work of one
637 /// of its calls (a lookup, a dereference, a resource read: plan section
638 /// 3.13), keeping no dependency cached: the handles the call reads
639 /// through are held while it runs, by the caller (a dependency's
640 /// top-level handle, as [`Self::load_dependency`] says) and by the call
641 /// itself (the asking file's container, `resource_data.rs`'s
642 /// `top_bundle`), so an eviction drops only the LRU's.
643 fn room<T, E: registry::PoolRefusal>(&self, f: impl FnMut() -> Result<T, E>) -> Result<T, E> {
644 registry::with_room(Some(&self.inner.cache), None, f)
645 }
646}
647
648/// Whether the options evict nothing ([`EnvOptions::dep_cache_bytes`]).
649fn never_evicts(options: &EnvOptions) -> bool {
650 options.keep_resolver_loads && options.dep_cache_bytes == u64::MAX
651}
652
653#[cfg(test)]
654mod tests;