Expand description
Hard bounds on what untrusted input can make the library allocate or do, and the one meter that enforces the memory bound.
UnityPy has none of these: where it would exhaust memory or the C stack,
unityrs stops with crate::Error::Limit (or the walker’s other kind) and
PARITY.md lists the case as a safety divergence (plan section 3.15).
One meter per input. Everything the core allocates in proportion to a
size or count the input chose is charged to the input’s Meter before
it is allocated: the input’s own bytes when the core reads a path, every
block decode (a first decode, a lazy re-decode, a measuring decode, a
decode after eviction), every metadata table and typetree row of every
entry parse (an entry parsed twice, or two directory rows over the same
bytes, pay twice), every compiled tree, every parsed value and every
writer output. A charge the meter refuses allocates nothing. The charge is
a Charge that lives inside what it paid for, so the bytes return when
that drops: a crate::Parsed carries the charge for its values, a
decoded buffer’s bytes::Bytes the charge for the buffer. So what one
input holds at any moment, retained values included, is at most
Limits::max_input_memory in the unit the meter charges (heap_cost,
a request in glibc’s chunks), plus the constant overhead plan section 3.15
states; in glibc’s own chunks a live allocation below 128 KiB may hold up
to CHUNK_SLACK bytes more than its charge, at most half again of it
(heap_cost’s docs), so the heap such an input holds is at most half
again of that limit, plus the constant. A top-level load makes the meter;
every file inside it (a bundle’s entries, nested bundles) charges the same
one.
A meter is one open of its input. Its memory is the input’s: an
account every open of the input shares, so the bound holds across opens.
Its work (Limits::max_decode_work) is the open’s own: the
Environment opens an evicted dependency again on Meter::reopen, a new
meter on the same account with a fresh allowance, while a handle of the
old open that someone still holds (a nested bundle a resource read kept)
goes on counting against the old meter’s, which nothing resets.
One accounting path. The meter books every charge it grants with the
load’s MemoryBudget as well (crate::FileContext::budget), so a
batch engine’s budget sees exactly what every input holds, and a refusal
says which side refused: ChargeError::Limit (this input’s bound) or
ChargeError::Budget (the shared budget, tagged with the meter’s
Reservation).
The other bounds are on work and nesting, per object read or per input.
Owner: the scaffold (implemented). Packages read these fields and charge through these types; a new limit is added only through the integration package.
Structs§
- Charge
- Bytes charged to a
Meter(and booked with its budget), returned when the charge drops. It rides inside whatever it paid for. - Limits
- Bounds on allocation, recursion and work per input.
- Meter
- One open of an input: what the input holds, against
Limits::max_input_memory, and what this open has done, againstLimits::max_decode_work(module docs). Thread-safe: a lazily read bundle’s entries charge one meter from any thread. Shared asArc<Meter>; everyChargekeeps it alive. - Metered
Vec - A
Vecwhose every growth is charged to aMeterbefore it happens (its capacity, atheap_cost). Parsers build every table this way, so a table can never hold more than the meter granted, however many records the input declares;Self::into_partshands the charge on to whatever keeps the table. - Tree
Traversal - A bounded tree operation. Its staging stays charged until handoff or refusal.
- Work
Past - An open’s work past its allowance, as
Meter::work_used_upreports it: a status, not an error, so no refusal is made, or counted, until the core refuses new work for it.
Enums§
- Charge
Error - A charge the meter or the budget refused. Nothing was allocated.
Constants§
- CHUNK_
SLACK - The most a live allocation whose chunk is below 128 KiB holds past its
heap_cost: glibc hands over a free chunk whole when what would be left is under its 32-byte minimum chunk, and chunks are multiples of 16, so what is left is 0 or 16 bytes (heap_cost’s docs). That is for an alignment of 16 bytes or less (malloc’s), which every allocation the core charges has.
Functions§
- heap_
cost - What one heap allocation of
bytescosts, which is what a charge for it is: glibc’s chunk on x86-64. The request gets its 8-byte header and is rounded to 16, with a 32-byte minimum (request2size); a chunk of 128 KiB (glibc’s defaultM_MMAP_THRESHOLD) or more may bemmapped instead, which adds another 8 bytes and rounds to a 4 KiB page (sysmalloc’sALIGN_UP(nb + SIZE_SZ, pagesize)), and that is the larger of the two. 0 for no allocation. - shrink_
moves - Whether a buffer whose block holds
capbytes gives back its spare room, down tolenbytes, by moving them to a new allocation rather than by asking the allocator to shrink the block: when the block may bemmapped (its chunk is 128 KiB or more) and the bytes left would be a heap chunk (below it). glibc’sreallockeeps a shrunk mapped block mapped (mremap_chunk), rounded up to a page, so 10 bytes cut from a 1 MiB buffer held a page against aheap_costof 32 (review round 13;tests/heap_rounding.rswith the move undone: “10 bytes hold 4088”, its usable bytes and an 8-byte header). A new allocation below 128 KiB is a heap chunk, whichheap_costcovers (CHUNK_SLACKaside). Both blocks exist while the bytes move, as they may whilereallocmoves them, so a caller charges both first.