Skip to main content

Module limits

Module limits 

Source
Expand description

Hard bounds on what untrusted input can make the library allocate or do, and the one meter that enforces the memory bound.

UnityPy has none of these: where it would exhaust memory or the C stack, unityrs stops with crate::Error::Limit (or the walker’s other kind) and PARITY.md lists the case as a safety divergence (plan section 3.15).

One meter per input. Everything the core allocates in proportion to a size or count the input chose is charged to the input’s Meter before it is allocated: the input’s own bytes when the core reads a path, every block decode (a first decode, a lazy re-decode, a measuring decode, a decode after eviction), every metadata table and typetree row of every entry parse (an entry parsed twice, or two directory rows over the same bytes, pay twice), every compiled tree, every parsed value and every writer output. A charge the meter refuses allocates nothing. The charge is a Charge that lives inside what it paid for, so the bytes return when that drops: a crate::Parsed carries the charge for its values, a decoded buffer’s bytes::Bytes the charge for the buffer. So what one input holds at any moment, retained values included, is at most Limits::max_input_memory in the unit the meter charges (heap_cost, a request in glibc’s chunks), plus the constant overhead plan section 3.15 states; in glibc’s own chunks a live allocation below 128 KiB may hold up to CHUNK_SLACK bytes more than its charge, at most half again of it (heap_cost’s docs), so the heap such an input holds is at most half again of that limit, plus the constant. A top-level load makes the meter; every file inside it (a bundle’s entries, nested bundles) charges the same one.

A meter is one open of its input. Its memory is the input’s: an account every open of the input shares, so the bound holds across opens. Its work (Limits::max_decode_work) is the open’s own: the Environment opens an evicted dependency again on Meter::reopen, a new meter on the same account with a fresh allowance, while a handle of the old open that someone still holds (a nested bundle a resource read kept) goes on counting against the old meter’s, which nothing resets.

One accounting path. The meter books every charge it grants with the load’s MemoryBudget as well (crate::FileContext::budget), so a batch engine’s budget sees exactly what every input holds, and a refusal says which side refused: ChargeError::Limit (this input’s bound) or ChargeError::Budget (the shared budget, tagged with the meter’s Reservation).

The other bounds are on work and nesting, per object read or per input.

Owner: the scaffold (implemented). Packages read these fields and charge through these types; a new limit is added only through the integration package.

Structs§

Charge
Bytes charged to a Meter (and booked with its budget), returned when the charge drops. It rides inside whatever it paid for.
Limits
Bounds on allocation, recursion and work per input.
Meter
One open of an input: what the input holds, against Limits::max_input_memory, and what this open has done, against Limits::max_decode_work (module docs). Thread-safe: a lazily read bundle’s entries charge one meter from any thread. Shared as Arc<Meter>; every Charge keeps it alive.
MeteredVec
A Vec whose every growth is charged to a Meter before it happens (its capacity, at heap_cost). Parsers build every table this way, so a table can never hold more than the meter granted, however many records the input declares; Self::into_parts hands the charge on to whatever keeps the table.
TreeTraversal
A bounded tree operation. Its staging stays charged until handoff or refusal.
WorkPast
An open’s work past its allowance, as Meter::work_used_up reports it: a status, not an error, so no refusal is made, or counted, until the core refuses new work for it.

Enums§

ChargeError
A charge the meter or the budget refused. Nothing was allocated.

Constants§

CHUNK_SLACK
The most a live allocation whose chunk is below 128 KiB holds past its heap_cost: glibc hands over a free chunk whole when what would be left is under its 32-byte minimum chunk, and chunks are multiples of 16, so what is left is 0 or 16 bytes (heap_cost’s docs). That is for an alignment of 16 bytes or less (malloc’s), which every allocation the core charges has.

Functions§

heap_cost
What one heap allocation of bytes costs, which is what a charge for it is: glibc’s chunk on x86-64. The request gets its 8-byte header and is rounded to 16, with a 32-byte minimum (request2size); a chunk of 128 KiB (glibc’s default M_MMAP_THRESHOLD) or more may be mmapped instead, which adds another 8 bytes and rounds to a 4 KiB page (sysmalloc’s ALIGN_UP(nb + SIZE_SZ, pagesize)), and that is the larger of the two. 0 for no allocation.
shrink_moves
Whether a buffer whose block holds cap bytes gives back its spare room, down to len bytes, by moving them to a new allocation rather than by asking the allocator to shrink the block: when the block may be mmapped (its chunk is 128 KiB or more) and the bytes left would be a heap chunk (below it). glibc’s realloc keeps a shrunk mapped block mapped (mremap_chunk), rounded up to a page, so 10 bytes cut from a 1 MiB buffer held a page against a heap_cost of 32 (review round 13; tests/heap_rounding.rs with the move undone: “10 bytes hold 4088”, its usable bytes and an 8-byte header). A new allocation below 128 KiB is a heap chunk, which heap_cost covers (CHUNK_SLACK aside). Both blocks exist while the bytes move, as they may while realloc moves them, so a caller charges both first.