Skip to main content

Module stream

Module stream 

Source
Expand description

A bounded reader over a byte slice with the semantics of UnityPy’s EndianBinaryReader: the memory reader, EndianBinaryReader_Memoryview (streams/EndianBinaryReader.py:231-284), or the stream reader, EndianBinaryReader_Streamable (:287-324), which UnityPy reads a file it loads from a path with (ReaderKind; kind’s module docs list what it reads that way and its rules).

Container and metadata parsers read through it; the value walker has its own cursor (UnityPyBoost’s bounds rule differs, plan section 3.10), which takes the same ReaderKind. No method panics: every read UnityPy refuses is an Error::OutOfBounds, or the stream reader’s StreamRefusal as an Error::Malformed.

Positions are signed (plan section 3.3). A Python Position is an unbounded integer that UnityPy moves with plain arithmetic, so on the memory reader it can go negative (a negative node_count in parse_blob, file_size - metadata_size below format 9, Length - compressedSize for blocks-info at the end, a negative TypelessData length on the pure-Python path) or past the end, and the reads after it go on with Python’s rules, which this module’s helpers implement once and an oracle fixture pins (tests/fixtures/scaffold/reader.json, from tools/fixtures/scaffold.py):

  • py_read: read(n), a Python slice (negative indexes count from the end; a reversed range is empty); the position moves by n whatever the slice held;
  • py_unpack_from: every fixed-width read of 2, 4 or 8 bytes is struct.unpack_from(view, Position) (:340-343): a negative offset counts from the end, and the read must fit;
  • one-byte reads (read_u8, read_i8, read_bool) are unpack(fmt, read(1)) (:93-97, :126-127): the slice of py_read must hold exactly one byte, and the position has moved by 1 even when it raises (at -1, view[-1:0] is empty);
  • py_align: align_stream, Python’s floor modulo (-7 aligns to -4);
  • py_string_to_null: re.search clamps its start and end into the buffer, a hit sets the absolute position after the NUL, and a miss reads max_length bytes as a slice unless pos + max_length >= len.

The stream reader’s position is never negative and stops at the end; the same fixture pins its rules (its stream_ cases), which ReaderKind implements.

Owner: package C. The free functions and Reader::read_i32_array are the scaffold’s (implemented).

Structs§

Reader
A cursor over data, read by the rules of one of UnityPy’s readers (ReaderKind). Positions are absolute within data and signed, as a Python Position (module docs): on the memory reader every read follows the helper of its Python primitive at whatever position it is, negative or past the end; the stream reader’s position is never negative.
ShortArray
What read(N) returns where UnityPy keeps the bytes as they are (a GUID, a hash): the py_read slice, inline. It holds N bytes, or fewer where the slice is short: none at a position in -N..0 (the slice view[p:p+N] runs from the end back to the start, so it is empty, though the position moves to p + N), the rest at the end.

Enums§

Endian
Byte order of a reader or a file.
ReaderKind
Which of UnityPy’s readers a file is read with (module docs).
StreamRefusal
An exception the stream reader raises where the memory reader reads on (module docs). Its text is UnityPy’s message.

Constants§

STREAM_READER
The what of an crate::Error::Malformed a StreamRefusal becomes.
STRING_TO_NULL_WINDOW
read_string_to_null’s default max_length (EndianBinaryReader.py:275). Within the window the string runs to its NUL. With no NUL in the window it is cut at the window when more bytes follow, and raises otherwise.

Functions§

py_align
align_stream(n) (streams/EndianBinaryReader.py:155-156): pos + (n - pos % n) % n with Python’s floor modulo, so a negative position aligns towards zero (-7 to -4 for n = 4). n 0 leaves the position (Python would raise; no caller passes 0). Implemented by the scaffold.
py_read
The memoryview reader’s read(size) at pos over len bytes (streams/EndianBinaryReader.py:249-256), with Python’s arithmetic: size 0 reads nothing and leaves the position; size -1 reads to the end; any other size is the slice view[pos : pos + size], where a negative index counts from the end, indexes clamp to 0..=len and a reversed range is empty. Returns the byte range and the new position, pos + size (len for -1), which may be negative or past the end, as a Python Position can be. A bundle entry (File.py:78-81), parse_blob’s node area and string buffer, and the pure-Python reader’s TypelessData all read this way (plan section 3.6). Implemented by the scaffold.
py_string_to_null
read_string_to_null(max_length) of the memoryview reader (streams/EndianBinaryReader.py:275-284) at pos over data: the string’s byte range and the new position, or None where it raises (“String not terminated”). re.search(view, pos, pos + max_length) clamps both bounds into 0..=len (py_string_window), so a negative start searches from 0; a hit returns the bytes up to the first NUL in the window and sets the position after that NUL (an absolute index). With no NUL in the window it raises when pos + max_length >= len, and otherwise returns read(max_length), a slice from pos that may hold a NUL or nothing at all (py_string_to_null_found, which a caller that indexed the buffer’s NULs answers with). Implemented by the scaffold.
py_unpack_from
struct.unpack_from(fmt, view, pos) for a format of n bytes over len bytes (CPython’s s_unpack_from): the start index of the bytes it reads, or None where it raises struct.error. A negative pos counts from the end and must leave the whole read before the end (pos + n <= 0) and inside the buffer (pos + len >= 0). Every fixed-width read of 2, 4 or 8 bytes UnityPy’s memoryview reader makes through its own methods (reader.read_int()) takes this path, then moves the position by n (only on success). The pure-Python typetree reader’s primitive nodes do not: they read through the base class’s unpack(fmt, read(n)) (walk::python, plan 3.10). Implemented by the scaffold.