unity-formats

A Rust library that reads the data files of Unity games: the SerializedFiles a built player keeps its scenes and assets in, and the meshes, textures, materials, shader programs, terrains and animation data inside them.

It reads SerializedFile containers (versions 21 and 22) and UnityFS bundles (AssetBundles, Addressables content, a player's data.unity3d), decodes their objects through the type trees a file stores or, for a player build that strips them, through a type tree package (a TPK file you supply), and follows pointers across a player's files and streamed .resS data, loose or bundled. On top of that it reads scenes (GameObjects, Transforms composed to world matrices, renderers, lights, cameras, colliders, terrains, ambient and fog settings), meshes (vertex streams, compressed meshes, skins, blend shapes), Texture2D pixels and mip chains, materials, a Shader's render state and compiled program variants, terrains, Avatars, AnimatorControllers and AnimationClips, and the Universal Render Pipeline's objects: a pipeline asset's renderers and settings, a renderer's features, layer masks and modes, the Decal and Full Screen Pass features' settings and a camera's URP data. It was written against Unity 2019.4 and Unity 6 players: SkateBIRD (2019.4.20f1), Sledding Game (6000.3.12f1), PEAK (6000.3.15f1) and Big Walk (6000.3.17f1).

Example

use std::path::Path;
use unity_formats::pipeline::read_urp_renderer;
use unity_formats::{TypeTreePackage, listing::list, mesh::UnityMeshData, player::Player};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // a type tree package for the game's Unity version, e.g. the one AssetRipper publishes
    let package = TypeTreePackage::parse(&std::fs::read("typetrees.tpk")?)?;
    // the game's `<Game>_Data` folder
    let player = Player::open(Path::new("Game_Data"), package)?;
    for scene in player.scenes()? {
        println!("{}: {}", scene.file, scene.name);
    }
    for entry in list(&player, Some(&["Mesh"]))?.entries.iter().take(3) {
        let mesh = UnityMeshData::read(&player, &entry.object)?;
        println!("{}: {} vertices", mesh.name, mesh.vertex_count);
    }
    // the Universal Render Pipeline asset every quality level draws with, and the renderer its cameras use by default
    if let Some(asset) = player.urp_asset(None)? {
        let settings = player.urp_settings(&asset)?;
        let default = settings.renderers.get(usize::try_from(settings.default_renderer)?);
        if let Some(r) = default.map(|p| player.resolve_pptr(&asset.file, *p)).transpose()?.flatten() {
            let renderer = read_urp_renderer(&player.behaviour(&r)?.1)?;
            println!("rendering mode {}, {} features", renderer.rendering_mode, renderer.features.len());
        }
    }
    Ok(())
}

cargo run --example inspect_player -- <Game_Data folder> <typetrees.tpk> is a runnable version of it. The API docs are on docs.rs (or cargo doc --open).

What it guarantees

Reads are bounds-checked, and a file it does not understand is refused with an Error that says what it found, for example SerializedFile version 19 (read: 21..=22), header file size 9758532, file has 30 bytes or <n> bytes wanted at offset <p>, <m> left. Only SerializedFile versions checked against real files are read; anything else is Error::Unsupported, never a guess.
Library code denies, by lint, panicking indexing, unchecked arithmetic and truncating casts, so a malformed file is an Error and not a panic. The test suite checks truncated headers and unsupported versions on synthetic bytes; it does not fuzz.
What a reader returns is what the file stores: values keep their stored units and Unity's coordinate frame (no handedness change, resampling or colour-space change), and pixels stay as stored. Where a module substitutes a default for a field the file may omit, its docs say so.
The library never writes to disk. It reads only the files in the folder it is given (a player's data folder and its Resources folder), the bundles it is told to open and the .resS files its objects name there (or the stream nodes of the bundle they are in); a streamed resource's path is reduced to a file name in that folder, or to a node of the bundle. unsafe code is denied.

What it does not do

It reads: nothing here writes SerializedFiles, bundles or any other Unity format.
It reads UnityFS bundles of format 8 with the header flags 0x243 (a block table stored with the directory, LZ4 or LZ4HC) whose blocks are stored or LZ4/LZ4HC; bundles with LZMA or LZHAM blocks, other format versions and Addressables catalogs are refused or not read. SerializedFiles of other container versions than 21 and 22 are refused.
It ships no type tree package: LZMA- and Brotli-compressed packages are refused (re-pack them uncompressed or LZ4).
A MonoBehaviour's script fields are returned as bytes for a file without type trees; a file that stores its trees (a bundle's) decodes them with Player::script_fields. [SerializeReference] managed references are refused.
Textures: eleven formats are named (Alpha8, ARGB4444, RGB24, RGBA32, DXT1, DXT5, BC4, BC5, BC6H, BC7, R8); crunched formats are refused, and no format is decoded to RGBA, only split into mip levels.
Meshes with compressed skin weights, legacy AnimationClips and clips with object-reference curves are refused.
A streamed AnimationClip whose frames name a curve past its stated curve count (some of PEAK's 6000.3.15f1 clips) is refused as Error::Unsupported, not guessed at.
It does not run, render or simulate anything: animation data, shaders and materials are read as stored, and what they mean is the caller's.
It contains no game files, and no tests need you to share any.

Running the tests

cargo test --workspace

runs the unit tests and the tests on synthetic bytes. The tests that read real games skip with a line naming what they need when GAME_DATA is not set. To run them, point the GAME_DATA environment variable at a TOML file naming your own copies, in the layout of primase-testdata:

[installs]
skatebird = 'D:/games/SkateBIRD'        # the folder that holds SkateBIRD_Data
[oracles]
root = 'D:/oracles'                     # <root>/unitypy/skatebird/census.json
[run]
require = 'gate'                        # optional: a missing input fails instead of skipping

The real-data tests are tests/serialized_census.rs, which parses every SerializedFile in SkateBIRD's data folder and compares each file's object table (path ids, class ids, offsets, sizes) with a census made by UnityPy (make it from your own install with pip install UnityPy and python tests/oracles/unitypy_census.py <SkateBIRD_Data folder> <oracles root>/unitypy/skatebird/census.json), and tests/bundles.rs, which reads every UnityFS bundle of PEAK, Sledding Game and Big Walk, compares each node's bytes and each serialized node's header, types, stored type trees, object table and externals with a census made by python tests/oracles/unitypy_bundles.py <bundles> > <oracles root>/unitypy/<game>/bundles.jsonl (UnityPy and lz4), and decodes every object by the tree its file stores. The bundle census takes minutes and the decoding about ten, with 4 threads; cargo run --release --example bundle_census -- <type tree package> <bundle>... prints the same census for one bundle. tests/inputs/README.md lists every input the tests can read (generated from tests/inputs/mod.rs).

CONTRIBUTING.md says how to build, and how to report a file this library cannot read.

The asset-conversion library primase is built on this crate.

License