whiteoutlib

Read and write the file formats Blizzard games use for their art assets — from Rust.

If you are writing a model viewer, an asset converter, a datamining tool or a modding pipeline, this crate turns those formats into ordinary Rust data you can walk, edit and write back out.

• Models — MDX (Warcraft III), M2 (World of Warcraft), M3 (StarCraft II, Heroes of the Storm): geometry, bones, animation tracks, materials, cameras, particle emitters.
• Textures — BLP, DDS, PNG, JPEG, BMP, TGA, TIFF, GIF: decode, convert between pixel formats including the BCn family, generate mipmaps, re-encode.
• Archives — CASC and MPQ, so you can read assets straight out of a game installation.

use whiteout::textures::{BlpParser, PixelFormat, PngWriter};

// A Warcraft III texture, converted to PNG.
let blp = std::fs::read("Textures/Arthas.blp")?;
let mut texture = BlpParser::new().parse(&blp).expect("not a BLP");
texture.convert_to(PixelFormat::RGBA8);
std::fs::write("arthas.png", &PngWriter::new().write(&texture))?;
# Ok::<(), std::io::Error>(())

Vertex and pixel data is borrowed directly out of the underlying buffers, so reading a model's geometry copies nothing — see Reading models.

This crate wraps WhiteoutLib, a C++20 library, and is generated from the same headers it wraps, so the two cannot drift apart. The C++ is bundled and built for you.

Installing

cargo add whiteoutlib --features casc,mpq

The package is whiteoutlib (the short name was taken on crates.io), but the library it provides is whiteout — so imports read:

use whiteout::mdx::Parser;

The C++ library is bundled and built for you, so this needs CMake and a C++20 compiler on the machine — and the first build takes a few minutes. Nothing else is required.

To link a prebuilt library instead, turn the default vendored feature off and point at it:

cargo add whiteoutlib --no-default-features --features casc,mpq
WHITEOUT_LIB_DIR=/path/to/lib cargo build      # add WHITEOUT_STATIC=1 for the static archive

build.rs resolves in that order: WHITEOUT_LIB_DIR (always wins), then the vendored build, then pkg-config.

The casc and mpq features mirror the CMake WHITEOUT_ENABLE_* options. Under vendored they configure the bundled build; against a prebuilt library they must match how it was configured, or linking fails — deliberately, rather than silently missing symbols.

Linking from another crate

This crate declares links = "whiteout_native": it is the single owner of the native library, and cargo rejects any other crate in the graph that also claims to provide it. Build your own copy of WhiteoutLib alongside this one and you get two sets of identically-mangled C++ symbols — which links on MSVC, silently keeping one definition of each, and quietly becomes an ODR violation the moment the two versions drift. Depend on this crate instead.

Calling in from Rust needs nothing extra; linking this crate's rlib pulls the native library in. A crate with its own C++ to compile can read what was built from build.rs, via the environment cargo sets for dependents of a links crate:

The HAS_* pair reports the configuration the library was actually built with, so a dependent can match it rather than infer it from its own features. These keys are a compatibility surface — treat them as public.

They are emitted on every path that produces a usable library. INCLUDE is the one that can be missing: with a prebuilt WHITEOUT_LIB_DIR that has no include/ beside it, set WHITEOUT_INCLUDE_DIR to supply it.

Working on the bindings

From a checkout, build.rs finds the repository above the crate, so the vendored path works with no setup. For the faster prebuilt loop:

./scripts/build-rust.ps1              # codegen + cmake + fmt + clippy + test
./scripts/pack-rust.ps1 -Verify       # stage vendored sources + package

Reading models

use whiteout::mdx::{MDLXFormat, Parser};

let bytes = std::fs::read("units/human/footman/footman.mdx")?;
let model = Parser::new().parse(&bytes, MDLXFormat::MDX).expect("parse failed");

for geoset in model.geosets_iter() {
    let positions: &[whiteout::math::Vector3f] = geoset.vertex_positions();
    let faces: &[u16] = geoset.faces();
    println!("{} verts, {} indices", positions.len(), faces.len());
}
# Ok::<(), std::io::Error>(())

Vertex data is borrowed straight out of the C++ allocation — nothing is copied, in either direction:

# use whiteout::{math::Vector3f, mdx::Model};
# let mut model = Model::new();
model.resize_geosets(1);
let mut geoset = model.geosets_mut(0).unwrap();
geoset.set_vertex_positions(&[Vector3f::new(0.0, 0.0, 0.0)]);
geoset.vertex_positions_mut()[0].z = 1.0;   // writes into C++ memory

That is safe because the slice borrows the model: the compiler rejects a resize, a second view, or a drop of the owner while it is alive.

Implementing the library's interfaces

whiteout::interfaces holds the traits the library calls into — supply your own file system, HTTP client, or thread pool:

use whiteout::interfaces::{FileSystem, HostFileSystem};

struct MyFs;
impl FileSystem for MyFs {
    fn read_file(&self, path: &str) -> Option<Vec<u8>> {
        std::fs::read(path).ok()
    }
}

let fs = HostFileSystem::new(MyFs);

Send + Sync is required, not defensive: the library calls these from worker threads. Panics are contained at the boundary — the C ABI is compiled without exceptions, so an escaping panic would be undefined behaviour.

A pool can be handed to any call that takes one:

# use whiteout::interfaces::{HostWorkerPool, WorkerPool, WorkerTask};
# use whiteout::textures::{PixelFormat, Texture};
# struct Inline;
# impl WorkerPool for Inline {
#     fn submit(&self, task: WorkerTask) { task.run() }
#     fn wait_idle(&self) {}
#     fn thread_count(&self) -> usize { 4 }
# }
let pool = HostWorkerPool::new(Inline);
let tex = Texture::create_2d(PixelFormat::RGBA8, 64, 64, 1).unwrap();
let bc1 = tex.copy_as_format(PixelFormat::BC1, Some(&pool)).unwrap();

Errors

The C++ library does not throw and reports absence via std::optional. This binding follows suit: operations that can simply find nothing return Option, and Result is reserved for the few calls that produce a real diagnostic. Parser diagnostics are a list, not an error:

# use whiteout::textures::BlpParser;
let mut parser = BlpParser::new();
let texture = parser.parse(b"not a blp");     // -> None
for issue in parser.issues() {
    eprintln!("{issue}");
}

Layout verification

Value types such as math::Vector3f are #[repr(C)] mirrors of their C++ counterparts and cross the boundary with no conversion. Sizes are pinned by const assertions at compile time; to also check the library you linked:

whiteout::math::check_abi()?;
# Ok::<(), whiteout::Error>(())

Regenerating

Everything except interfaces.rs, support.rs and lib.rs is generated. Do not edit those files:

python -m tools.codegen.codegen textures --backend rust-abi-header
python -m tools.codegen.codegen textures --backend rust-abi-source
python -m tools.codegen.codegen textures --backend rust-math
python -m tools.codegen.codegen <module> --backend rust

License

BSD-3-Clause.