basisu

A pure-Rust transcoder for Basis Universal supercompressed GPU textures. It reads a .ktx2 or .basis Basis texture and transcodes it, at load time, into whatever block-compressed format the target GPU supports: BC7 and the other BCn formats on desktop, ASTC and ETC on mobile, plus PVRTC, ATC, FXT1, and uncompressed RGBA fallbacks.

Basis Universal addresses GPU texture-format fragmentation with a compress-once, transcode-at-load model. You encode an image once into a compact intermediate, ship that single file everywhere, and at load time transcode it (without re-encoding) into the GPU-native format the device actually wants. This crate is the transcoder half of that pipeline, the runtime decode side. The encoder is the upstream basisu toolchain.

The implementation is a port of the reference C++ transcoder, verified byte-for-byte identical to it across a corpus of real textures.

Quick start

[dependencies]
basisu = "0.1"

The library name is basisu, so you write use basisu::...:

use basisu::{Transcoder, TargetFormat, DecodeFlags};

let bytes = std::fs::read("texture.ktx2")?;
let tex = Transcoder::new(&bytes)?;

// transcode mip level 0 to ASTC 4x4
let astc = tex.transcode(0, TargetFormat::Astc4x4Rgba, DecodeFlags::NONE)?;

Pick the target at runtime from what the GPU reports, so one shipped file serves every device:

let target = if gpu.supports_astc() {
    TargetFormat::Astc4x4Rgba
} else if gpu.supports_bc7() {
    TargetFormat::Bc7Rgba
} else {
    TargetFormat::Rgba32 // uncompressed fallback
};

for level in 0..tex.level_count() {
    let data = tex.transcode(level, target, DecodeFlags::NONE)?;
    upload_mip(level, &data);
}

How it works

A Basis texture stores its image data in a source codec chosen when the file is encoded:

• ETC1S is a supercompressed subset of ETC1. It is small (roughly 0.3 to 3 bits per pixel after the BasisLZ entropy stage) at low to medium quality, and transcodes very fast. A good default when size dominates.
• UASTC LDR is a high-quality 4x4 codec (8 bits per pixel) built on the ASTC bit layout. It is larger, but transcodes to high-quality targets like BC7 and ASTC with little quality loss. A good choice for normal maps and detail-heavy content.
• UASTC HDR is the high-dynamic-range variant: every block is a restricted ASTC HDR 4x4 block, transcoding to BC6H, ASTC HDR, or half-float/RGB-9E5 pixels.

Transcoding turns that source codec into a specific GPU format without a full decode and recompress. Each source block maps onto the target block format directly, which is fast enough to run while a texture loads. The same file becomes BC7 on a desktop, ASTC on a phone, and uncompressed RGBA where neither is available, with no per-platform asset builds.

You produce .ktx2 or .basis Basis files with the upstream encoder (basisu, or toktx from KTX-Software). This crate consumes them.

Supported formats

Every source codec the reference transcoder accepts is implemented: ETC1S, UASTC LDR, UASTC HDR 4x4, raw ASTC LDR (all 14 block sizes, 4x4 through 12x12), raw ASTC HDR 6x6, UASTC HDR 6x6 (the bitwise-compressed intermediate format), and XUASTC LDR (all 14 block sizes; the arithmetic- or zstd-coded intermediate format). All decode bit-exactly.

Every transcoder target for these codecs is implemented, 500 source/target combinations verified byte-identical to the reference:

Not every target is valid for every source. UASTC LDR does not transcode to PVRTC2, ATC, or FXT1; ETC1S covers all of them; raw ASTC LDR and XUASTC LDR reach the 15 LDR targets plus only their own block size's ASTC pass-through. The HDR sources transcode only to HDR targets (each reaching only its own block size's ASTC pass-through), and the LDR sources never do. Transcoder::supports(target) reports what a given texture can produce.

Container features: multi-mip textures, cubemaps, texture arrays, Zstandard-supercompressed levels (with the optional zstd feature), and shared (global) ETC1S codebooks across .basis files.

API

Transcoder::new(bytes) parses and prepares a texture. The rest queries it and transcodes:

• transcode(level, target, flags) -> Vec<u8> transcodes one mip level.
• transcode_image(level, layer, face, target, flags) selects a cubemap face or array layer.
• transcode_into(level, target, flags, out) writes into a caller-provided buffer without allocating the output (codecs that inherently need scratch, such as a Zstandard-compressed level or an intermediate stream, still allocate that internally, as the reference transcoder does).
• transcode_video_frame(state, level, frame, target, flags) decodes one ETC1S video frame; frames decode in order against a caller-owned VideoState.
• etc1s_codebook() lifts a self-contained ETC1S file's shared codebook out, and Transcoder::new_with_codebook(bytes, &codebook) opens a global-codebook .basis (one that references a codebook stored in a separate file) with it.
• output_size(level, target) returns the exact byte length a transcode produces.
• image_level_info(level), base_dimensions(), level_count(), layer_count(), face_count() describe the contents.
• source_format(), has_alpha(), supercompression(), is_video(), supports(target) report capabilities.

TargetFormat and DecodeFlags carry the same numeric values as the C++ transcoder_texture_format and basisd_decode_flags. The flags cover the reference decode options, such as HIGH_QUALITY for higher-quality UASTC to BCn transcodes and TRANSCODE_ALPHA_TO_OPAQUE.

Features

• std (default): the standard library. On std the crate has no required dependencies and no C or C++.
• zstd (default): decode Zstandard-supercompressed KTX2 levels via the pure-Rust ruzstd. Without it, such levels return Error::ZstdRequired (most ETC1S files use BasisLZ and need nothing here).

The crate also builds #![no_std] (with alloc) for embedded use: disable std and enable the libm feature. That is a niche path most consumers can ignore.

Relationship to the upstream project

This is a Rust port of the Basis Universal C++ transcoder, tracking upstream v2_1_0. It is not a binding: the published crate contains no C++. It is a derivative work: the port was written against the reference source, and the lookup tables are generated from the upstream table sources rather than transcribed by hand.

Status

• LDR transcoding is complete: all 21 target formats, both source codecs, every mip level and decode-flag variant, byte-identical to the reference.
• Container support: both .ktx2 and .basis, cubemaps, texture arrays, Zstandard levels, and ETC1S video are done. Video frames decode in order through transcode_video_frame with a caller-owned VideoState (P-frames replenish from the previous frame); the stateless entry points detect video (Transcoder::is_video) and reject it rather than decode P-frames as garbage.
• Shared (global) ETC1S codebooks are done: a .basis file that references a codebook stored in a separate file decodes through Transcoder::new_with_codebook (the codebook is lifted from the donor file with etc1s_codebook), byte-identical to the reference's set_global_codebooks. KTX2 has no equivalent (its codebook is always embedded). The stateless entry points still reject a global-codebook file, so a caller that has no codebook never decodes garbage.
• UASTC HDR 4x4 is complete: BC6H, ASTC 4x4 HDR, half-float, and RGB-9e5 output, byte-identical to the reference.
• Raw ASTC LDR sources (all 14 block sizes) are complete: the matching-block-size ASTC pass-through, all four uncompressed targets (with the deblocking filter), BC1/BC3/BC4/BC5, EAC R11/RG11, ETC1/ETC2, PVRTC1, and BC7. Every target is byte-identical to the reference over the full corpus.
• The 6x6 HDR sources are complete. Raw ASTC HDR 6x6: the 6x6 ASTC pass-through, BC6H (via the reference's real-time fast_encode_bc6h re-encoder, with the HIGH_QUALITY 2-subset search), half-float, and RGB-9E5 output. UASTC HDR 6x6 (the intermediate format, KTX2 supercompression scheme 4 or raw .basis slices): the run/solid/reuse/ block stream decompresses to standard ASTC 6x6 blocks, then reaches the same five targets.
• XUASTC LDR (all 14 block sizes) is complete: the three stream syntaxes (adaptive arithmetic, zstd side channels, and the hybrid) decompress to standard ASTC blocks, which reach the same-block-size ASTC pass-through and every LDR target, including the dedicated BC7 fast paths for the 4x4, 8x6, and 6x6 sources and the fast-BC7-disable decode flag (the reference's cDecodeFlagXUASTCLDRDisableFastBC7Transcoding, value 1024).
• All 500 (source, target) combinations in the conformance matrix are verified byte-identical over the corpus.

One documented divergence exists on malformed input only: the pure-Rust zstd decoder (ruzstd) rejects some corrupted zstd frames that libzstd tolerates. Valid files are unaffected.

Development

The repository is a workspace:

• basisu is the published crate (package name and library name are the same). Its tests and benches depend on repo-level files (corpus/, goldens/), so the published tarball excludes them; they run from the repository checkout.
• conformance is a dev-only crate. It compiles the vendored reference C++ as a golden oracle and runs the Rust transcoder against it, asserting zero differing bytes for every implemented format over the corpus. A committed goldens/manifest.tsv of SHA-256 hashes backs a pure-Rust replay test that needs no C++ toolchain.
• xtask drives the checks.

cargo xtask doctor        # what this machine can build/run, and how to fix the gaps
cargo xtask corpus        # fetch the full conformance corpus (~7800 files, ~220 MB)
cargo xtask verify        # conformance (byte-equality, needs a C++ compiler) and benchmarks
cargo xtask check         # fast pure-Rust path: fmt, clippy, golden replay
cargo xtask wasm          # wasm32 runtime tests under node
cargo xtask qemu          # bare-metal harnesses on emulated Cortex-M3 and M0
cargo xtask all           # check + verify + wasm + qemu (hard-fails on missing tooling)
cargo xtask ci            # the full local gate: all + MSRV + no_std + package dry-run
cargo xtask install-hooks # run `cargo xtask check` automatically on every git push
cargo xtask coverage      # transcoder-matrix coverage report
cargo xtask vendor <ref>  # re-vendor the reference C++ transcoder at an upstream git ref
cargo xtask bake-goldens  # re-bake goldens/manifest.tsv from the oracle (after vendoring)

Start with cargo xtask doctor: it reports every capability the gates use (C++ compiler, system zstd, corpus, QEMU, wasm tooling) with the install command for anything missing, on Linux and macOS alike. cargo xtask all deliberately fails, rather than skips, when tooling is missing, so a green all means the same thing on every machine.

The benchmarks in verify compare against a machine-local criterion baseline named pre-opt when one exists (create it with cargo bench -p basisu --bench transcode -- --save-baseline pre-opt); on a fresh checkout there is none, so they run and record timings without a regression comparison.

CI

This project uses no cloud CI: there are no workflow files, and nothing runs on third-party infrastructure. cargo xtask ci runs every gate (check, verify, wasm, qemu, MSRV, no_std, cargo package dry-run) on the machine it is invoked on, keeps going after a failure so one report shows the full picture, and writes a timestamped report to target/ci/. Run it after a work session and before publishing. cargo xtask install-hooks wires the fast check gate to git's pre-push hook, so every push is checked before it leaves the machine (git push --no-verify bypasses it in an emergency). Benchmarks are excluded from ci by design: timings are only meaningful against the machine-local baseline described above.

The corpus lives under corpus/: a small committed smoke/ set that every test falls back to, plus the full set fetched by cargo xtask corpus (three pinned, license-clean upstream sources). Running cargo test -p conformance --release after that fetch reproduces the full byte-equality result (the 500/500 conformance matrix).

Two environment notes, both surfaced by doctor:

• The oracle decodes Zstandard-supercompressed files only when a system zstd is found at build time (Debian/Ubuntu libzstd-dev, Fedora libzstd-devel, macOS brew install zstd, or ZSTD_INCLUDE_DIR/ZSTD_LIB_DIR). Without it, the build prints a warning and the conformance run counts those files as zstd-no-oracle skips; the committed goldens still gate them in pure Rust.
• The wasm tests (cargo xtask wasm) need cargo install wasm-bindgen-cli --version 0.2.118 (paired with the pinned wasm-bindgen-test 0.3.68) and node; the QEMU harnesses (cargo xtask qemu) need qemu-system-arm plus the two thumb targets (rustup target add thumbv7m-none-eabi thumbv6m-none-eabi).

Staying in sync with upstream

The vendored C++ oracle is the reference every format is proven against, so moving to a newer upstream release goes like this:

1. cargo xtask vendor <ref> re-vendors the transcoder-only source set at the given git ref, leaving a reviewable diff under conformance/csrc/basisu_transcoder/.
2. Pin the new ref in that directory's UPSTREAM_VERSION.
3. cargo xtask verify runs the Rust transcoder against the new oracle; any behavioral change surfaces as a failure localized to the exact (format, file) rows that diverged.
4. Port each divergence in the Rust code, repeating step 3 until it is green.
5. cargo xtask bake-goldens refreshes the committed goldens/manifest.tsv, so the toolchain-free cargo xtask check gate tracks the new reference.

License

Apache-2.0. The vendored C++ under conformance/csrc/basisu_transcoder/ is Basis Universal (Apache-2.0, (c) Binomial LLC) and is used only as the test oracle. It is not part of the published crate.