vrm-runtime 0.1.0

Renderer-agnostic VRM 1.0 loader, validator, animation system, and runtime
Documentation
# vrm-runtime

[![Crates.io](https://img.shields.io/crates/v/vrm-runtime.svg)](https://crates.io/crates/vrm-runtime)
[![Documentation](https://docs.rs/vrm-runtime/badge.svg)](https://docs.rs/vrm-runtime)

A renderer-agnostic VRM 1.0 loader, validator, animation system, and runtime for Rust.

`vrm-runtime` keeps immutable avatar data in `AvatarAsset` and stores expressions,
LookAt, animation, constraints, and SpringBone simulation in independent
`AvatarRuntime` instances. It does not require a particular graphics API or game
engine.

[한국어 README](https://github.com/nanowater/vrm-runtime-rs/blob/main/README.ko.md)

## Highlights

- Strict VRM 1.0 loading with structured errors and optional permissive diagnostics
- Renderer-independent GPU and CPU deformation paths
- Expressions, LookAt, first-person visibility, node constraints, and SpringBone
- MToon, glTF PBR, unlit materials, emissive strength, and texture transforms
- Standalone `.vrma` loading, typed animation tracks, and humanoid retargeting
- Arbitrary consecutive `TEXCOORD_n` and `JOINTS_n`/`WEIGHTS_n` sets
- Validated PNG/JPEG images from GLB buffer views, data URIs, and relative file URIs
- Type-safe IDs and builders for programmatically assembled avatars

## API map

| Task | Start here |
| --- | --- |
| Load and inspect a VRM | [`AvatarAsset`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.AvatarAsset.html, [`LoadOptions`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.LoadOptions.html |
| Evaluate an avatar each frame | [`AvatarRuntime`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.AvatarRuntime.html |
| Feed a GPU renderer | [`GpuFrameView`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.GpuFrameView.html, [`GpuDraw`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.GpuDraw.html |
| Render deformed CPU meshes | [`FrameView::bake_cpu`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.FrameView.html#method.bake_cpu |
| Load and play VRMA | [`VrmAnimation`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.VrmAnimation.html, [`PlaybackOptions`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.PlaybackOptions.html |
| Inspect materials | [`Material`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.Material.html, [`EvaluatedMaterial`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.EvaluatedMaterial.html |
| Construct an asset in code | [`AvatarAssetBuilder`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.AvatarAssetBuilder.html, [`RenderPrimitiveBuilder`]https://docs.rs/vrm-runtime/latest/vrm_runtime/struct.RenderPrimitiveBuilder.html |

The same API is also grouped by responsibility in the crate's `animation`,
`avatar`, `material`, `render`, `runtime`, and `spring_bone` modules.

## Installation

```toml
[dependencies]
vrm-runtime = "0.1"
```

## Quick start

```rust,no_run
use vrm_runtime::{AvatarAsset, AvatarRuntime};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let asset = AvatarAsset::load("avatar.vrm")?;
    let mut runtime = AvatarRuntime::new(asset);

    runtime.set_expression("happy", 0.8)?;
    let frame = runtime.update(1.0 / 60.0)?;

    // GPU path: upload immutable vertices plus frame-local deformation data.
    for draw in frame.gpu().draws() {
        let _primitive = draw.primitive;
        let _morph_weights = draw.morph_weights;
        let _joint_matrices = draw.joint_matrices;

        if let Some(material) = draw.evaluated_material {
            let _renderer_ready_material = material;
        }
    }

    // CPU path: morphing, skinning, and world transforms are already applied.
    let meshes = frame.bake_cpu();
    println!("baked {} meshes", meshes.len());

    Ok(())
}
```

Controls persist between frames. Use `clear_expression`, `clear_bone_rotation`,
`clear_look_at_target`, or `reset` to remove them. Multiple runtimes can share one
`Arc<AvatarAsset>` without sharing animation or simulation state.

## Loading and diagnostics

Strict validation is the default. Invalid references, unsupported required
extensions, malformed accessors, and unsafe numeric values are returned as errors.

```rust,no_run
use vrm_runtime::{AvatarAsset, LoadOptions, ValidationMode};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let outcome = AvatarAsset::load_with_options(
        "avatar.vrm",
        LoadOptions {
            validation: ValidationMode::Permissive,
        },
    )?;

    for diagnostic in outcome.diagnostics {
        eprintln!(
            "{} at {}: {}",
            diagnostic.code, diagnostic.path, diagnostic.message
        );
    }

    Ok(())
}
```

`AvatarAsset::load` and `AvatarAsset::from_file` resolve percent-encoded relative
buffer and image URIs against the avatar file. `AvatarAsset::from_bytes` has no base
directory, so external references are rejected; GLB buffer views and data URIs remain
available. HTTP(S) retrieval is intentionally left to the application.

## Rendering model

Each `AvatarRuntime::update` returns a coherent frame with both rendering paths:

- `FrameView::gpu()` exposes static primitives, world matrices, morph weights, joint
  matrices, typed IDs, and renderer-ready dynamic material values.
- `FrameView::bake_cpu()` returns meshes with morphing, skinning, and world transforms
  already applied.
- `FrameView::evaluated_materials()` provides expression colors and UV transforms,
  effective HDR emissive values, and MToon UV animation state.
- `FrameView::to_owned()` creates a `FrameSnapshot` suitable for queues or another
  thread.
- `CameraMode` and first-person index selection apply VRM first-person visibility
  rules, including automatic head-mesh separation.

For a GPU path, process each `GpuDraw` in this order:

1. Add the weighted morph-target deltas from `morph_weights` to the base vertex.
2. When `skin_id` is present, blend `joint_matrices` with every
   `JOINTS_n`/`WEIGHTS_n` set. These matrices already produce world-space output;
   do not apply `world_matrix` again.
3. When `skin_id` is absent, transform the vertex by `world_matrix`.
4. Draw `indices(camera_mode)` with the primitive topology and
   `evaluated_material`.

Zero-total skin weights intentionally leave a vertex in its rest position. The CPU
path implements the same rules and is a useful reference when integrating a shader.

The crate does not create GPU resources or render pipelines, so applications
remain free to choose their graphics API and resource layout.

## VRM animation

`VrmAnimation` supports programmatic clips and standalone
`VRMC_vrm_animation` (`.vrma`) files.

```rust,no_run
use vrm_runtime::{AvatarAsset, AvatarRuntime, PlaybackOptions, VrmAnimation};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let avatar = AvatarAsset::load("avatar.vrm")?;
    let animation = VrmAnimation::load("motion.vrma")?;
    let mut runtime = AvatarRuntime::new(avatar);

    runtime.play(animation, PlaybackOptions::default())?;
    // Call update once per application frame with elapsed seconds.
    let frame = runtime.update(1.0 / 60.0)?;

    println!("{} draws", frame.gpu().draws().len());
    Ok(())
}
```

Loaded VRMA clips preserve STEP, LINEAR, and CUBICSPLINE interpolation; humanoid
rotation; Hips translation; expression weights; and LookAt channels. Playback
retargets rotations through the source and target T-poses. Hips translation is
scaled by avatar height by default and can be disabled with
`PlaybackOptions::scale_hips_translation`.

`PlaybackOptions::default()` plays at normal speed and loops. Use
`PlaybackMode::Once` for one-shot playback. `pause`, `resume`, `seek`,
`set_playback_speed`, `playback_time`, and `stop` control the active clip.

The complete [`vrma` example](https://github.com/nanowater/vrm-runtime-rs/blob/main/examples/vrma.rs)
evaluates a clip through one-shot playback at 60 Hz.

Programmatic clips use typed `Track<T>`, `Keyframe<T>`, and `BoneTrack` values:

The following example also requires `glam = "0.29"` in your dependencies.

```rust
use glam::Quat;
use vrm_runtime::{
    BoneTrack, HumanBone, Interpolation, Keyframe, Track, VrmAnimation,
};

fn main() -> Result<(), String> {
    let rotation = Track::new(
        Interpolation::Linear,
        vec![
            Keyframe::new(0.0, Quat::IDENTITY),
            Keyframe::new(1.0, Quat::from_rotation_y(0.5)),
        ],
    );

    let animation = VrmAnimation::new(1.0).with_bone_track(
        HumanBone::Head,
        BoneTrack::new().with_rotation(rotation),
    );

    animation.validate()?;
    Ok(())
}
```

## Programmatic assets

Use `AvatarAssetBuilder` to assemble a complete immutable asset. Renderer data can
be constructed with `RenderPrimitiveBuilder`, `Material`, `AvatarTexture`, and
`AvatarImage`. Primitive-local buffers are validated by
`RenderPrimitiveBuilder::build`; cross-object references and VRM invariants are
validated by `AvatarAssetBuilder::build`.

```rust
use vrm_runtime::{NodeId, RenderPrimitiveBuilder};

let primitive = RenderPrimitiveBuilder::new(
    NodeId::from_index(0),
    vec![[0.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
    vec![0, 1, 2],
)
.build()?;

assert_eq!(primitive.positions().len(), 3);
# Ok::<(), vrm_runtime::BuildError>(())
```

A complete `AvatarAssetBuilder` additionally requires consistent metadata,
humanoid nodes, object identifiers, and cross-object references. Its final
`build` call validates those relationships before returning an immutable asset.

## Supported specifications

| Specification or feature | Status |
| --- | --- |
| `VRMC_vrm` 1.0 metadata, humanoid, expressions, LookAt, first person | Supported |
| glTF 2.0 PBR materials, textures, skins, morph targets, and animation accessors | Supported |
| Arbitrary consecutive `TEXCOORD_n` and `JOINTS_n`/`WEIGHTS_n` | Supported |
| `KHR_texture_transform` | Supported |
| `KHR_materials_unlit` | Supported |
| `KHR_materials_emissive_strength` | Supported |
| `VRMC_materials_mtoon` 1.0 | Supported |
| Archived `VRMC_materials_hdr_emissiveMultiplier` | Supported with diagnostics |
| `VRMC_node_constraint` 1.0 | Supported |
| `VRMC_springBone` 1.0 | Supported |
| `VRMC_springBone_extended_collider` | Supported |
| `VRMC_vrm_animation` 1.0 | Supported |
| VRM 0.x | Not supported; loading returns an explicit error |

Runtime support can be queried through `Capabilities::current()` or
`AvatarAsset::capabilities()`.

## Examples and validation

From the repository root:

```text
cargo run --example basic -- <path/to/avatar.vrm>
cargo run --example vrma -- <avatar.vrm> <motion.vrma>
cargo test
cargo clippy --all-targets -- -D warnings
```

Tests that use a locally licensed avatar can be enabled without redistributing
the file:

```text
VRM_RUNTIME_TEST_SAMPLE=/absolute/path/to/avatar.vrm cargo test
```

The referenced file is read from outside the crate and is never included in the
published package.

The conformance suite includes synthetic malformed documents and official VRM 1.0
samples. Set `VRM_SPECIFICATION_DIR` to a local checkout of the VRM specification
repository to run the official sample tests. They are not intended to run from the
published crate archive.

## Project status

The crate is currently in its `0.1` development series. Public APIs may still change
before `1.0`; semantic-versioning rules will be followed for published releases.

## License

Licensed under either of the following, at your option:

- [Apache License, Version 2.0]https://github.com/nanowater/vrm-runtime-rs/blob/main/LICENSE-APACHE
- [MIT License]https://github.com/nanowater/vrm-runtime-rs/blob/main/LICENSE-MIT

Unless explicitly stated otherwise, contributions intentionally submitted for
inclusion in this project are licensed under the same terms.