Skip to main content

Crate rm2k

Crate rm2k 

Source
Expand description

§rm2k-lib

Reads and writes the binary formats RPG Maker 2000 and 2003 use: LDB (database), LMU (map), LMT (map tree) and LSD (save). A from-scratch Rust rewrite of liblcf, built around three goals: run free-standing under no_std, be perfectly round-trip, be BLAZINGLY 🦀 FAST 🔥🔥

Round-trip correctness is verified byte-for-byte against real RPG Maker games - see Verification below.

Ports:

  • C API - C bindings, installable via cargo-c.
  • WASM - WebAssembly bindings.

§Installation

cargo add rm2k-lib

§Feature tiers

FeaturePulls inGives you
(none)nothingThe wire codec, chunk framing, and allocation-free event iteration. Runs on no_std with no allocator at all.
allocallocTyped structs - Database, Map, TreeMap, Save - and loading/saving them.
std (default)alloc, stdIoSink over std::io::Write.
serdealloc, serdeSerialize/Deserialize on every schema struct, enum and flag bundle. See Serde.

§Quick example

use rm2k::file;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let bytes = std::fs::read("RPG_RT.ldb")?;
    let loaded = file::load_database(&bytes)?;

    for actor in loaded.value.actors.iter() {
        println!("{}", actor.name.as_utf8().unwrap_or("<non-UTF-8 name>"));
    }
    Ok(())
}

§Strings

This crate never transcodes text, and never guesses an encoding. RPG Maker 2k/3 stores strings in whatever legacy codepage the authoring machine used - Shift-JIS for Japanese games, CP1252 for western ones - and an LCF file contains no marker recording which. So strings come out as &[u8], exactly as they sit in the file. Decode with whatever you prefer once you know the codepage, typically from RPG_RT.ini’s [EasyRPG] Encoding key.

§Leniency

Real files are frequently malformed and RPG Maker itself loads them anyway, so this crate does too. The difference is that recoveries are reported: pass a DiagSink and you get a structured list of what was wrong and where. Pass StrictSink to turn the first problem into an error.

Chunks this schema does not know are kept, not dropped. RPG Maker’s forward-compatibility story is “write chunks the reader may not understand”, so real files carry ids from later RPG_RT builds, EasyRPG extensions and third-party editors. Each one is stored verbatim in its struct’s unknown_chunks, anchored next to the last field that was recognised, and written back in that position - so byte-exactness does not depend on the schema being complete. They are still reported as DiagKind::UnknownChunk; preserved is not understood.

§Serde

With the serde feature, the schema’s structs, enums and flag bundles derive Serialize and Deserialize, so any serde format can read and write a loaded file. The rm2k-wasm bindings and the to_yaml example both use it. Each struct becomes an object keyed by its schema field names (system.system_name), each list becomes an array, and each string becomes escaped text: printable ASCII stays as-is, \ becomes \\, and every other byte becomes \xNN. Human-readable formats (JSON, YAML, WASM’s JS values) get that text; binary formats get the raw bytes. src/serde/mod.rs has the full rules.

§Examples

ExampleWhat it shows
examples/fixture_check.rsByte-exact round-trip checking against real game files on disk.
examples/dump_text.rsListing the message and choice strings in each map’s event commands. The event walk borrows from the file buffer, but each map is still loaded to read its header.
examples/to_yaml.rsConverting a database to YAML with the serde tier (cargo run --example to_yaml --features serde -- game.ldb).
capi/examples/to_json.cConverting a database’s named lists to JSON from C (via yyjson, vendored next to the example), using only the C API’s owned/“alloc” tier.

Each example has its own README with usage, output semantics, and (for the C ones) build steps: examples/README.md for the Rust examples, capi/examples/README.md for to_json.c. Run a Rust example with cargo run --release --example <name> [--features ...] -- <args>; the C example is not wired into Cargo.

§Verification

examples/fixture_check.rs (usage) loads every .ldb/.lmu/.lmt/.lsd file in a directory, writes it straight back out, and compares bytes. Run against three real, unmodified RPG Maker games (RPG Maker 2000 and 2003, 419 files across all four formats): 419/419 byte-exact, zero diagnostics. The C API gets the same treatment independently in capi/tests/test_c_api.c (build steps), round-tripping a real 262 KB database through the FFI boundary byte-for-byte.

The games in question:

benches/ has a real, measured comparison against liblcf: parsing RPG_RT.ldb is ~6.2x faster, and parsing a .lmu map file is ~8.8x faster (AMD Ryzen 5 2600, in-memory parse only, see benches/README.md for the full methodology, build steps, and raw numbers). Speed was not the goal, though.

§Design notes

  • Zero-copy core tier. Parsing borrows slices out of the input buffer; nothing is copied unless you ask for it (e.g. via IntoOwned::into_owned()).
  • Cleaner codebase. No legacy C++ shit. Boilerplate enums are generated by declarative macro_rules! macros, not some external Python pipelines.
  • Setup is opt-in, not automatic. RPG_RT’s post-load fixups (resolving -1 sentinels, padding stat curves) change the byte representation, so running them automatically on load would make a load/save cycle non-idempotent. Call Setup::setup explicitly when you want them.
  • Diagnostics instead of a global error string. Recoverable problems are reported through a DiagSink you provide, not a process-global mutable buffer.
  • Round-trip fidelity does not depend on schema completeness. Every struct keeps the chunks no field claimed and re-emits them where they were found, so a file using ids this schema has never seen still round-trips byte-for-byte. See Leniency.

§Acknowledgements

Most of the RPG Maker 2k/3 format handling was written referencing liblcf, a part of the EasyRPG project, a MIT-licensed library. However, the implementation that handles the format itself is very different from liblcf, both because Rust enforces different idioms and because the logic was made with free-standing usage in mind. This work wouldn’t be possible without them.

yyjson (yyjson.h/yyjson.c, MIT licensed) is vendored in capi/examples/yyjson for the to_json.c example, its license is alongside it.

§Support

Me, the maintainer of this project, is a poor college student from Eastern Europe.

If you could, please consider supporting us through:

Even if you don’t, we will still continue working on the project anyways.

§License

Licensed under WTFPL.

Re-exports§

pub use cursor::Chunk;
pub use cursor::ChunkIter;
pub use cursor::Ctx;
pub use cursor::Cursor;
pub use cursor::ListItem;
pub use cursor::ListWalker;
pub use diag::DiagKind;
pub use diag::DiagSink;
pub use diag::Diagnostic;
pub use diag::NullSink;
pub use diag::SliceSink;
pub use diag::StrictSink;
pub use engine::Engine;
pub use engine::SaveOpt;
pub use error::ReadError;
pub use error::WriteError;
pub use event::EventCommandIter;
pub use event::EventCommandRef;
pub use event::MoveCommandIter;
pub use event::MoveCommandRef;
pub use field::BitArray;
pub use field::DbStr;
pub use field::LcfField;
pub use field::LcfIndexed;
pub use field::LcfStruct;
pub use field::List;
pub use field::Packed;
pub use field::UnindexedList;
pub use field::UnknownChunk;
pub use field::UnknownChunks;
pub use raw::DatabaseVersion;
pub use raw::EmptyBlock;
pub use raw::Equipment;
pub use raw::EventCommand;
pub use raw::EventCommandList;
pub use raw::MoveCommand;
pub use raw::MoveCommandList;
pub use raw::Parameters;
pub use raw::Rect;
pub use raw::StrList;
pub use setup::Setup;
pub use sink::CountSink;
pub use sink::Sink;

Modules§

cursor
Position tracking and chunk framing over a borrowed byte slice.
diag
Recoverable problems found while parsing.
engine
Which RPG Maker produced the file.
error
Fatal errors.
event
Allocation-free iteration over event and move-route command lists.
field
Per-field wire codecs.
file
Loading and saving whole files.
owned
Detaching a loaded file from the buffer it borrows.
raw
Codecs for the handful of types that do not fit the chunked-struct shape.
rpg
The RPG Maker 2000/2003 data model.
schema
The lcf_struct!, lcf_flags! and lcf_enum! schema macros: the declarations that build every struct, flag bundle and enum in crate::rpg.
setup
RPG_RT’s post-load fixups, as an explicit opt-in.
sink
Where serialized bytes go.
wire
The LCF wire codec: compressed integers, fixed-width primitives, file magics.

Macros§

impl_identity_owned
Implement a no-op detach for types that never borrow.
lcf_enum
Define the named constants for an LCF enumeration.
lcf_flags
Define a bit-packed boolean bundle.
lcf_strings
Declare which of a struct’s fields hold text.
lcf_struct
Define a chunked LCF struct.