dfraw_json_parser

A solution to present information about what raws are in your save game, in a searchable format.

I made this because I was playing with Splint's Vanilla Expanded Mod and Primal and at the prepare carefully screen I had no way to figure out what some of the animals were.

Yes there is the raw explorer program but I found that difficult to search with and the information was displayed in basically the same format as the raw file itself, so it was hard to read.

There is a Typescript type definition file.

Current Functionality

Creates JSON from Dwarf Fortress raw files. This JSON can be used with the small web client to search through, be plopped into Algolia and search or, you could simply CTRL+F or grep for what you are looking for in the file itself. I find the JSON easier to read than the RAW txt files, and it currently doesn't include a lot of items that were not important to me when looking up creatures. I was most concerned with the description of the animal, if they laid eggs, if they were milkable, and how big they were.

Parses raw files for creatures

Outputs

There is a Typescript definition file for the format of the generated JSON. Here are detailed descriptions of Rust structs used when performing the serde_json serialization.

DF Info File

It parses the info.txt file in each raw module directory it goes through.

Property	Description	Type
identifier	id of the raw module	`String`
sourced_directory	directory found in, `vanilla`, `mods` or `installed_mods`	`String`
numeric_version	version of raw module represented as a number	`u32`
displayed_version	human-readable display for `numeric_version`	`String`
earliest_compatible_numeric_version	earliest "upgrade-compatible" version of this raw module, as a number	`u32`
earliest_compatible_displayed_version	human readable display for `earliest_compatible_numeric_version`	`String`
author	author of the raw module	`String`
name	human readable name of the module	`String`
description	description of the module	`String`
display_title	built-in formatting for `{name} v{displayed_version}`	`String`

Creature Token

Creature data and its castes are included. However, currently doesn't apply COPY_TAG_FROM.

Property	Description	Type
identifier	defined in CREATURE token	`String`
parent_raw	name of the raw file its located	`String`
raw_module	id of the raws module raw is from	`String`
raw_module_version	version of the raws module raw is from	`String`
dfraw_display	human readable "Name Version" of the parent module	`String`
objectId	unique id for creature	`String`
name	species name	`String`
names_map	names by castes	`HashMap<String, Vec<String>>`
descriptions	descriptions by castes	`HashMap<String, String>`
max_age	max age by castes	`HashMap<String, [u16; 2]>`
clutch_size	clutch size by castes	`HashMap<String, [u16; 2]>`
based_on	defined by copy_tags_from token	`String`
biomes	biomes creature found in	`Vec<String>`
cluster_range	cluster range (how many appear at once)	`[u16; 2]`
underground_depth	depth found	`[u16; 2]`
body_size	body size by castes	`HashMap<String, Vec<DFBodySize>>`
grown_at	age when adult by castes	`HashMap<String, u32>`
child_at	age when adolescent by castes	`HashMap<String, u32>`
egg_sizes	egg size by castes	`HashMap<String, u32>`
pet_value	pet value by castes	`HashMap<String, u16>`
intelligence	intelligence by castes	`HashMap<String, [bool; 2]>`
flier	flier by castes	`HashMap<String, bool>`
gnawer	gnawer by castes	`HashMap<String, bool>`
trainable	trainability by castes	`HashMap<String, u8>`
active_time	active time by castes	`HashMap<String, u8>`
inactive_season	NO_SEASON by castes	`HashMap<String, u8>`
creature_class	creature class by castes	`HashMap<String, Vec<String>>`
tags	tags on the creature	`Vec<CreatureTag>`
caste_tags	tags on each caste	`HashMap<String, Vec<CasteTag>>`
difficulty	difficulty by castes	`HashMap<String, u32>`
grass_trample	grass trample by castes	`HashMap<String, u8>`
grazer	grazer by castes	`HashMap<String, u32>`
low_light_vision	low light vision by castes	`HashMap<String, u32>`
pop_ratio	population ratio by castes	`HashMap<String, u16>`
milkable	milk production by castes	`HashMap<String, DFMilkable>`
pref_string	preference string for creature	`Vec<String>`
population_number	pop num (how many exist per valid world tile)	`[u16; 2]`

Rust Program

An example rust program which will parse a directory for DF raw files and then output the raws as JSON to be consumed by the simple web client in www/ for simple search and display functionality. The rust program is capable of serving the web client itself.

You can run this program after cloning this repository:

cargo run --example cli -- --help

cli.rs

Library which parses Dwarf Fortress raw files into JSON

Usage: cli.exe [OPTIONS]

Options:
  -g, --game-dir <GAME_DIR>
          Specify the directory where Dwarf Fortress is installed.

          This directory will likely include the 'gamelog.txt' file, and it should have a 'data' subdirectory.

          [default: ]

  -o, --out-dir <OUT_DIR>
          Specify the directory that the JSON database should be saved into.

          If raw files are parsed, a JSON database (an array of objects) is
          saved to disk in a location specified by this argument. This will
          create an 'raws.json' file in the directory specified by this argument.

          Alongside raws.json will be a modules.json which is a JSON database for the
          raw modules that were found and parsed.

          [default: ./www/]

  -h, --help
          Print help information (use `-h` for a summary)

  -V, --version
          Print version information

How to use

The example is a usable tool to write the parsed raws into JSON, this could then be uploaded to Algolia search for searching through, or can be easily ingested by a website with javascript.

There is a Typescript definition file for the format of the generated JSON.

Existing Projects

These are projects which use this library.

Overseer's Reference Manual for Dwarf Fortress

Perl script

This all started with a perl script, I've archived that to a gist since it lost feature parity (and object similarity) to this project.

dfraw_json_parser 0.12.0