yaml-rt
yaml-rt is a YAML 1.2.2 parser and editor for Rust that keeps the original
source text intact. It is designed for tools that need to change YAML without
reformatting everything around the change: comments, whitespace, line endings,
scalar styles, directives, tags, anchors, aliases, document markers, and
original spelling are retained where practical.
Untouched documents are emitted byte-for-byte. Edits are applied as localized source patches, so a changed value normally produces only the diff you asked for.
Version 0.1 is suitable for production use within the guarantees and limitations documented below. Public APIs may still evolve between 0.x releases.
Playground
Try the yaml-rt command playground to query and edit YAML directly in a browser. It showcases every CLI operation, exact RFC 6901 JSON Pointer targets, multi-target RFC 9535 JSONPath selection, and transactional YAML or JSON patches while displaying the minimally changed document beside the original. Both editors show line numbers, changed output lines are highlighted, and gutter markers identify source lines removed by an edit. Source-aware failures use the same rustc-style diagnostics as the CLI.
The playground runs the Rust parser and editor locally through WebAssembly.
Documents are not uploaded to a server. Its interface is plain HTML, CSS, and
JavaScript with a pinned CodeMirror bundle built into the deployed site.
The input and output panels each have an optional Schema tab. Enter a JSON or
YAML schema to validate every input document and the resulting YAML after a
command. Input schema failures do not stop edits, so a command can repair an
invalid document. For get and query, the output schema checks each returned
value; test has no YAML output to validate. Browser schemas are entered as
text, so references requiring separate files are unavailable. These two
playground checks are independent of the CLI command preview.
The playground's schema command generates a copyable JSON Schema from one
input document; Use as input schema places it in the Input Schema tab. Several
examples include input and output schemas, and each
example keeps schema edits when you switch away and back; Reset restores its
original schemas.
When to use yaml-rt
Use yaml-rt when you are building a configuration editor, migration tool,
formatter-aware automation, or command-line utility where preserving a human's
YAML matters. For ordinary typed data interchange where presentation does not
matter, the optional Serde integration provides a more conventional conversion
API.
Installation
The facade crate enables typed-overlay derives by default:
[]
= "0.1.0"
Choose features explicitly when needed:
[]
= { = "0.1.0", = false }
# or
= { = "0.1.0", = ["serde"] }
Install the command-line editor with:
The installed binary is named yaml-rt.
Lossless editing
Parse a document, queue an edit, and render the minimally changed source:
use ;
YamlDoc::to_string() previews the source with pending patches applied.
YamlDoc::commit_edits() reparses that result and makes it the new baseline for
subsequent edits.
Mapping keys can be renamed without moving or reconstructing their entries.
YamlDoc::rename_key_at() accepts a JSON Pointer to the member value, while
YamlDoc::rename_keys_at() applies one decoded destination name to several
members transactionally. Key quoting changes only when required to keep the
new name a YAML string; entry position, values, comments, tags, anchors, and
surrounding whitespace remain source-owned.
Sequences have a focused, transactional editor. Its callbacks receive lossless node handles, so filtering can use semantic values without reparsing snippets:
use ;
#
The lower-level API exposes the lossless concrete syntax tree, semantic node metadata, JSON Pointer operations, fragments, diagnostics with spans, and patch-oriented editing primitives.
Typed round-trip overlays
YamlRt maps Rust configuration models onto YAML without turning the
Rust value into the source of truth. Reading does not discard syntax, and
writing patches only modeled values unless configured otherwise.
use ;
Supported field attributes are:
rename = "yaml-key"alias = "legacy-key"defaultanddefault = expressioncomment = "Comment for inserted entries."skipskip_serializing_if = "path::to::predicate"flattenwith = "module::path"
Rust doc comments become comments for newly inserted entries when no explicit
comment attribute is present. Struct-level policies control unknown fields
with preserve_unknown_fields or prune_unknown_fields, and insertion order
with insert_order = "append" or insert_order = "struct". Struct-level
rename_all supports lowercase, snake_case, kebab-case,
SCREAMING_SNAKE_CASE, camelCase, and PascalCase; an explicit field
rename takes precedence.
flatten accepts both nested derived structs and one catch-all
BTreeMap<String, T> or HashMap<String, T> across the recursively flattened
field graph. The catch-all map reads every entry not claimed by a canonical
field name, alias, skipped field, or nested flattened struct. Applying the
overlay synchronizes those entries exactly: existing entries are patched in
place, missing map keys are removed from YAML, and new keys are inserted
deterministically. A catch-all entry that collides with a modeled key is
rejected before edits are queued.
Single-field tuple structs are transparent automatically. Their inner value is
represented directly, so struct Port(u16) reads and writes 8080, not a
mapping or tag. Multi-field tuple structs use fixed-length YAML sequences, and
unit structs use YAML null. Unnamed fields may use yaml(with = "module").
Existing tuple elements are patched positionally, preserving block or flow
style and element presentation.
Enums use the same YAML representation as yaml-rt-serde: unit variants are
strings, and variants with data use local tags.
# use YamlRt;
Enum variants support rename and repeated alias, plus rename_all on a
named-field variant to transform its payload keys. Enum-level rename_all
accepts lowercase, snake_case, kebab-case, SCREAMING_SNAKE_CASE,
camelCase, and PascalCase, following Serde's transformations.
Aliases are accepted when reading; new values emit the canonical name. An
existing alias spelling is retained while the variant stays the same.
Same-variant writes patch newtype payloads, tuple elements, and struct fields incrementally. That retains scalar spelling, collection style, comments, anchors, and unknown struct-variant fields. Switching variants replaces the enum node deterministically, retains its anchor and surrounding entry comment, and removes comments owned by the old payload.
Common configuration shapes are supported directly:
| Shape | Coverage |
|---|---|
| Scalars | String, bool, char, all signed and unsigned integer widths including 128-bit values, f32, and f64 |
| Wrappers | Option<T>, Box<T>, and fixed arrays [T; N] |
| Collections | Vec<T>, BTreeMap<String, T>, and HashMap<String, T> |
| Structs | Named mapping structs, transparent newtypes, positional tuple structs, and null-valued unit structs |
| Enums | Unit, newtype, tuple, and named-field variants; data variants use local YAML tags |
| Nested models | Derived structs, newtypes, and enums inside the wrappers and collections above |
| Generics | Type, lifetime, and const generics with inferred field bounds and retained user where clauses |
| Document roots | Typed scalar, sequence, mapping, newtype, and enum roots |
| Presentation | Existing block and flow collections are patched in their original style |
Sequence elements are matched positionally. Existing elements retain comments,
unknown nested keys, anchors, tags, spelling, and style; appended elements use
deterministic formatting. Typed maps synchronize modeled keys exactly while a
derived struct still preserves unknown fields by default. A semantically
unchanged scalar is not rewritten, so spellings such as TRUE and 0x10
survive. Missing and explicit-null Option<T> fields both read as None;
writing None emits null unless skip_serializing_if omits the field.
Use with when a field has a configuration representation different from its
Rust type. The adapter module declares a supported YAML representation and two
conversions:
use Duration;
use YamlError;
# use YamlRt;
Repr must implement YamlValue + ToYamlFragment and can be a scalar or a
collection. rename, alias, default, comment, and
skip_serializing_if continue to apply around a named-field adapter. Adapters
also work on transparent newtype fields and unnamed enum payload fields. See the
config_models example for nested
struct sequences, a flow-style update, optional/null fields, generics, and the
duration adapter together, and the
enum_overlays example for
transparent newtypes and tagged enums.
YamlRt is a lossless overlay, not a general-purpose Serde data-model
implementation: it reads from an existing YamlDoc and applies localized
patches back to that same source. Serde conversion creates ordinary Rust
values and deterministic YAML when retaining the original presentation is not
required.
Serde conversion
Enable the serde feature when source presentation does not need to survive a
typed conversion:
use ;
For dynamic YAML, the same feature exposes a yaml_serde-compatible value
model with ordered mappings, local tags, indexing, typed conversion, and
explicit merge-key expansion:
use ;
#
Value, Number, Sequence, Mapping, Index, to_value, and
from_value are available from yaml-rt-serde directly and through the
facade. Number additionally retains i128 and u128 values. Value
conversion resolves aliases and intentionally discards comments, anchors,
styles, and original scalar spelling. YAML << keys remain ordinary mapping
entries until Value::apply_merge() is called.
Serde serialization emits deterministic block-style YAML. Use a typed round-trip overlay instead when comments, quoting, whitespace, or other source presentation must be retained.
Command-line query and editor
The CLI queries YAML with RFC 9535 JSONPath and applies JSON Pointer operations while preserving the rest of the document:
Invalid YAML is reported with a source excerpt and an underline at the failing
span, including the file path (or <stdin>), line, and column. The binary uses
standard terminal colors when stderr is interactive, stays plain when output is
redirected, and honors NO_COLOR.
Resolution errors expose YAML source spans when they identify concrete syntax, including unresolved aliases reached through JSON Pointer or JSONPath.
Library users can render the same dependency-free diagnostic format directly:
use ;
let input = "items: [a, , b]\n";
let error = parse.unwrap_err;
let rendered = error
.render
.with_source_name
.with_color
.to_string;
assert!;
# Find every service port and print JSON Pointer/value pairs.
# Validate one file, or recursively validate a directory.
# Validate each YAML document against a JSON or YAML schema file.
# Infer a permissive JSON Schema from one YAML document.
# Read a node.
# Read every matching node as a YAML document stream.
# Replace a value and print the edited document.
# Replace every node selected by JSONPath.
# Rename a mapping key without moving its entry.
# Give every selected mapping member the same new key name.
# Edit a file atomically in place.
# Copy a complete YAML node from a file.
# Select the second document in a YAML stream.
# Apply several changes transactionally from YAML or JSON.
# Recursively edit every .yaml or .yml file under a directory.
Available operations are validate, schema, query, get, add, remove, replace,
rename-key, move, copy, test, and patch. Query results are emitted in
nodelist order as one compact JSON Pointer/value pair per line. JSONPath
evaluation uses the YAML 1.2 core schema and rejects YAML values that are not
JSON-compatible, including non-string or duplicate mapping keys and non-finite
numbers.
Each operation has a short alias: v for validate, q for query, g for
get, a for add, d for remove, r for replace, k for
rename-key, m for move, c for copy, t for test, and p for
patch.
get, add, remove, replace, rename-key, and test also accept
--query QUERY in place of their positional JSON Pointer. get --query emits
each match as a separate --- YAML document in nodelist order. Query-targeted
mutations apply to all matched nodes transactionally and duplicate targets are
edited once. Removal and value replacement normalize overlapping ancestor and
descendant selections; key rename resolves all owning mapping entries before
editing, so nested keys can be renamed together. A mutation or test fails
when the query matches nothing, while get succeeds with empty output.
rename-key takes the decoded destination name through --to KEY. Every
selected node must be a mapping member, and every affected mapping must retain
unique final string keys. Plain, single-quoted, and double-quoted keys are
supported in both implicit and explicit-key syntax. Block-scalar, alias, and
complex key occurrences are rejected transactionally.
patch accepts an RFC 6902-style operation sequence through either
--patch YAML or --patch-file FILE. JSON patch documents are valid because
JSON is a subset of YAML; YAML syntax additionally permits complete YAML nodes
as operation values. Operations run in order against the selected document,
and the entire patch is rolled back if any operation or test fails.
An input may be a single file, a directory, or - for standard input. Omitting
the input scans the current directory. Directory inputs are searched
recursively for .yaml and .yml files using case-insensitive extensions;
hidden paths are included, symbolic links are skipped, and files are processed
in sorted relative-path order. A directory with no matching files succeeds
without output.
validate parses the complete YAML stream and produces no output when it is
valid. Empty and multi-document streams are accepted. Directory validation
continues after failures and reports each invalid file in the batch summary.
With --schema (-s), every document is also checked against one compiled
JSON Schema. Schema files can be JSON or YAML. Local file references are read
relative to the schema file; references requiring a network fetch fail.
YAML values outside the JSON data model, including non-string mapping keys,
duplicate keys, non-finite numbers, and recursive aliases, fail validation.
schema <FILE> requires one YAML document and prints a deterministic,
pretty-printed JSON Schema. It infers types, properties, and array items while
leaving properties optional and objects open. Use --output to write a file.
Batch query and get --query output identifies each input containing a match
with an ==> relative/path.yaml <== header and a blank line between files;
files without matches produce no section. Pointer-based batch get identifies
every successful input. The --output option writes the same combined stream
to one file. Batch mutations
require --in-place; each file is replaced atomically only after its operation
succeeds. Failures are reported with relative paths, processing continues for
the remaining files, and the command exits unsuccessfully if any file or
directory traversal failed.
Single-file mutations write to standard output unless --output or
--in-place is used. Values passed with --value or --value-file must be
complete YAML nodes. --patch-file - reads the patch from standard input only
when the target YAML does not also use -; both inputs cannot use standard
input at once.
Run yaml-rt help <operation> for operation-specific arguments.
Crates and features
| Package | Purpose | Published |
|---|---|---|
yaml-rt-core |
Dependency-free source model, parser, CST, semantic graph, diagnostics, editor, and emitter | Yes |
yaml-rt-rfc9535 |
Native RFC 9535 JSONPath parsing and evaluation over YamlDoc |
Yes |
yaml-rt-schema |
JSON Schema validation and permissive inference over YamlDoc |
No |
yaml-rt-derive |
YamlRt procedural derive |
Yes |
yaml-rt-serde |
Serde serializer and deserializer | Yes |
yaml-rt |
Facade re-exporting the public APIs | Yes |
yaml-rt-cli |
yaml-rt command-line editor |
Yes |
yaml-rt-wasm |
Filesystem-free command engine and browser bindings for the GitHub Pages playground | No |
yaml-rt-bench |
Local comparison benchmarks | No |
fuzz |
Separate cargo-fuzz workspace | No |
Parser fuzzing
Parser fuzzing uses cargo-fuzz 0.13.2 and a nightly Rust toolchain. The target
accepts UTF-8 text, exercises YamlDoc::parse, and requires every successful
parse to emit byte-identically, reparse successfully, and retain the same YAML
Test Suite event stream.
Generate the reproducible YAML Test Suite and hand-written seeds separately from libFuzzer's evolving corpus, then start a bounded local run:
LSAN_OPTIONS=detect_leaks=0
New coverage inputs are written only to corpus/parse_yaml; regenerating the
canonical seeds does not remove them. Periodically minimize that learned corpus
before longer runs:
LSAN_OPTIONS=detect_leaks=0
Reproduce and minimize a crash with a backtrace, then copy the minimized input
under a descriptive name into fuzz/regressions/parse_yaml/ so CI replays it:
LSAN_OPTIONS=detect_leaks=0 RUST_BACKTRACE=1 \
LSAN_OPTIONS=detect_leaks=0
CI replays committed regressions on every run, fuzzes for 30 seconds on pushes and pull requests, fuzzes for five minutes on scheduled/manual runs, and uploads failure artifacts for reproduction.
The facade features are:
| Feature | Default | Effect |
|---|---|---|
derive |
Yes | Re-exports yaml_rt_derive::YamlRt |
serde |
No | Re-exports the yaml-rt-serde conversion API |
yaml-rt-core has no third-party dependencies and retains the foundational RFC
6901 JsonPointer API. Its YamlPatch and YamlPatchOperation APIs parse and
transactionally apply RFC 6902-style operation sequences with full YAML values.
The independently usable yaml-rt-rfc9535 crate depends on regex for the
standard match() and search() functions. Its JSONPath parser, semantic
evaluator, compatibility validator, and pointer construction operate directly
on YamlDoc without a generic JSON value dependency. Compact JSON rendering
remains private to yaml-rt-cli; the RFC 9535 crate is not re-exported by the
yaml-rt facade.
The schema crate reuses yaml-rt-core for JSON and YAML parsing and writes JSON
through its own small value model. Schema::validate_pointer checks a selected
YAML value while retaining source locations for diagnostics. Exact decimal
arithmetic uses decimal digits.
regex checks patterns, url resolves references, iri-string and idna check
internationalized formats, and jiff checks dates and times. These dependencies
do not enter the core parser. The official JSON Schema Test Suite is pinned at
third_party/json-schema-test-suite and runs in CI.
YAML 1.2.2 conformance
The parser is tested against all 402 cases in the YAML Test Suite tag
data-2022-01-17, including semantic JSON comparisons where fixtures provide
them. The expected-failure list is empty.
The conformance harness is opt-in for ordinary package tests. A complete local run uses the pinned submodule:
YAML_TEST_SUITE_RUN_ALL=1 \
YAML_TEST_SUITE_CHECK_JSON=1 \
Guarantees
- Parsing and emitting an untouched valid document is byte-identical.
- Local edits retain unrelated source bytes and aim for the smallest practical diff.
- Batch patches are transactional; a failing operation leaves the target document unchanged.
- User-visible syntax and diagnostics carry source spans.
- YAML streams, directives, tags, anchors, aliases, explicit document markers, collection styles, scalar styles, comments, whitespace, and line endings are represented by the lossless model.
- Typed round-trip overlays preserve unknown fields by default.
Current limitations
- This is a round-trip editor, not a canonical YAML formatter.
- Editing an anchored node does not propagate the edit through aliases; aliases continue to refer to the same anchor in the emitted YAML.
- CLI
copyandmovereject cases where anchor ownership would become ambiguous or invalid. - JSON Pointer lookup reports duplicate mapping keys and cannot address non-string mapping keys.
- Mapping-key rename supports plain, single-quoted, and double-quoted string key occurrences; block-scalar, alias, and complex keys are not rewritten.
- CLI and patch
testoperations compare YAML values using the supported YAML 1.2 core scalar and collection model; they are not a general tag-aware application schema. - JSON Schema 2020-12 validation passes the selected official core and format
assertion cases. Remote fetches, custom vocabularies, and ECMAScript regex
compatibility remain outside this release; see
docs/json-schema-conformance.md. - Serde
Valueconversion does not preserve presentation metadata and only expands merge keys whenValue::apply_merge()is requested. - Typed-overlay
flattenhas intentionally conservative combinations with field and struct policies; unsupported combinations produce derive errors. - Typed mappings use string keys. Borrowed overlay fields and full Serde attribute compatibility remain future work.
- Enum data variants use local tags only. Internally tagged, adjacently tagged, externally mapped, and untagged enum representations are not implemented.
- Collection identity is positional for sequences. Mapping insertion accepts
string keys only; newly inserted
HashMapkeys are sorted for deterministic output while existing source order is retained. - Flow collections may be nested up to 1,024 levels. Deeper input is rejected with a span-aware parser diagnostic.
These constraints are checked rather than silently producing lossy or surprising output.
Stability
The 0.1 line is production-usable for the documented behavior. Patch releases will preserve source and semantic behavior unless fixing a correctness or safety issue. Because the project is pre-1.0, public Rust APIs and CLI details may change in minor releases; such changes are documented in the changelog and follow Conventional Commits.
Architecture
The lossless CST remains the source of truth. Semantic information and typed
Rust values are overlays that read from it and queue source patches. See
docs/architecture.md for the component boundaries and
data flow. Block parsing advances a sequential line cursor over compact entry
and collection frames in one iterative state machine. Flow collections are
parsed left-to-right with a bounded explicit frame stack. Both machines
register semantic metadata directly in CST nodes during recognition, so deeply
nested input does not rely on call-stack recursion, no completed-CST semantic
pass is required, and each CST node is attached in its grammatical context
exactly once.
Roadmap
Near-term work focuses on expanding ergonomic edit operations, richer schema-aware scalar handling, broader typed-overlay shapes, sustained fuzzing, and performance profiling while retaining zero dependencies in the core crate.
Development
The workspace requires Rust 1.96 or newer and tests the latest stable toolchain. Useful release-readiness commands are:
RUSTDOCFLAGS="-D warnings" \
See RELEASING.md for the publication process.
License
Licensed under either the Apache License, Version 2.0 or the MIT license, at your option.