# yaml-rt
`yaml-rt` is a YAML 1.2.2 parser and editor for Rust that keeps the original
source text intact. It is designed for tools that need to change YAML without
reformatting everything around the change: comments, whitespace, line endings,
scalar styles, directives, tags, anchors, aliases, document markers, and
original spelling are retained where practical.
Untouched documents are emitted byte-for-byte. Edits are applied as localized
source patches, so a changed value normally produces only the diff you asked
for.
Version 0.1 is suitable for production use within the guarantees and
limitations documented below. Public APIs may still evolve between 0.x
releases.
## When to use yaml-rt
Use `yaml-rt` when you are building a configuration editor, migration tool,
formatter-aware automation, or command-line utility where preserving a human's
YAML matters. For ordinary typed data interchange where presentation does not
matter, the optional Serde integration provides a more conventional conversion
API.
## Installation
The facade crate enables typed-overlay derives by default:
```toml
[dependencies]
yaml-rt = "0.1.0"
```
Choose features explicitly when needed:
```toml
[dependencies]
yaml-rt = { version = "0.1.0", default-features = false }
# or
yaml-rt = { version = "0.1.0", features = ["serde"] }
```
Install the command-line editor with:
```sh
cargo install yaml-rt-cli --locked
```
The installed binary is named `yaml-rt`.
## Lossless editing
Parse a document, queue an edit, and render the minimally changed source:
```rust
use yaml_rt::{YamlDoc, YamlError};
fn main() -> Result<(), YamlError> {
let input = "\
# local development
server:
host: \"localhost\"
port: 8080 # keep this comment
";
let mut doc = YamlDoc::parse(input)?;
doc.set_scalar(&["server", "port"], "9090")?;
assert_eq!(
doc.to_string(),
"\
# local development
server:
host: \"localhost\"
port: 9090 # keep this comment
"
);
Ok(())
}
```
`YamlDoc::to_string()` previews the source with pending patches applied.
`YamlDoc::commit_edits()` reparses that result and makes it the new baseline for
subsequent edits.
The lower-level API exposes the lossless concrete syntax tree, semantic node
metadata, JSON Pointer operations, fragments, diagnostics with spans, and
patch-oriented editing primitives.
## Typed round-trip overlays
`YamlRoundTrip` maps Rust configuration models onto YAML without turning the
Rust value into the source of truth. Reading does not discard syntax, and
writing patches only modeled values unless configured otherwise.
```rust
use yaml_rt::{FromYamlDoc, ToYamlDoc, YamlDoc, YamlError, YamlRoundTrip};
#[derive(Debug, PartialEq, Eq, YamlRoundTrip)]
struct Config {
host: String,
#[yaml(default = 8080)]
port: u16,
#[yaml(rename = "log-level")]
log_level: String,
}
fn main() -> Result<(), YamlError> {
let input = "\
host: \"localhost\"
port: 3000 # selected by the user
log-level: info
extra: keep-me
";
let mut doc = YamlDoc::parse(input)?;
let mut config = Config::from_yaml_doc(&doc)?;
config.port = 9090;
config.apply_to_yaml_doc(&mut doc)?;
assert_eq!(
doc.to_string(),
"\
host: \"localhost\"
port: 9090 # selected by the user
log-level: info
extra: keep-me
"
);
Ok(())
}
```
Supported field attributes are:
- `rename = "yaml-key"`
- `alias = "legacy-key"`
- `default` and `default = expression`
- `comment = "Comment for inserted entries."`
- `skip`
- `skip_serializing_if = "path::to::predicate"`
- `flatten`
- `with = "module::path"`
Rust doc comments become comments for newly inserted entries when no explicit
`comment` attribute is present. Struct-level policies control unknown fields
with `preserve_unknown_fields` or `prune_unknown_fields`, and insertion order
with `insert_order = "append"` or `insert_order = "struct"`.
Single-field tuple structs are transparent automatically. Their inner value is
represented directly, so `struct Port(u16)` reads and writes `8080`, not a
mapping or tag. The unnamed field may use `yaml(with = "module")`.
Enums use the same YAML representation as `yaml-rt-serde`: unit variants are
strings, and variants with data use local tags.
```rust
# use yaml_rt::YamlRoundTrip;
#[derive(YamlRoundTrip)]
#[yaml(rename_all = "lowercase")]
enum Mode {
A, // a
Value(u16), // !value 42
Pair(u8, bool), // !pair [1, true]
Server { host: String }, // !server {host: api}
}
```
Enum variants support `rename` and repeated `alias`. Enum-level `rename_all`
accepts `lowercase`, `snake_case`, `kebab-case`, `SCREAMING_SNAKE_CASE`,
`camelCase`, and `PascalCase`, following Serde's variant transformations.
Aliases are accepted when reading; new values emit the canonical name. An
existing alias spelling is retained while the variant stays the same.
Same-variant writes patch newtype payloads, tuple elements, and struct fields
incrementally. That retains scalar spelling, collection style, comments,
anchors, and unknown struct-variant fields. Switching variants replaces the
enum node deterministically, retains its anchor and surrounding entry comment,
and removes comments owned by the old payload.
Common configuration shapes are supported directly:
| Scalars | `String`, `bool`, `char`, all signed and unsigned integer widths including 128-bit values, `f32`, and `f64` |
| Wrappers | `Option<T>`, `Box<T>`, and fixed arrays `[T; N]` |
| Collections | `Vec<T>`, `BTreeMap<String, T>`, and `HashMap<String, T>` |
| Structs | Named mapping structs and transparent single-field tuple structs |
| Enums | Unit, newtype, tuple, and named-field variants; data variants use local YAML tags |
| Nested models | Derived structs, newtypes, and enums inside the wrappers and collections above |
| Generics | Type, lifetime, and const generics with inferred field bounds and retained user `where` clauses |
| Document roots | Typed scalar, sequence, mapping, newtype, and enum roots |
| Presentation | Existing block and flow collections are patched in their original style |
Sequence elements are matched positionally. Existing elements retain comments,
unknown nested keys, anchors, tags, spelling, and style; appended elements use
deterministic formatting. Typed maps synchronize modeled keys exactly while a
derived struct still preserves unknown fields by default. A semantically
unchanged scalar is not rewritten, so spellings such as `TRUE` and `0x10`
survive. Missing and explicit-null `Option<T>` fields both read as `None`;
writing `None` emits `null` unless `skip_serializing_if` omits the field.
Use `with` when a field has a configuration representation different from its
Rust type. The adapter module declares a supported YAML representation and two
conversions:
```rust
use std::time::Duration;
use yaml_rt::YamlError;
mod duration_seconds {
use super::{Duration, YamlError};
pub type Repr = u64;
pub fn from_yaml(value: Repr) -> Result<Duration, YamlError> {
Ok(Duration::from_secs(value))
}
pub fn to_yaml(value: &Duration) -> Result<Repr, YamlError> {
Ok(value.as_secs())
}
}
# use yaml_rt::YamlRoundTrip;
#[derive(YamlRoundTrip)]
struct Service {
#[yaml(with = "duration_seconds", rename = "timeout-seconds")]
timeout: Duration,
}
```
`Repr` must implement `YamlValue + ToYamlFragment` and can be a scalar or a
collection. `rename`, `alias`, `default`, `comment`, and
`skip_serializing_if` continue to apply around a named-field adapter. Adapters
also work on transparent newtype fields and unnamed enum payload fields. See the
[`config_models` example](crates/yaml-rt/examples/config_models.rs) for nested
struct sequences, a flow-style update, optional/null fields, generics, and the
duration adapter together, and the
[`enum_overlays` example](crates/yaml-rt/examples/enum_overlays.rs) for
transparent newtypes and tagged enums.
`YamlRoundTrip` is a lossless overlay, not a general-purpose Serde data-model
implementation: it reads from an existing `YamlDoc` and applies localized
patches back to that same source. Serde conversion creates ordinary Rust
values and deterministic YAML when retaining the original presentation is not
required.
## Serde conversion
Enable the `serde` feature when source presentation does not need to survive a
typed conversion:
```rust
use serde::{Deserialize, Serialize};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
struct Service {
name: String,
replicas: u16,
}
fn main() -> Result<(), yaml_rt::Error> {
let service: Service = yaml_rt::from_str("name: api\nreplicas: 3\n")?;
assert_eq!(
service,
Service {
name: "api".to_owned(),
replicas: 3,
}
);
assert_eq!(
yaml_rt::to_string(&service)?,
"name: api\nreplicas: 3\n"
);
Ok(())
}
```
Serde serialization emits deterministic block-style YAML. Use a typed
round-trip overlay instead when comments, quoting, whitespace, or other source
presentation must be retained.
## Command-line editor
The CLI applies JSON Pointer operations while preserving the rest of the
document:
```sh
# Read a node.
yaml-rt get /server/port config.yaml
# Replace a value and print the edited document.
yaml-rt replace /server/port --value 9090 config.yaml
# Edit a file atomically in place.
yaml-rt add /server/debug --value true --in-place config.yaml
# Copy a complete YAML node from a file.
yaml-rt add /server/tls --value-file tls.yaml config.yaml
# Select the second document in a YAML stream.
yaml-rt get /name --doc 1 stream.yaml
```
Available operations are `get`, `add`, `remove`, `replace`, `move`, `copy`, and
`test`. An omitted input file, or `-`, reads YAML from standard input.
Mutations write to standard output unless `--output` or `--in-place` is used.
Values passed with `--value` or `--value-file` must be complete YAML nodes.
Run `yaml-rt help <operation>` for operation-specific arguments.
## Crates and features
| `yaml-rt-core` | Dependency-free source model, parser, CST, semantic graph, diagnostics, editor, and emitter | Yes |
| `yaml-rt-derive` | `YamlRoundTrip` procedural derive | Yes |
| `yaml-rt-serde` | Serde serializer and deserializer | Yes |
| `yaml-rt` | Facade re-exporting the public APIs | Yes |
| `yaml-rt-cli` | `yaml-rt` command-line editor | Yes |
| `yaml-rt-bench` | Local comparison benchmarks | No |
| `fuzz` | Separate cargo-fuzz workspace | No |
The facade features are:
| `derive` | Yes | Re-exports `yaml_rt_derive::YamlRoundTrip` |
| `serde` | No | Re-exports the `yaml-rt-serde` conversion API |
`yaml-rt-core` has no third-party dependencies.
## YAML 1.2.2 conformance
The parser is tested against all 402 cases in the YAML Test Suite tag
`data-2022-01-17`, including semantic JSON comparisons where fixtures provide
them. The expected-failure list is empty.
The conformance harness is opt-in for ordinary package tests. A complete local
run uses the pinned submodule:
```sh
YAML_TEST_SUITE_RUN_ALL=1 \
YAML_TEST_SUITE_CHECK_JSON=1 \
cargo test -p yaml-rt-core --test yaml_test_suite
```
## Guarantees
- Parsing and emitting an untouched valid document is byte-identical.
- Local edits retain unrelated source bytes and aim for the smallest practical
diff.
- User-visible syntax and diagnostics carry source spans.
- YAML streams, directives, tags, anchors, aliases, explicit document markers,
collection styles, scalar styles, comments, whitespace, and line endings are
represented by the lossless model.
- Typed round-trip overlays preserve unknown fields by default.
## Current limitations
- This is a round-trip editor, not a canonical YAML formatter.
- Editing an anchored node does not propagate the edit through aliases; aliases
continue to refer to the same anchor in the emitted YAML.
- CLI `copy` and `move` reject cases where anchor ownership would become
ambiguous or invalid.
- JSON Pointer lookup reports duplicate mapping keys and cannot address
non-string mapping keys.
- CLI `test` compares YAML values using the supported YAML 1.2 core scalar and
collection model; it is not a general tag-aware application schema.
- Serde conversion does not expose a generic YAML `Value`, merge-key expansion,
or presentation metadata.
- Typed-overlay `flatten` has intentionally conservative combinations with
field and struct policies; unsupported combinations produce derive errors.
- Typed mappings use string keys. Borrowed overlay fields, catch-all map
flattening, standalone unit structs, standalone multi-field tuple structs,
and full Serde attribute compatibility remain future work.
- Enum data variants use local tags only. Internally tagged, adjacently tagged,
externally mapped, and untagged enum representations are not implemented.
- Collection identity is positional for sequences. Mapping insertion accepts
string keys only; newly inserted `HashMap` keys are sorted for deterministic
output while existing source order is retained.
These constraints are checked rather than silently producing lossy or
surprising output.
## Stability
The 0.1 line is production-usable for the documented behavior. Patch releases
will preserve source and semantic behavior unless fixing a correctness or
safety issue. Because the project is pre-1.0, public Rust APIs and CLI details
may change in minor releases; such changes are documented in the changelog and
follow Conventional Commits.
## Architecture
The lossless CST remains the source of truth. Semantic information and typed
Rust values are overlays that read from it and queue source patches. See
[`docs/architecture.md`](docs/architecture.md) for the component boundaries and
data flow.
## Roadmap
Near-term work focuses on expanding ergonomic edit operations, richer
schema-aware scalar handling, broader typed-overlay shapes, sustained fuzzing,
and performance profiling while retaining zero dependencies in the core crate.
## Development
The workspace requires Rust 1.96 or newer and tests the latest stable toolchain.
Useful release-readiness commands are:
```sh
cargo fmt --all -- --check
cargo test --workspace --all-features --locked
cargo clippy \
-p yaml-rt-core -p yaml-rt-derive -p yaml-rt-serde \
-p yaml-rt -p yaml-rt-cli \
--all-targets --all-features --locked -- -D warnings
RUSTDOCFLAGS="-D warnings" \
cargo doc --workspace --all-features --no-deps
```
See [`RELEASING.md`](RELEASING.md) for the publication process.
## License
Licensed under either the [Apache License, Version 2.0](LICENSE-APACHE) or the
[MIT license](LICENSE-MIT), at your option.