rust-yaml 0.0.5

# rust-yaml

[![CI](https://github.com/elioetibr/rust-yaml/actions/workflows/ci.yml/badge.svg)](https://github.com/elioetibr/rust-yaml/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/elioetibr/rust-yaml/graph/badge.svg?token=u35QeHs4KV)](https://codecov.io/gh/elioetibr/rust-yaml)
[![Crates.io](https://img.shields.io/crates/v/rust-yaml.svg)](https://crates.io/crates/rust-yaml)
[![docs.rs](https://docs.rs/rust-yaml/badge.svg)](https://docs.rs/rust-yaml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A complete, fast, and safe YAML 1.2 library for Rust with advanced features and security-first design. This library provides comprehensive YAML processing capabilities with full specification compliance, advanced security features, and excellent performance.

> **Status**: Production-ready with comprehensive test coverage (134 unit tests + 150+ integration tests passing). All YAML 1.2 core features implemented and battle-tested.

## Table of Contents

- [rust-yaml](#rust-yaml)
  - [Table of Contents](#table-of-contents)
  - [Why rust-yaml?](#why-rust-yaml)
  - [Features](#features)
    - [Core Features](#core-features)
    - [Advanced Features](#advanced-features)
  - [Quick Start](#quick-start)
    - [Basic Usage](#basic-usage)
    - [Multi-Document Support](#multi-document-support)
    - [Custom Configuration](#custom-configuration)
    - [Advanced Features Examples](#advanced-features-examples)
      - [Anchors and Aliases](#anchors-and-aliases)
      - [Multi-line Strings](#multi-line-strings)
      - [Explicit Type Tags](#explicit-type-tags)
      - [Complex Keys (Sequences and Mappings as Keys)](#complex-keys-sequences-and-mappings-as-keys)
  - [Real-World Usage](#real-world-usage)
    - [Configuration Management](#configuration-management)
    - [Data Processing with Type Safety](#data-processing-with-type-safety)
    - [Smart Serialization](#smart-serialization)
  - [Value Types](#value-types)
  - [Error Handling](#error-handling)
  - [Loader Types](#loader-types)
  - [Performance](#performance)
    - [Benchmarks](#benchmarks)
  - [Feature Flags](#feature-flags)
  - [Feature Status](#feature-status)
    - [✅ Core YAML 1.2 Implementation (COMPLETE)](#-core-yaml-12-implementation-complete)
    - [✅ Advanced Features (COMPLETE)](#-advanced-features-complete)
    - [✅ **Comment Preservation (IMPLEMENTED)**](#-comment-preservation-implemented)
    - [✅ **Schema Validation (IMPLEMENTED)**](#-schema-validation-implemented)
    - [🎯 Future Enhancements](#-future-enhancements)
  - [Contributing](#contributing)
    - [Development Setup](#development-setup)
    - [Development Commands](#development-commands)
  - [Documentation](#documentation)
    - [Complete Documentation Index](#complete-documentation-index)
  - [License](#license)
  - [Acknowledgments](#acknowledgments)
  - [Related Projects](#related-projects)

## Why rust-yaml?

rust-yaml stands out from other YAML libraries by providing:

- **🎯 Complete YAML 1.2 compliance** - Full specification support, not just basic parsing
- **⚓ Smart serialization** - Automatic anchor/alias generation for shared data structures
- **🛡️ Security first** - No unsafe code, protection against malicious input, configurable limits
- **🚀 High performance** - Zero-copy parsing, minimal allocations, optimized algorithms
- **📍 Developer-friendly errors** - Precise error locations with visual context and suggestions
- **🔄 Structural round-trips** - Parse → serialize → parse with data consistency
- **🏗️ Production ready** - Comprehensive testing, fuzzing, and real-world validation

## Features

### Core Features

- 🚀 **Fast**: Zero-copy parsing where possible, optimized for performance
- 🛡️ **Safe**: Memory safety guaranteed, no unsafe code, secure deserialization
- 🔒 **Reliable**: Comprehensive error handling with precise position information
- 🧹 **Clean**: Well-documented API following Rust best practices
- 📝 **YAML 1.2**: Full YAML 1.2 specification compliance
- 🔄 **Round-trip**: Structural parse → serialize → parse consistency

### Advanced Features

- ⚓ **Anchors & Aliases**: Full support for `&anchor` and `*alias` references
- 📄 **Multi-line Strings**: Literal (`|`) and folded (`>`) block scalars
- 🏷️ **Type Tags**: Explicit type specification (`!!str`, `!!int`, `!!map`, `!!seq`)
- 🎯 **Smart Serialization**: Automatic anchor/alias generation for shared values
- 📍 **Enhanced Errors**: Contextual error reporting with visual indicators
- 🔧 **Flexible API**: Multiple loader types for different security levels

## Quick Start

Add this to your `Cargo.toml`:

```toml
[dependencies]
rust-yaml = "0.0.5"
```

### Basic Usage

```rust
use rust_yaml::Yaml;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let yaml = Yaml::new();

    // Parse YAML from a string
    let yaml_content = r#"
        name: "rust-yaml"
        version: "0.0.5"
        features:
          - fast
          - safe
          - reliable
        config:
          debug: true
          max_depth: 100
    "#;

    let parsed = yaml.load_str(yaml_content)?;
    println!("Parsed: {:#?}", parsed);

    // Dump back to YAML
    let output = yaml.dump_str(&parsed)?;
    println!("Output:\n{}", output);

    Ok(())
}
```

### Multi-Document Support

```rust
use rust_yaml::Yaml;

let yaml = Yaml::new();
let multi_doc = r#"
document: 1
data: [1, 2, 3]
---
document: 2
data: [4, 5, 6]
---
document: 3
data: [7, 8, 9]
"#;

let documents = yaml.load_all_str(multi_doc)?;
println!("Found {} documents", documents.len());
```

### Custom Configuration

```rust
use rust_yaml::{Yaml, YamlConfig, LoaderType};

let config = YamlConfig {
    loader_type: LoaderType::Safe,
    allow_duplicate_keys: false,
    explicit_start: Some(true),
    width: Some(120),
    ..Default::default()
};

let yaml = Yaml::with_config(config);
```

### Advanced Features Examples

#### Anchors and Aliases

```rust
use rust_yaml::Yaml;

let yaml = Yaml::new();
let yaml_with_anchors = r#"
defaults: &defaults
  timeout: 30
  retries: 3

development:
  <<: *defaults
  debug: true

production:
  <<: *defaults
  debug: false
"#;

let parsed = yaml.load_str(yaml_with_anchors)?;
```

#### Multi-line Strings

```rust
let yaml_with_multiline = r#"

# Literal block scalar (preserves newlines)
sql_query: |
  SELECT name, age
  FROM users
  WHERE active = true
  ORDER BY name;

# Folded block scalar (folds newlines to spaces)
description: >
  This is a long description that will be
  folded into a single line when parsed,
  making it easier to read in the YAML file.
"#;

let parsed = yaml.load_str(yaml_with_multiline)?;
```

#### Explicit Type Tags

```rust
let yaml_with_tags = r#"
string_value: !!str 42          # Force as string
int_value: !!int "123"          # Force as integer
float_value: !!float "3.14"     # Force as float
bool_value: !!bool "yes"        # Force as boolean
sequence: !!seq [a, b, c]       # Explicit sequence
mapping: !!map {key: value}     # Explicit mapping
"#;

let parsed = yaml.load_str(yaml_with_tags)?;
```

#### Complex Keys (Sequences and Mappings as Keys)

```rust
// YAML 1.2 supports using complex structures as mapping keys
let yaml_with_complex_keys = r#"

# Sequence as a key
? [name, age]
: [John, 30]

# Mapping as a key  
? {first: Alice, last: Smith}
: person_data

# Mixed with regular keys
simple_key: simple_value
? [complex, sequence, key]
: complex_value
"#;

let parsed = yaml.load_str(yaml_with_complex_keys)?;
// The parser correctly handles complex keys and can round-trip them
let serialized = yaml.dump_str(&parsed)?;
```

## Real-World Usage

### Configuration Management

```rust
use rust_yaml::Yaml;

// Load application configuration with anchors for shared settings
let config_yaml = r#"
defaults: &defaults
  timeout: 30
  retry_count: 3
  log_level: info

database:
  <<: *defaults
  host: localhost
  port: 5432

api:
  <<: *defaults
  host: 0.0.0.0
  port: 8080
  cors_enabled: true
"#;

let config = yaml.load_str(config_yaml)?;
```

### Data Processing with Type Safety

```rust
// Process data with explicit types to ensure correctness
let data_yaml = r#"
users:
  - id: !!int "1001"
    name: !!str "Alice"
    active: !!bool "yes"
    score: !!float "95.5"
  - id: !!int "1002"
    name: !!str "Bob"
    active: !!bool "no"
    score: !!float "87.2"
"#;

let users = yaml.load_str(data_yaml)?;
```

### Smart Serialization

```rust
use rust_yaml::{Yaml, Value};
use indexmap::IndexMap;

// Create shared data structure
let shared_config = {
    let mut map = IndexMap::new();
    map.insert(Value::String("timeout".to_string()), Value::Int(30));
    map.insert(Value::String("retries".to_string()), Value::Int(3));
    Value::Mapping(map)
};

// Use it multiple times - rust-yaml will automatically create anchors/aliases
let mut root = IndexMap::new();
root.insert(Value::String("dev".to_string()), shared_config.clone());
root.insert(Value::String("prod".to_string()), shared_config.clone());

let output = yaml.dump_str(&Value::Mapping(root))?;
// Output will contain: dev: &anchor0 ... prod: *anchor0
```

## Value Types

The library provides a comprehensive `Value` enum for representing YAML data:

```rust
use rust_yaml::Value;

let values = vec![
    Value::Null,
    Value::Bool(true),
    Value::Int(42),
    Value::Float(3.14),
    Value::String("hello".to_string()),
    Value::Sequence(vec![Value::Int(1), Value::Int(2)]),
    Value::mapping(), // Empty mapping
];
```

## Error Handling

Advanced error reporting with precise position information and visual context:

```rust
use rust_yaml::{Yaml, Error};

let yaml = Yaml::new();
let result = yaml.load_str("invalid: yaml: content:");

match result {
    Ok(value) => println!("Success: {:?}", value),
    Err(Error::Parse { position, message, context }) => {
        println!("Parse error at line {}, column {}: {}",
                 position.line, position.column, message);

        // Enhanced error context with visual indicators
        if let Some(ctx) = context {
            println!("Context:\n{}", ctx.line_content);
            println!("{}^", " ".repeat(ctx.column_position));
            if let Some(suggestion) = &ctx.suggestion {
                println!("Suggestion: {}", suggestion);
            }
        }
    }
    Err(e) => println!("Other error: {}", e),
}
```

## Loader Types

Different loader types provide varying levels of functionality and security:

- **`Safe`** (default): Only basic YAML types, no code execution
- **`Base`**: Minimal type set for simple use cases
- **`RoundTrip`**: Preserves formatting and comments (partial implementation available)
- **`Full`**: All features including potentially unsafe operations

## Performance

rust-yaml is designed for high performance:

- **Zero-copy parsing** where possible to minimize allocations
- **Efficient memory usage** with `IndexMap` for preserving key order
- **Smart serialization** with automatic anchor/alias generation reduces output size
- **Streaming parser support** for memory-efficient processing
- **Optimized string handling** with intelligent quoting decisions
- **Minimal allocations** during parsing and serialization

### Benchmarks

```bash

# Run performance benchmarks
cargo bench

# Run with release optimizations
cargo test --release

# Profile memory usage
cargo test --features large-documents
```

## Feature Flags

```toml
[dependencies]
rust-yaml = { version = "0.0.5", features = ["serde", "large-documents"] }
```

- **`default = ["mmap", "preserve-order"]`**: Default feature set with memory mapping and order preservation
- **`serde`**: Enable serde serialization support for Rust structs
- **`preserve-order`**: Always preserve key insertion order (uses IndexMap)
- **`large-documents`**: Optimizations for very large YAML documents
- **`async`**: Async/await support with tokio integration
- **`mmap`**: Memory-mapped file support for large documents
- **`full`**: All features enabled

## Feature Status

### ✅ Core YAML 1.2 Implementation (COMPLETE)

**Parsing & Generation**

- ✅ **Complete YAML 1.2 parsing and generation** - Full specification support
- ✅ **Core data types** - null, bool, int, float, string, sequence, mapping
- ✅ **Safe loading/dumping** - Security controls and configurable limits
- ✅ **Multi-document support** - Proper document boundaries with `---`
- ✅ **Flow and block styles** - Automatic detection and round-trip preservation
- ✅ **Structural round-trip support** - Parse → serialize → parse with data consistency

### ✅ Advanced Features (COMPLETE)

**References & Inheritance**

- ✅ **Anchors and aliases** (`&anchor`, `*alias`) with proper nested mapping support
- ✅ **Merge keys** (`<<`) - Complete inheritance support with override behavior
- ✅ **Smart serialization** - Automatic anchor/alias generation for shared values

**Text Processing**

- ✅ **Multi-line strings** - Literal (`|`) and folded (`>`) block scalars
- ✅ **Quote style preservation** - Single vs double quotes round-trip
- ✅ **Indentation style preservation** - Spaces vs tabs detection and preservation

**Type System**

- ✅ **Explicit type tags** (`!!str`, `!!int`, `!!map`, `!!seq`) with normalization
- ✅ **Complex key support** - Sequences and mappings as mapping keys

**Enterprise Features**

- ✅ **Complete comment correlation system** - Full round-trip comment preservation
- ✅ **Schema validation with custom rules** - Comprehensive validation framework
- ✅ **Enhanced error handling** - Contextual reporting with visual indicators

### ✅ **Comment Preservation (IMPLEMENTED)**

**Full Round-trip Comment Preservation**

- ✅ **Infrastructure complete** - CommentedValue, RoundTripConstructor, comment-aware scanner
- ✅ **API available** - `load_str_with_comments()`, `dump_str_with_comments()`
- ✅ **Comment correlation** - Comments properly correlated with parsed values  
- ✅ **Full round-trip** - Comments preserved during parse/serialize cycle
- ✅ **All comment types** - Leading, trailing, and inner comments supported

```rust
// Full comment preservation example
let config = YamlConfig { 
    preserve_comments: true, 
    loader_type: LoaderType::RoundTrip, 
    ..Default::default() 
};
let yaml = Yaml::with_config(config);

// Parse with full comment preservation
let commented_value = yaml.load_str_with_comments(yaml_with_comments)?;

// Serialize back with comments intact
let output = yaml.dump_str_with_comments(&commented_value)?;
// Output includes all original comments in appropriate positions
```

### ✅ **Schema Validation (IMPLEMENTED)**

**Enterprise-Grade Validation System**

- ✅ **Comprehensive rule system** - Type, pattern, range, length, enum validation
- ✅ **Complex validation logic** - Conditional (if-then-else), anyOf, allOf, oneOf, not
- ✅ **Object validation** - Property schemas, required fields, additional properties control
- ✅ **Array validation** - Item schemas, length constraints
- ✅ **Custom validators** - User-defined validation functions
- ✅ **Detailed error reporting** - Path-specific errors with human-readable messages
- ✅ **Integration with YAML API** - Direct validation during parsing

```rust
use rust_yaml::{Yaml, Schema, SchemaRule, ValueType};
use regex::Regex;
use std::collections::HashMap;

// Create comprehensive schema
let mut properties = HashMap::new();
properties.insert("name".to_string(), 
    Schema::with_type(ValueType::String)
        .rule(SchemaRule::Length { min: Some(1), max: Some(100) }));
properties.insert("email".to_string(), 
    Schema::with_type(ValueType::String)
        .rule(SchemaRule::Pattern(Regex::new(r"^[^@]+@[^@]+\.[^@]+$").unwrap())));
properties.insert("age".to_string(), 
    Schema::with_type(ValueType::Integer)
        .rule(SchemaRule::Range { min: Some(0.0), max: Some(150.0) }));

let schema = Schema::with_type(ValueType::Object)
    .rule(SchemaRule::Properties(properties))
    .rule(SchemaRule::Required(vec!["name".to_string(), "email".to_string()]))
    .info("User Schema", "Validates user information");

let yaml = Yaml::new();

// Load and validate in one step
let user_data = yaml.load_str_with_schema(r#"
name: "Alice Johnson"
email: "alice@example.com"
age: 30
"#, &schema)?;

// Or validate existing data
let parsed_data = yaml.load_str("name: Bob")?;
yaml.validate_with_schema(&parsed_data, &schema)?; // Will fail - missing email
```

### 🎯 Future Enhancements

**Enterprise Features**

- [ ] Custom type registration for Rust structs
- [ ] Plugin system for extensible functionality

**Performance & Developer Tools**

- [ ] Streaming optimizations for very large documents
- [ ] Configuration file editing utilities
- [ ] YAML formatting and linting tools
- [ ] Advanced profiling and debugging tools

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

### Development Setup

```bash

# Clone the repository
git clone https://github.com/elioetibr/rust-yaml.git
cd rust-yaml

# Set up development environment (git hooks, components, etc.)
make setup

# Run all tests
make test

# Run CI pipeline locally (same as GitHub Actions)
make ci

# Quick development checks (format, lint, test)
make quick-check
```

### Development Commands

The project includes a comprehensive Makefile with 60+ commands for development workflow:

**Testing**

```bash
make test              # Run all tests
make test-lib          # Run library tests only
make test-integration  # Run integration tests
make test-security     # Run security-specific tests
make test-release      # Run tests in release mode
```

**Code Quality**

```bash
make format           # Format code with rustfmt
make lint             # Run clippy lints
make clippy-strict    # Run strict clippy with CI settings
make audit            # Run security audit
make deny             # Run cargo deny checks
```

**Documentation & Reports**

```bash
make doc              # Build documentation
make doc-open         # Build and open documentation
make bench            # Run performance benchmarks
```

**Coverage Reports**

```bash
make coverage         # Generate test coverage report (CI-compatible)
make coverage-html    # Generate HTML coverage report
make coverage-view    # Generate and open HTML coverage in browser
make coverage-clean   # Clean coverage artifacts
make coverage-install-tools  # Install coverage tools

# Alternative coverage tools
make coverage-llvm    # Generate coverage using llvm-cov
make coverage-llvm-html  # Generate HTML coverage with llvm-cov
```

**CI/CD & Checks**

```bash
make ci               # Full CI pipeline (format, lint, test, security)
make check            # Basic checks (build, test, format, lint)
make check-all        # Comprehensive checks with audit and coverage
make release-check    # Check if ready for release
```

**Markdown & Documentation**

```bash
make check-markdown   # Check markdown formatting issues
make fix-markdown     # Fix common markdown formatting issues
make help             # Show all available commands
```

## Documentation

### Complete Documentation Index

The project includes comprehensive documentation in the [`docs/`](./docs/) directory:

**Core Documentation**

- [**Development Guide**](./docs/DEVELOPMENT.md) - Complete development setup and workflow
- [**Migration Guide**](./docs/MIGRATION_GUIDE.md) - Migrating from other YAML libraries
- [**Roadmap**](./docs/ROADMAP.md) - Current status and future planned features

**Feature Documentation**

- [**Merge Keys**](./docs/MERGE_KEYS.md) - Complete guide to YAML merge key inheritance (`<<`)
- [**Tag System**](./docs/TAG_SYSTEM.md) - Type tags and explicit typing system
- [**Directives**](./docs/DIRECTIVES.md) - YAML directives (`%YAML`, `%TAG`) support
- [**Zero-Copy Parsing**](./docs/ZERO_COPY.md) - Memory-efficient parsing techniques
- [**Streaming**](./docs/STREAMING.md) - Large document processing and async support

**Performance & Analysis**

- [**Performance Optimizations**](./docs/PERFORMANCE_OPTIMIZATIONS.md) - Technical performance details
- [**Benchmark Results**](./docs/BENCHMARK_RESULTS.md) - Comprehensive performance comparisons
- [**Library Comparison**](./docs/COMPARISON.md) - Comparison with other Rust YAML libraries

**Development Tools**

- [**Pre-commit Setup**](./docs/PRE_COMMIT.md) - Git hooks and code quality automation
- [**Version Management**](./docs/VERSION_MANAGEMENT.md) - Release process and versioning

**Project Files**

- [**Code of Conduct**](./CODE_OF_CONDUCT.md) - Community guidelines
- [**Contributing**](./CONTRIBUTING.md) - Contribution guidelines and process
- [**Security**](./SECURITY.md) - Security policy and vulnerability reporting

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

- Inspired by [ruamel-yaml](https://pypi.org/project/ruamel.yaml/) Python library and its focus on round-trip preservation
- Built following Rust security and performance best practices with zero unsafe code
- Thanks to the YAML 1.2 specification contributors for the comprehensive standard
- Grateful to the Rust community for excellent libraries like `indexmap` and `thiserror`

## Related Projects

- [serde_yaml](https://github.com/dtolnay/serde-yaml): Serde-based YAML library
- [yaml-rust](https://github.com/chyh1990/yaml-rust): Pure Rust YAML library
- [ruamel-yaml](https://pypi.org/project/ruamel.yaml/): Original Python implementation