fast-yaml-parallel 0.1.0

Multi-threaded YAML processing with work-stealing parallelism
Documentation

fast-yaml-parallel: Multi-threaded YAML processing.

This crate provides parallel parsing for multi-document YAML streams, leveraging Rayon for work-stealing parallelism.

Performance

Expected speedup on multi-document files:

  • 4 cores: 3-3.5x faster
  • 8 cores: 6-6.5x faster
  • 16 cores: 10-12x faster

When to Use

Use parallel processing when:

  • Processing multi-document YAML streams (logs, configs, data dumps)
  • Input size > 1MB with multiple documents
  • Running on multi-core hardware (4+ cores recommended)

Use sequential processing when:

  • Single document files
  • Small files (<100KB)
  • Memory constrained environments

Examples

Basic usage:

use fast_yaml_parallel::parse_parallel;

let yaml = "---\nfoo: 1\n---\nbar: 2\n---\nbaz: 3";
let docs = parse_parallel(yaml)?;
assert_eq!(docs.len(), 3);
# Ok::<(), Box<dyn std::error::Error>>(())

Custom configuration:

use fast_yaml_parallel::{parse_parallel_with_config, ParallelConfig};

let config = ParallelConfig::new()
    .with_thread_count(Some(8))
    .with_min_chunk_size(2048);

let yaml = "---\nfoo: 1\n---\nbar: 2";
let docs = parse_parallel_with_config(yaml, &config)?;
# Ok::<(), Box<dyn std::error::Error>>(())