Module batch 

Batch processing for multiple HEDL files with parallel execution and progress reporting.

This module provides efficient batch processing capabilities for operations on multiple HEDL files. It uses Rayon for parallel processing when beneficial and provides real-time progress reporting with detailed error tracking.

Features

• Parallel Processing: Automatic parallelization using Rayon’s work-stealing scheduler
• Progress Reporting: Real-time progress with file counts and success/failure tracking
• Error Resilience: Continues processing on errors, collecting all failures for reporting
• Performance Optimization: Intelligent parallel/serial mode selection based on workload
• Type Safety: Strongly typed operation definitions with compile-time guarantees

Architecture

The batch processing system uses a functional architecture with:

• Operation trait for extensible batch operations
• Result aggregation with detailed error context
• Atomic counters for thread-safe progress tracking
• Zero-copy file path handling

Examples

use hedl_cli::batch::{BatchProcessor, BatchConfig, ValidationOperation};
use std::path::PathBuf;

// Create a batch processor with default configuration
let processor = BatchProcessor::new(BatchConfig::default());

// Validate multiple files in parallel
let files = vec![
    PathBuf::from("file1.hedl"),
    PathBuf::from("file2.hedl"),
    PathBuf::from("file3.hedl"),
];

let operation = ValidationOperation { strict: true };
let results = processor.process(&files, operation, true)?;

println!("Processed {} files, {} succeeded, {} failed",
    results.total_files(),
    results.success_count(),
    results.failure_count()
);

Performance Characteristics

• Small batches (< 10 files): Serial processing to avoid overhead
• Medium batches (10-100 files): Parallel with Rayon thread pool
• Large batches (> 100 files): Chunked parallel processing with progress updates

Thread Safety

All progress tracking uses atomic operations for lock-free concurrent access. Operations are required to be Send + Sync for parallel execution.

Structs

BatchConfig

Configuration for batch processing operations.

BatchProcessor

High-performance batch processor for HEDL files.

BatchResults

Aggregated results from a batch processing operation.

FileResult

Result of processing a single file in a batch operation.

FormatOperation

Batch format operation.

LintOperation

Batch lint operation.

ValidationOperation

Batch validation operation.

Traits

BatchOperation

Trait for batch operations on HEDL files.