calculate_checksums_parallel

agpm_cli::utils::fs

Function calculate_checksums_parallel 

Source 
pub async fn calculate_checksums_parallel(
    paths: &[PathBuf],
) -> Result<Vec<(PathBuf, String)>>
Expand description

Calculates SHA-256 checksums for multiple files concurrently.

This function processes multiple files in parallel using Tokio’s thread pool, which can significantly improve performance when processing many files or large files on systems with multiple CPU cores.

§Arguments

  • paths - A slice of file paths to process

§Returns

A vector of tuples containing each file path and its corresponding checksum, in the same order as the input paths. Returns an error if any file fails to be processed.

§Examples

use agpm_cli::utils::fs::calculate_checksums_parallel;
use std::path::PathBuf;

let files = vec![
    PathBuf::from("file1.txt"),
    PathBuf::from("file2.txt"),
    PathBuf::from("file3.txt"),
];

let results = calculate_checksums_parallel(&files).await?;
for (path, checksum) in results {
    println!("{}: {}", path.display(), checksum);
}

§Performance

This function uses tokio::task::spawn_blocking to run checksum calculations on separate threads, allowing for true parallelism. Benefits:

  • CPU-bound work doesn’t block the async runtime
  • Multiple files processed simultaneously
  • Scales with available CPU cores
  • Maintains order of results

§Error Handling

If any file fails to be processed, the entire operation fails and returns an error with details about all failures. This “all-or-nothing” approach ensures data consistency.

§See Also