read_files_parallel

agpm_cli::utils::fs::parallel

Function read_files_parallel 

Source 
pub async fn read_files_parallel(
    paths: &[PathBuf],
) -> Result<Vec<(PathBuf, String)>>
Expand description

Reads multiple files concurrently and returns their contents.

This function reads multiple text files in parallel, which can improve performance when processing many files, especially on systems with fast storage and multiple CPU cores.

§Arguments

  • paths - A slice of file paths to read

§Returns

A vector of tuples containing each file path and its content as a UTF-8 string, in the same order as the input paths. Returns an error if any file fails to be read or contains invalid UTF-8.

§Examples

use agpm_cli::utils::fs::parallel::read_files_parallel;
use std::path::PathBuf;

let config_files = vec![
    PathBuf::from("agpm.toml"),
    PathBuf::from("agents/agent1.md"),
    PathBuf::from("snippets/snippet1.md"),
];

let results = read_files_parallel(&config_files).await?;
for (path, content) in results {
    println!("{}: {} characters", path.display(), content.len());
}

§Performance

This function uses tokio::task::spawn_blocking to perform file I/O on separate threads:

  • Multiple files read simultaneously
  • Non-blocking for the async runtime
  • Scales with available CPU cores and I/O bandwidth
  • Maintains result ordering

§UTF-8 Handling

All files must contain valid UTF-8 text. If any file contains invalid UTF-8 bytes, the operation will fail with a descriptive error.

§Error Handling

If any file fails to be read (due to permissions, missing file, or invalid UTF-8), the entire operation fails. This ensures consistency when processing related files that should all be available.

§See Also