Skip to main content

Module parallel

Module parallel 

Source
Expand description

Shared bounded-parallelism helper used by stages that run one subprocess per sub-config (makeself, nfpm, snapcraft, flatpak, upx, …).

The stages share the same Phase 1 / Phase 2 / Phase 3 shape:

  1. Phase 1 (serial, &mut ctx): render templates, stage files, collect a Vec<Job> of fully-owned work units.
  2. Phase 2 (parallel, bounded by ctx.options.parallelism): run one subprocess per job in std::thread::scope.
  3. Phase 3 (serial, &mut ctx): register the returned artifacts.

Before this helper every stage hand-rolled the Phase 2 loop — for chunk in jobs.chunks(n) { thread::scope(|s| …) } with its own join-unwrap-or-panic handling. The pattern is now shared here so new parallelized stages just write run_job.

Semantics match the previous hand-rolled loops exactly:

  • Bounded concurrency: at most parallelism workers run at once, enforced by chunking the job list and scoping threads per-chunk.
  • Fail-fast within a chunk: if any worker in a chunk fails, the whole chunk still runs to completion (threads are already spawned), but the caller receives the first error and processes no further chunks.
  • Panic-safe: a worker panic becomes an anyhow::Error annotated with stage_name, so a panicked thread doesn’t leave the pool deadlocked or drop all other results on the floor.
  • Order-preserving: results are collected in job-submission order, so downstream artifact registration remains deterministic.

Functions§

run_parallel_chunks
Run run_job across jobs with bounded parallelism. Returns the per-job results in submission order.