Skip to main content

Module parallel

Module parallel 

Source
Expand description

Shared bounded-parallelism helper used by stages that run one subprocess per sub-config (makeself, nfpm, snapcraft, flatpak, upx, …).

The stages share the same Step 1 / Step 2 / Step 3 shape:

  1. Step 1 (serial, &mut ctx): render templates, stage files, collect a Vec<Job> of fully-owned work units.
  2. Step 2 (parallel, bounded by ctx.options.parallelism): run one subprocess per job in std::thread::scope.
  3. Step 3 (serial, &mut ctx): register the returned artifacts.

Before this helper every stage hand-rolled the Step 2 loop — for chunk in jobs.chunks(n) { thread::scope(|s| …) } with its own join-unwrap-or-panic handling. The pattern is now shared here so new parallelized stages just write run_job.

Semantics match the previous hand-rolled loops exactly:

  • Bounded concurrency: at most parallelism workers run at once, enforced by chunking the job list and scoping threads per-chunk.
  • Fail-fast within a chunk: if any worker in a chunk fails, the whole chunk still runs to completion (threads are already spawned), but the caller receives the first error and processes no further chunks. The completed siblings’ work is still accounted for: additional failures in the batch are logged, and a warn summarizes partial progress.
  • Panic-safe: a worker panic becomes an anyhow::Error annotated with stage_name, so a panicked thread doesn’t leave the pool deadlocked or drop all other results on the floor.
  • Order-preserving: results are collected in job-submission order, so downstream artifact registration remains deterministic.

Functions§

join_panic_to_err
Translate a thread::JoinHandle::join result’s panic payload into an anyhow::Error tagged with label. The two common panic payload shapes (&'static str / String) are downcast so the surfaced message is readable rather than the opaque Any placeholder.
lock_recover
Acquire a Mutex guard, recovering from poison rather than panicking.
run_parallel_chunks
Run run_job across jobs with bounded parallelism. Returns the per-job results in submission order.