Skip to main content

Module parallel

Module parallel 

Source
Expand description

Shared bounded-parallelism helper used by stages that run one subprocess per sub-config (makeself, nfpm, snapcraft, flatpak, upx, …).

The stages share the same Step 1 / Step 2 / Step 3 shape:

  1. Step 1 (serial, &mut ctx): render templates, stage files, collect a Vec<Job> of fully-owned work units.
  2. Step 2 (parallel, bounded by ctx.options.parallelism): run one subprocess per job in std::thread::scope.
  3. Step 3 (serial, &mut ctx): register the returned artifacts.

Before this helper every stage hand-rolled the Step 2 loop — for chunk in jobs.chunks(n) { thread::scope(|s| …) } with its own join-unwrap-or-panic handling. The pattern is now shared here so new parallelized stages just write run_job.

Semantics match the previous hand-rolled loops exactly:

  • Bounded concurrency: at most parallelism workers run at once, enforced by chunking the job list and scoping threads per-chunk.
  • Fail-fast within a chunk: if any worker in a chunk fails, the whole chunk still runs to completion (threads are already spawned), but the caller receives the first error and processes no further chunks.
  • Panic-safe: a worker panic becomes an anyhow::Error annotated with stage_name, so a panicked thread doesn’t leave the pool deadlocked or drop all other results on the floor.
  • Order-preserving: results are collected in job-submission order, so downstream artifact registration remains deterministic.

Functions§

join_panic_to_err
Translate a thread::JoinHandle::join result’s panic payload into an anyhow::Error tagged with label. The two common panic payload shapes (&'static str / String) are downcast so the surfaced message is readable rather than the opaque Any placeholder.
lock_recover
Acquire a Mutex guard, recovering from poison rather than panicking.
run_parallel_chunks
Run run_job across jobs with bounded parallelism. Returns the per-job results in submission order.