Module parwrite 

Parallel-writer gatling — the shared extract/decode/write fan-out that removes the single ordered writer from the critical path (independent files, or independent regions of one pre-sized stream). Unix-only: it uses positional pread/pwrite (std::os::unix::fs::FileExt) so N workers read and write disjoint regions concurrently with no shared file offset. Parallel-writer gatling — the shared extract/decode/write fan-out.

The streaming crate::gatling engine and the fork-join crate::gatling_forkjoin both funnel their decoded output through a single ordered sink (one collector thread draining an mpsc, one writer). For codecs whose output units are independent files or independent regions of one pre-sized stream, that single writer is the last un-parallelised stage — the serial drain that leaves cores idle while one thread copies and writes gigabytes.

This module removes it. It is the parallel-writer sibling of gatling_for_each: the same no-barrier, self-dispatching (atomic-cursor) worker pool built on std::thread::scope, but each worker also writes its own output — there is no collector and no writer thread on the critical path.

Two shapes, both format-agnostic (the caller plugs in the decode):

• [extract_entries_unordered] — N workers self-dispatch independent entries (ZIP/JAR members). Each worker preads its entry’s compressed bytes from the shared input (thread-safe positional reads, no shared file offset), decodes, and writes its own output file. Entries are independent, so the file writes never conflict and no ordering is needed.

• [write_segments_positional] — N workers self-dispatch independent segments of one output stream (concatenated gzip members). The caller pre-computes each segment’s byte offset (a prefix-sum of known output sizes) and pre-sizes the output file; each worker decodes its segment and pwrites it at the segment’s offset. Exact byte order is preserved with no ordered collector and no serial concatenation.

Both reuse each worker’s scratch buffers across the units it claims (zero-alloc hot loop after warm-up), never copy input into the pool (the decode reads directly from the shared source), and are rayon-free: pure std::thread::scope + one AtomicUsize cursor — the gatling soul.

Structs

EntryJob

One independent extraction unit for extract_entries_unordered: read comp_len compressed bytes at data_offset from the shared input, decode, and write the result to out_dir.join(name).

WriteStats

Outcome of a parallel-writer run: how many units were written and how many failed (a failed unit is logged to stderr and skipped, never aborting the siblings).

Functions

extract_entries_unordered

Extract independent entries across a no-barrier, self-dispatching worker pool — each worker decodes AND writes its own output file, with no ordered collector and no single writer thread.

write_segments_positional

Write independent segments of one output stream at pre-computed offsets, across a no-barrier, self-dispatching worker pool — each worker decodes AND pwrites its own segment, with no ordered collector and no serial concatenation.

write_slices_positional

Write already-decoded slices of one output stream at pre-computed offsets, across a no-barrier, self-dispatching worker pool — zero-copy parallel pwrite, no ordered collector and no serial concatenation.