Module batch 

Batch parallel ZIP extraction pipeline.

Reads multiple ZIP files into a ~200 MB buffer, resolves all entries (Central Directory parse), then dispatches inflate work across N workers. While workers inflate batch N, the reader fills the buffer with batch N+1.

This amortises I/O, Central Directory parsing, and rayon dispatch overhead across many ZIPs — achieving throughput that scales with core count.

Architecture

Reader thread                        Worker pool (8 cores)
────────────────                     ─────────────────────
read ZIP₁..ZIPₖ into buf            inflate entries from prev buf
parse CDs → entry list              write decompressed to disk
partition entries across workers     ↓
         ──── swap buffers ────→     receive new entry list

Structs

BatchStats

Statistics returned after batch extraction.

Functions

decompress_zips

Extract multiple ZIPs in-memory (no disk write). Returns all entries.

extract_zips

Extract multiple ZIP files in parallel using a pipelined batch approach.