lzip-rs
"Allfadern ser varje bit"
Pure Rust parallel ZIP decompressor for Windows/Linux/macOS .zip files. Reads the ZIP Central Directory, then decodes all DEFLATE entries in parallel using linflate (shared SIMD inflate engine).
1.8× faster than unzip -t on multi-core hardware.
Performance
| Workload | lzip | unzip | Speedup |
|---|---|---|---|
| 2000-entry ZIP (6.9 MB compressed) | 1.86 s | 5.37 s | 1.8× |
- Each ZIP entry is decoded independently (ZIP format is fully parallel by design)
- Large entries are also split internally at DEFLATE full-flush boundaries
Usage
Architecture
- Parse ZIP End-of-Central-Directory (EOCD)
- Read Central Directory → build entry list
- For each entry: resolve local header → get compressed data pointer
- Parallel decode all entries via rayon (large entries also split internally at flush boundaries)
- Write/output in order
Ring-slot pipeline
Same architecture as lbzip2-rs / lgz-rs:
- 6 slots × 232 MB (200 MB data + 32 MB carry headroom)
- AVX-512 / AVX2 / scalar SIMD flush scanner for intra-entry parallelism
- Dedicated reader, decoder (N threads), collector, and writer threads
- Zero-copy compressed input: workers read directly from ring-buffer slots
Dependencies
| Crate | Role |
|---|---|
| linflate | SIMD DEFLATE decode (shared with lgz-rs) |
| rayon | Parallel entry decode |
Optional features
zlib-ng— use zlib-ng backend via flate2 (requires C toolchain)gen— ZIP generation utilities (dev/testing)
Sister projects
All at codeberg.org/nordisk/znippy:
- lbzip2-rs — parallel bzip2 decompressor
- lgz-rs — parallel gzip decompressor
- ljar-rs — parallel JAR decompressor
- linflate-rs — shared SIMD DEFLATE engine
License
MIT OR Apache-2.0