lzip-parallel 0.2.0

Pure Rust parallel ZIP decompressor — multi-core DEFLATE decode for .zip archives
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

# lzip-rs

*"Allfadern ser varje bit"*

Pure Rust parallel ZIP decompressor for Windows/Linux/macOS .zip files. Reads
the ZIP Central Directory, then decodes all DEFLATE entries in parallel using
[linflate](https://github.com/Ignalina/linflate-rs) (shared SIMD inflate engine).

**1.8× faster than `unzip -t`** on multi-core hardware.

## Performance

| Workload | lzip | unzip | Speedup |
|----------|------|-------|---------|
| 2000-entry ZIP (6.9 MB compressed) | 1.86 s | 5.37 s | **1.8×** |

- Each ZIP entry is decoded independently (ZIP format is fully parallel by design)
- Large entries are also split internally at DEFLATE full-flush boundaries

## Usage

```bash
lzip archive.zip -o dir/    # extract to directory
lzip archive.zip            # list entries to stdout
```

## Architecture

1. Parse ZIP End-of-Central-Directory (EOCD)
2. Read Central Directory → build entry list
3. For each entry: resolve local header → get compressed data pointer
4. Parallel decode all entries via rayon (large entries also split internally at flush boundaries)
5. Write/output in order

### Ring-slot pipeline

Same architecture as lbzip2-rs / lgz-rs:

- **6 slots × 232 MB** (200 MB data + 32 MB carry headroom)
- AVX-512 / AVX2 / scalar SIMD flush scanner for intra-entry parallelism
- Dedicated reader, decoder (N threads), collector, and writer threads
- Zero-copy compressed input: workers read directly from ring-buffer slots

## Dependencies

| Crate | Role |
|-------|------|
| [linflate](https://github.com/Ignalina/linflate-rs) | SIMD DEFLATE decode (shared with lgz-rs) |
| [rayon](https://crates.io/crates/rayon) | Parallel entry decode |

### Optional features

- `zlib-ng` — use zlib-ng backend via flate2 (requires C toolchain)
- `gen` — ZIP generation utilities (dev/testing)

## Sister projects

All at [github.com/Ignalina](https://github.com/Ignalina/):

- [lbzip2-rs](https://github.com/Ignalina/lbzip2-rs) — parallel bzip2 decompressor
- [lgz-rs](https://github.com/Ignalina/lgz-rs) — parallel gzip decompressor
- [ljar-rs](https://github.com/Ignalina/ljar-rs) — parallel JAR decompressor
- [linflate-rs](https://github.com/Ignalina/linflate-rs) — shared SIMD DEFLATE engine

## License

MIT OR Apache-2.0