# znippy
> A parallel, random-access archive: pack a directory at all-core speed, pull any
> single file back without unpacking the rest — and query the index in
> **DuckDB / Polars / DataFusion**, because it's just Arrow IPC.
⚡ **GATLING** — one all-core engine · zero rayon · zero-copy · ~11/12 cores
## Mashup — one archive vs tar/zip/7z/parquet
znippy is the only archive that's all-core on *both* ends, random-access per file,
**and** directly queryable — without pretending to beat zstd/7z on raw ratio:
| All-core **compress** | **✅** | ~ (`-T0`) | ◐ | ✗ | ✅ |
| All-core **decompress** | **✅** | ✗ serial | ✗ | ✗ | ✅ |
| Random single-file read | **✅** | ✗ | ✅ | ✗ (solid) | — (columnar) |
| **Query without the tool** (DuckDB / Polars) | **✅** | ✗ | ✗ | ✗ | ✅ |
| Best pure **ratio** | ◐ | ✅ (`-19`) | ✗ | ✅ | — |
*The full capability matrix and every measured per-backend throughput table (JAR /
ZIP / gzip / bzip2 decode vs the legacy C tools) are in the full overview.*
### Measured — znippy vs Arrow-IPC / zstd
One byte-identical mixed corpus (compressible text + incompressible blobs) packed
into a single compressed, queryable Arrow-IPC container two ways — znippy's
all-core pipeline vs a stock arrow-rs `FileWriter` with `CompressionType::ZSTD` —
then fully read back. Same box, same run, warmup + iterations:
**v0.9.0 · threadripper-3975wx · 32 cores · 2026-06-18**
`footer_seal_l` — **1.30** delta_pct · **1M** rows · **0.58** seal_mrows_s _(single-system bench; competitive matrix in [`README-full`](.nornir/README-full.md))_
*Not the same job: znippy stores every file as its own random-access, blake3-verified
chunk behind a queryable FST index and an immutable seal; the Arrow-IPC/zstd rival
concatenates ~1000 files per record-batch into one zstd stream — smaller granularity
traded away, no per-file addressing. So arrow's simpler batching wins raw
compress throughput while znippy wins decompress and keeps capabilities arrow can't
offer; both hit the same ratio on identical distinct data. Each side uses its
library's default codec settings.*
## Read the real docs
- 📖 **[The complete manual — `docs/manual.pdf`](docs/manual.pdf)** — every subsystem chapter (format, Gatling pipeline, random-access, Iceberg/seal, decompressors).
- 📄 **[Full overview — `.nornir/README-full.md`](.nornir/README-full.md)** — the detailed pitch, quickstart, what's-new, the full competitive matrix, and all measured benchmarks.
- 📝 [`docs/book.md`](docs/book.md) — the manual, as Markdown.