ZEXC
zexc is a modern, pure-Rust lossless compression library and command-line tool. It implements an original codec and container format (.zex, magic ZEX1)—not a wrapper around LZ4 or Zstandard.
| Author | Ali Zain <alizain@zainiumdynamics.tech> |
| Website | zainiumdynamics.tech |
| Language | Rust 2024 (MSRV 1.85) |
| License | GPL-3.0-only — GNU GPLv3 strict (no “or later”) |
| crates.io / crate | zexc (use zexc::…) |
| CLI binary | zexc |
| C library | libzexc / zexc.h |
| File extension | .zex |
| Version | 1.2.0 (see crates.io/crates/zexc) |
Highlights
- Native algorithm — hash-chain match finding, lazy matching, dual hashing, optional Huffman literals, and rANS sequence coding with repeat offsets
- Performance profiles —
turbo(speed),balanced(default),max(ratio), levels 1–22 - SIMD-oriented hot paths — wide
match_lenand fast RLE-aware copies - Streaming I/O —
StreamEncoder/StreamDecoderimplementingstd::io::Write/Read - Dictionary history — shared dictionary as virtual prefix for small-message workloads
- Optional AEAD — ChaCha20-Poly1305 with BLAKE3 password-based key derivation
- Safe defaults — CRC32 content checksum, independent blocks, hard size caps
- Production CLI — compress, decompress, info, train-dict; stdin/stdout pipes via
-
ZEX is designed for systems that need a self-contained compressor: auditable Rust, no libzstd / liblz4 dependency, and a clear path from library API to CLI and packaging.
Table of contents
- Installation
- Command-line usage
- Library usage
- Language bindings
- Compression profiles
- Security model
- Project layout
- Documentation
- Building, testing, and benchmarks
- Performance expectations
- Roadmap
- License
Build configuration (zexc.toml)
Shared vs static vs CLI is not hard-coded in the Makefile. Edit zexc.toml:
[]
= true # libzexc.so
= true # libzexc.a
= true
[]
= true # CLI: zexc
[]
= true
= true
= true
See docs/CONFIG.md.
Installation
From source
# binary: target/release/zexc
Install the CLI
From crates.io (after publish):
From source:
# or
Feature flags
| Feature | Default | Description |
|---|---|---|
std |
yes | Standard library support |
simd |
yes | Wide-load match helpers |
encryption |
yes | ChaCha20-Poly1305 + password KDF |
cli |
yes | Build the zexc binary |
c-api |
yes | C ABI (libzexc / zexc.h) |
# Minimal build without encryption
Command-line usage
Primary form (positional input and output):
zexc <COMMAND> ...
Commands:
compress input output.zex
decompress input.zex output
info file.zex
test file.zex
train-dict out.dict samples...
archive create|list|extract
seek-compress input output.zexs
bench file
Options (common)
| Flag | Meaning |
|---|---|
-l N |
Level 1–22 |
-m turbo|balanced|max |
Profile |
-e PASS |
Encrypt / decrypt password |
-D dict |
Dictionary file |
-T N |
Threads |
-q |
Quiet |
--rm |
Delete input after success |
Pipes
|
Dictionary
Archive
Library usage
Add the package to your Cargo.toml. crates.io name and import are both zexc.
[]
= "1.1"
One-shot API
use ;
Streaming API
use ;
use ;
See docs/STREAMING.md for lifecycle rules (always call finish() on encode). Streaming does not support whole-frame AEAD; use the one-shot API for encrypted frames.
Dictionary and encryption (library)
use ;
use EncryptionKey;
let dict = train_simple?;
let opts = default.level.dictionary;
let opts = opts.encrypt;
let frame = compress_with_dict?;
let plain = decompress_with_options?;
More examples: examples/simple_compress.rs, examples/stream_compress.rs, examples/dictionary_train.rs.
Language bindings
| Language | Path | Build |
|---|---|---|
| Python (PyO3 / maturin) | bindings/python |
pip install maturin && cd bindings/python && maturin develop --release |
| Go (cgo, over the C ABI) | bindings/go |
cargo build --release --features c-api && cd bindings/go && go test ./... |
Both wrap the same native codec — no libzstd/liblz4 anywhere in the call chain.
See each directory's README for the full API and licensing notes (GPLv3 applies
to distributed binaries/wheels linking libzexc, same as the C ABI).
Compression profiles
| Profile | CLI / API | Intended use |
|---|---|---|
| Turbo | -m turbo / CompressionMode::Turbo |
Latency-sensitive paths; shallow search |
| Balanced | -m balanced (default) |
General-purpose default |
| Max | -m max |
Prefer ratio; deeper search and entropy |
Levels 1–22 scale hash table size, chain depth, lazy matching, dual-hash, literal Huffman, and sequence rANS. Exact parameters are documented in docs/ARCHITECTURE.md.
Security model
| Layer | Behavior |
|---|---|
| Memory safety | Codec implemented in safe Rust (no unsafe in the match/copy hot path) |
| Integrity | Optional but default CRC32 over plaintext |
| Confidentiality | Optional ChaCha20-Poly1305 over the compressed body |
| DoS bounds | SecurityPolicy caps on block size and total output |
| Match validation | Offsets and lengths checked against history (including dictionary) |
Encryption authenticates the compressed payload; a wrong password fails closed (authentication error). Dictionary id mismatches also fail closed.
Use .secure() / --secure for stricter absolute size limits.
Project layout
zexc/ (this tree may still be named zex-compressor on disk)
├── src/
│ ├── algorithms/zex_native.rs # core compressor / decompressor
│ ├── core/ # frame encode, decode, stream, dictionary
│ ├── format/ # header and block wire format
│ ├── utils/ # SIMD, Huffman, rANS/FSE, CRC, security
│ ├── crypto/ # AEAD + KDF (feature-gated)
│ ├── cli/ # clap definitions
│ ├── lib.rs
│ └── main.rs # zexc binary
├── include/zexc.h # C API
├── bindings/
│ ├── python/ # PyO3 / maturin
│ └── go/ # cgo, over the C API
├── docs/ # architecture, format, API, packaging
├── examples/
├── benches/
├── scripts/
├── fuzz/ # cargo-fuzz targets + seed corpus
├── tests/
├── zexc.toml # shared/static/CLI build config
├── Makefile
├── ROADMAP.md
└── README.md
Documentation
| Document | Description |
|---|---|
| docs/ARCHITECTURE.md | System design, data path, module map |
| docs/FORMAT.md | Binary .zex specification |
| docs/API.md | Library API guide |
| docs/STREAMING.md | Streaming encoder/decoder |
| docs/BENCHMARKS.md | How to measure vs zstd/lz4 |
| docs/PACKAGING.md | Install, features, release tarballs |
| docs/zex.1 | Manual page |
| ROADMAP.md | Near- and long-term plan |
Building, testing, and benchmarks
# Unit, integration, and doctests
# or: make test
# Optimized binary
# Criterion microbenchmarks (ZEX only)
# ZEX vs external zstd CLI (if installed)
# Shell comparison table vs zstd (and lz4 if present)
# Package
Performance expectations
ZEX is under active development. Mature C implementations (LZ4, Zstd) benefit from years of tuning; absolute “always faster and smaller than both” claims are not the project goal.
Design targets:
| Mode | Goal |
|---|---|
| Turbo | LZ4-class throughput orientation |
| Balanced | Practical default for mixed data |
| Max | Spend CPU for higher ratio |
| Secure / AEAD | Integrity and confidentiality with acceptable overhead |
Always benchmark on your data and hardware:
Roadmap
Planned work includes fuzzing and golden corpora, optimal parsing at high levels, turbo SIMD specialization, COVER-style dictionary training, seekable frames, and a stable C ABI. Details and sprint ordering live in ROADMAP.md.
Contributing
By contributing, you agree that your contributions are licensed under the same
GPL-3.0-only terms as the rest of the project (no dual-license unless agreed in writing).
- Keep the codec native (no silent dependency on libzstd/liblz4 for the default path).
- Prefer additive wire flags; do not break existing
ZEX1decoders without a version bump. - Add tests for format and round-trip behavior.
- Update
docs/FORMAT.mdwhen the on-wire layout changes. - Measure non-trivial performance claims with the provided bench scripts.
License
ZEX / zexc is free software licensed under the
GNU General Public License version 3 only
(SPDX: GPL-3.0-only).
| License | GNU General Public License v3 only |
| SPDX | GPL-3.0-only |
| Full text | LICENSE |
| “or later”? | No — version 3 only (not GPL-3.0-or-later) |
Copyright (C) 2026 Ali Zain <alizain@zainiumdynamics.tech>
Website: https://zainiumdynamics.tech
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License ONLY.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Author / contact
| Name | Ali Zain |
| alizain@zainiumdynamics.tech | |
| Website | https://zainiumdynamics.tech |
What this means (short)
- You may use, study, modify, and redistribute ZEX under GPLv3.
- If you distribute binaries or derived works, you must provide corresponding source under the same GPLv3-only terms.
- Linking this library (including
libzexc/ the C API) into a larger program typically makes that combination a covered work under GPLv3 — proprietary closed-source distribution is not allowed without a separate commercial license (none is offered here by default). - There is no warranty.
Design ideas were informed by the open specifications and literature around LZ4 and Zstandard; the implementation and .zex format are original.