oxiarc-deflate [Stable]

Pure Rust implementation of the DEFLATE compression algorithm (RFC 1951).

Version 0.3.1 (2026-05-30) — 212 tests passing.

What's new in 0.3.x (latest):

• Parallel GZIP compression (parallel feature): pigz-style multi-member GZIP via gzip_compress_parallel() and ParallelGzipEncoder builder.
• LZ77 match heuristics tuning: Lz77Params / Lz77Preset structs and Deflater::with_lz77_params() / Deflater::with_lz77_preset() builder methods for fine-grained speed/ratio trade-offs.
• DeflatePool memory pool: DeflatePool for thread-safe window/hash buffer reuse with Deflater::with_pool() and PoolStats tracking.
• OptimalParser: Zopfli-style graph-based optimal DEFLATE parsing strategy. Enable via Deflater::with_optimal_parsing(level).

What's new in 0.2.8: Added async streaming support for raw DEFLATE with RawDeflateWriter and RawInflateReader (requires async-io feature).

What's new in 0.2.6: Added streaming support with GzipStreamEncoder/GzipStreamDecoder and ZlibStreamEncoder/ZlibStreamDecoder with flush modes for fine-grained control over compressed output.

Overview

DEFLATE is the compression algorithm used in:

• ZIP archives
• GZIP compressed files
• PNG images
• HTTP compression
• Many other formats

This crate provides both compression and decompression with no external dependencies.

Features

• Pure Rust - No C bindings or unsafe code
• Full RFC 1951 compliance - All block types supported
• Compression levels 0-9 - From stored to maximum compression
• Streaming API - Process data in chunks
• One-shot API - Convenient functions for simple cases
• Async I/O - async_deflate module with Tokio-based async streaming (enable async-io feature)
• GZIP support - gzip module for RFC 1952 GZIP format encoding/decoding
• Parallel GZIP compression - pigz-style multi-member GZIP using multiple threads (enable parallel feature)
• LZ77 heuristics tuning - Lz77Params / Lz77Preset for speed/ratio trade-off control
• Memory pool - DeflatePool for reusing window/hash buffers across compression calls

All features are implemented and tested. API is stable.

Cargo Features

Usage

Add to your Cargo.toml:

[dependencies]
oxiarc-deflate = "0.3.1"

With async I/O support:

[dependencies]
oxiarc-deflate = { version = "0.3.1", features = ["async-io"] }

With parallel GZIP compression:

[dependencies]
oxiarc-deflate = { version = "0.3.1", features = ["parallel"] }

Quick Start

use oxiarc_deflate::{deflate, inflate};

// Compress data
let original = b"Hello, World! Hello, World! Hello, World!";
let compressed = deflate(original, 6)?;  // Level 6 (default)

// Decompress data
let decompressed = inflate(&compressed)?;
assert_eq!(&decompressed, original);

Compression Levels

API

High-Level Functions

// Compress with specified level
let compressed = deflate(data, level)?;

// Decompress
let decompressed = inflate(&compressed)?;

Streaming API

use oxiarc_deflate::{Deflater, Inflater};

// Streaming compression
let mut deflater = Deflater::new(6);
let compressed = deflater.compress_all(data)?;

// Streaming decompression
let mut inflater = Inflater::new();
loop {
    let (consumed, produced, status) = inflater.decompress(input, output)?;
    // Handle status...
}

LZ77 Encoder

use oxiarc_deflate::{Lz77Encoder, Lz77Token};

let mut encoder = Lz77Encoder::new(6);
for token in encoder.encode(data) {
    match token {
        Lz77Token::Literal(byte) => { /* literal byte */ }
        Lz77Token::Match { length, distance } => { /* back-reference */ }
    }
}

Huffman Trees

use oxiarc_deflate::{HuffmanTree, HuffmanBuilder};

// Build tree from code lengths
let lengths = [3, 3, 3, 3, 3, 2, 4, 4];
let tree = HuffmanTree::from_lengths(&lengths)?;

// Decode symbols
let symbol = tree.decode(&mut bit_reader)?;

Algorithm Details

DEFLATE Structure

+------------------+
| Block Header     | (3 bits: BFINAL + BTYPE)
+------------------+
| Block Data       | (varies by type)
+------------------+
| ... more blocks  |
+------------------+

Block Types

LZ77 Parameters

• Window size: 32768 bytes (32KB)
• Match length: 3-258 bytes
• Match distance: 1-32768 bytes
• Minimum match: 3 bytes

Huffman Alphabets

Literal/Length (286 symbols):

• 0-255: Literal bytes
• 256: End of block
• 257-285: Length codes (3-258)

Distance (30 symbols):

• 0-29: Distance codes (1-32768)

Fixed Huffman Code Lengths

Modules

GZIP API

use oxiarc_deflate::gzip::{gzip_encode, gzip_decode};

// Encode to GZIP format
let compressed = gzip_encode(data, 6)?;

// Decode GZIP data
let decompressed = gzip_decode(&compressed)?;

Async DEFLATE (requires async-io feature)

use oxiarc_deflate::async_deflate::{AsyncDeflater, AsyncInflater};
use tokio::io::BufReader;

// Async compression
let mut deflater = AsyncDeflater::new(6);
let compressed = deflater.compress_all(reader).await?;

// Async decompression
let mut inflater = AsyncInflater::new();
let decompressed = inflater.decompress_all(reader).await?;

Parallel GZIP (requires parallel feature)

use oxiarc_deflate::gzip::{gzip_compress_parallel, ParallelGzipEncoder};

// One-shot parallel GZIP (pigz-style multi-member output)
let compressed = gzip_compress_parallel(data, 6, 1 << 17)?; // level 6, 128 KB chunks

// Builder API
let compressed = ParallelGzipEncoder::new()
    .level(6)
    .chunk_size(1 << 17)   // 128 KB per chunk
    .num_threads(4)
    .encode(data)?;

LZ77 Heuristics Tuning

use oxiarc_deflate::{Deflater, Lz77Params, Lz77Preset};

// Use a preset
let compressed = Deflater::new(9)
    .with_lz77_preset(Lz77Preset::Ultra)
    .compress_all(data)?;

// Fine-grained control
let params = Lz77Params {
    nice_length: 128,
    max_chain: 256,
    good_length: 32,
};
let compressed = Deflater::new(9)
    .with_lz77_params(params)
    .compress_all(data)?;

Available presets:

DeflatePool (memory pool)

use oxiarc_deflate::{Deflater, DeflatePool};

// Create a shared pool (e.g., once per application / thread pool)
let pool = DeflatePool::new();

// Reuse window/hash buffers across calls — reduces allocations
let compressed = Deflater::new(6)
    .with_pool(&pool)
    .compress_all(data)?;

// Inspect pool statistics
let stats = pool.stats();
println!("hits={} allocs={}", stats.hits, stats.allocations);

Performance

Compression ratios on typical data (Calgary Corpus):

References

• RFC 1951 - DEFLATE Compressed Data Format Specification
• An Explanation of the Deflate Algorithm

License

Apache-2.0