alice-zip 0.5.1

Procedural Generation Compression Engine Store algorithms, not data.

日本語版 README

ALICE-Zip is a next-generation compression tool that stores "how to generate the data" instead of the data itself.

For patterns, waves, and mathematical data, it achieves compression ratios of 10x to 1000x. For everything else, it falls back to LZMA, ensuring it's never worse than standard tools.

Features

• Procedural Compression: Sine waves, polynomials, and mathematical patterns
• Adaptive Fallback: Automatically selects LZMA when procedural methods don't help
• Lossless: Bit-perfect reconstruction
• Cross-Platform: Python, Rust, C#/Unity, C++/UE5

Repository layout

Installation

# Python
pip install alice-zip

# Rust
cargo add alice-zip                       # std (default): + zlib wrappers
cargo add alice-zip --features fft,parallel,lzma   # rustfft analysis, rayon textures, LZMA residual containers
cargo add alice-zip --no-default-features     # no_std + alloc (libm float math)

Rust crate

use alice_zip::prelude::*;

// Compression primitives (no_std)
let tokens = lz77_encode(b"abcabcabcabc", 256, 32);
assert_eq!(lz77_decode(&tokens)?, b"abcabcabcabc");
assert_eq!(shannon_entropy(&(0..=255u8).collect::<Vec<_>>()), 8.0);

// Generator laws: fit a series, keep the coefficients, regenerate
let series: Vec<f32> = (0..64).map(|i| 3.0 + 2.0 * i as f32).collect();
let (coeffs, degree, err) = fit_polynomial(&series, 4, 1e-6).unwrap();
assert_eq!((degree, generate_polynomial(64, &coeffs) == series), (1, true));

let (bins, dc) = analyze_signal(&generate_sine_wave(32, 1.0, 2.0, 0.0, 0.5), 4, 0.99);
let regenerated = generate_from_coefficients(32, &bins, dc);
# Ok::<(), alice_zip::ZipError>(())

Two persisted coefficient conventions coexist under explicit names (they are different laws and are never silently interchangeable): fit_polynomial / generate_polynomial (x = 0..n-1, ascending — alice-db segments) and fit_polynomial_unit / generate_polynomial_unit (x ∈ [0, 1], descending — the .alice container). Every law is checked against a closed-form answer in tests/analytic_oracle.rs and fuzzed (7 targets, including naive-DFT ↔ FFT parity). MSRV 1.87.

Quick Start

Command Line

# Compress
alice-zip compress data.bin -o data.alice

# Decompress
alice-zip decompress data.alice -o restored.bin

# Show file info
alice-zip info data.alice

Python API

from alice_zip import ALICEZip
import numpy as np

zipper = ALICEZip()

# Compress sine wave data
data = np.sin(np.linspace(0, 100*np.pi, 100000)).astype(np.float32)
compressed = zipper.compress(data)

print(f"Original: {data.nbytes:,} bytes")
print(f"Compressed: {len(compressed):,} bytes")
print(f"Ratio: {data.nbytes / len(compressed):.1f}x")

# Decompress
restored = zipper.decompress(compressed)

How It Works

Traditional compression finds patterns in bytes. ALICE finds patterns in mathematics.

Original Data = Generated(parameters) + Residual

Where:
  - Generated()  = Mathematical function (polynomial, sine wave, etc.)
  - parameters   = Tiny description (~100 bytes)
  - Residual     = Compressed difference (often near-zero)

Example

Input:  Sine wave, 100,000 samples (400 KB)
        ↓
Analysis: Detected as "Sine wave, freq=50Hz, amp=1.0"
        ↓
Output: Parameters only (~280 bytes)
        ↓
Result: 400 KB → 280 bytes = 1400x compression

Laws with evidence (law module)

law::SignalLaw keeps a fitted polynomial together with the evidence it came from, the measured residual, the x range the evidence covers, its provenance and reference values it must reproduce. It is recomputed at new conditions but never extrapolated, and new evidence is judged rather than appended:

use alice_zip::law::{IngestPolicy, Provenance, SignalLaw, Verdict};

let pts: Vec<(f64, f64)> = (0..5).map(|i| (i as f64, 1.0 + 2.0 * i as f64)).collect();
let law = SignalLaw::fit_polynomial(&pts, 1, Provenance::new("bench run 1", "least squares"))?;
assert!((law.evaluate(2.5)? - 6.0).abs() < 1e-12); // a condition that was not measured
assert!(law.evaluate(9.0).is_err());               // outside the measured range

let policy = IngestPolicy { abs_tolerance: 0.01, break_factor: 4.0 };
assert!(matches!(law.ingest(&[(0.5, 2.0), (3.5, 8.0)], &policy), Verdict::Supports { .. }));
# Ok::<(), alice_zip::law::LawError>(())

The rules and their order are documented on the module and pinned by tests/analytic_law.rs.

Benchmarks

Ideal Use Cases

• Scientific Data: Simulation outputs, sensor readings, waveforms
• Game Assets: Procedural textures, terrain heightmaps
• IoT/Edge: Sensor logs on bandwidth-constrained devices
• Time Series: Telemetry, monitoring logs with patterns

When NOT to Use

Building from Source

git clone https://github.com/ext-sakamoro/ALICE-Zip
cd ALICE-Zip

# Install Python package
pip install -e .

# Run tests
pytest tests/ -v

# Rust core crate — every CI gate, locally (scripts/preflight.sh --quick ≈ 3 min)
scripts/preflight.sh

# CLI / FFI crate
cargo build --manifest-path libalice/Cargo.toml --release

Related Projects

The same idea applied to other domains — this crate stores the recipe for a signal; these store the recipe for a shape, a physical state, or a frame.

All projects share the core philosophy: encode the generation process, not the data itself.

License

• Rust core crate alice-zip (/): MIT OR Apache-2.0, at your option
• CLI / FFI crate (libalice/), Python package (alice_zip/), bindings: MIT
• libalice-enterprise/: separate, unpublished proprietary crate (see its own LICENSE)

Author

Created by Moroya Sakamoto

"The best compression is not to store data, but to store the recipe for generating it."