v0.3.0 — Now available! All core features of the infomeasure Python package have been reimplemented in Rust. Try it out with our Rust Guide and report issues or suggestions. Find the benchmark and interactive Rust vs Python performance comparison here.

infomeasure-rs

High-performance Rust library for information-theoretic measures with multiple estimation approaches.

What This Does

infomeasure-rs computes entropy, mutual information, and transfer entropy from data using four different estimation strategies:

Discrete: For categorical data with 11+ bias-corrected estimators
Kernel: For continuous data with optional GPU acceleration
Ordinal: For time series using permutation patterns
Exponential Family: For high-dimensional data using k-NN

Installation

[dependencies]
infomeasure = "0.3.0"

Quick Start

use infomeasure::estimators::entropy::Entropy;
use infomeasure::estimators::traits::GlobalValue;
use ndarray::array;

// Discrete entropy
let data = array![1, 2, 1, 3, 2, 1];
let entropy = Entropy::new_discrete(data).global_value();
println!("Entropy: {}", entropy);

// Kernel entropy for continuous data
let continuous = array![[1.0, 1.5], [2.0, 3.0], [4.0, 5.0]];
let kernel_entropy = Entropy::nd_kernel::<2>(continuous, 1.0).global_value();
println!("Kernel entropy: {}", kernel_entropy);

Optional Features

gpu: Enable GPU-accelerated kernel density estimation. Useful for large datasets with continuous variables (kernel approach) and batch processing.
fast_exp: Use fast exponential approximations for improved performance in the exponential family (k-NN) estimators.

[dependencies]
infomeasure = { version = "0.3.0", features = ["gpu"] }

[dependencies]
infomeasure = { version = "0.3.0", features = ["fast_exp"] }

Feature Status

All core measures from the Python package are implemented across all four estimation approaches:

Measure	Discrete	Kernel	Ordinal	k-NN (Exp. Family)
Entropy $H(X)$	✅	✅	✅	✅
Joint Entropy $H(X,Y)$	✅	✅	✅	✅
Conditional Entropy $H(X\|Y)$	✅	✅	✅	✅
Cross-Entropy $H_Q(P)$	✅[^1]	✅	✅	✅
KLD $D_{KL}(P\|Q)$	⚠️[^2]	⚠️	⚠️	⚠️
JSD $JSD(P\|Q)$	❌	❌	❌	❌
MI $I(X;Y)$	✅	✅	✅	✅
CMI $I(X;Y\|Z)$	✅	✅	✅	✅
TE $T_{X \to Y}$	✅	✅	✅	✅
CTE $T_{X \to Y\|Z}$	✅	✅	✅	✅

✅ = Implemented | ⚠️ = Via trait (see docs) | ❌ = Planned

[^1]: Discrete cross-entropy is available for MLE, Miller-Madow, and Bayesian estimators only. [^2]: KLD is available via the CrossEntropy trait.

Documentation

Rust Guide — Comprehensive walkthrough of all estimators with mathematical background
API Reference — Full API documentation with examples
Examples — Usage examples

Rust vs Python

This crate reimplements the infomeasure Python package in Rust for users who need:

Rust (`infomeasure-rs`)	Python (`infomeasure`)
Compile-time type safety via Rust's type system	Runtime string-based approach selection
Up to ~40x faster execution (detailled benchmarks to follow)	Flexible, scriptable interface
GPU acceleration for kernel estimators	GPU support via numba
Compile-time optimized estimator code	Runtime dispatch
`[dependencies]` in `Cargo.toml`	`pip install infomeasure`

Choose Rust if you need maximum performance for production or large-scale analysis. Choose Python if you need rapid prototyping, interactive analysis, or academic flexibility.

Full head-to-head benchmarks are being prepared and will be published soon. See the Rust Guide for a detailed comparison of the architecture.

Python Compatibility

This crate maintains functional compatibility with the infomeasure Python package, implementing the same measures and estimation approaches, while providing 8-40x performance improvements.

Repository Structure

src/ - Main source code
- estimators/ - Estimation techniques implementations
  - approaches/ - Specific implementations (discrete, kernel, ...)
  - traits/ - Shared interfaces for estimators
benches/ - Performance benchmarks using Criterion
tests/ - Unit and integration tests
examples/ - Example usage and demonstrations

Development Setup

Prerequisites

Rust 1.70+ (for building)
uv Python package manager (for validation tests)

Python Environment Setup

The validation tests require a Python environment with infomeasure package. Set it up once before running tests:

# Create virtual environment in validation crate directory
cd tests/validation_crate
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv pip install -r requirements.txt

Running Tests

# Run all tests (includes Python validation)
cargo test

# Run only Rust unit tests (skip Python validation)
cargo test --lib

Testing and Validation

The project includes a validation crate that compares results with Python implementation to ensure compatibility and correctness.

Benchmarks

Performance benchmarks are available for different estimation methods:

cargo bench

Contributing

Contributions welcome! Please feel free to submit a Pull Request.

License

MIT OR Apache-2.0

infomeasure 0.3.0