1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
// SPDX-FileCopyrightText: 2025-2026 Carlson Büth <code@cbueth.de>
//
// SPDX-License-Identifier: MIT OR Apache-2.0
//! High-performance Rust library for information-theoretic measures including entropy,
//! mutual information, and transfer entropy with multiple estimation approaches.
//!
//! # Quick Start
//!
//! ```rust
//! use infomeasure::estimators::entropy::Entropy;
//! use infomeasure::estimators::traits::GlobalValue;
//! use ndarray::array;
//!
//! // Discrete entropy
//! let data = array![1, 2, 1, 3, 2, 1];
//! let entropy = Entropy::new_discrete(data).global_value();
//!
//! // Kernel entropy for continuous data
//! let continuous = array![[1.0, 1.5], [2.0, 3.0], [4.0, 5.0]];
//! let kernel_entropy = Entropy::nd_kernel::<2>(continuous, 1.0).global_value();
//! ```
//!
//! # Features
//!
//! | Measure | Discrete | Kernel | Ordinal | Exp. Family | Notes |
//! |---------|----------|--------|---------|-------------|-------|
//! | Entropy | ✅ | ✅ | ✅ | ✅ | All variants |
//! | Joint Entropy | ✅ | ✅ | ✅ | ✅ | Via multi-variable estimators |
//! | Conditional Entropy | ✅ | ✅ | ✅ | ✅ | |
//! | Cross-Entropy | ✅[^1] | ✅ | ✅ | ✅ | All approaches |
//! | KLD | ⚠️ | ⚠️ | ⚠️ | ⚠️ | Via cross-entropy |
//! | JSD | ❌ | ❌ | ❌ | ❌ | Planned |
//! | MI | ✅ | ✅ | ✅ | ✅ | All variants |
//! | CMI | ✅ | ✅ | ✅ | ✅ | Conditional MI |
//! | TE | ✅ | ✅ | ✅ | ✅ | Transfer Entropy |
//! | CTE | ✅ | ✅ | ✅ | ✅ | Conditional TE |
//!
//! ✅ = Implemented | ⚠️ = Available via trait | ❌ = Not implemented
//!
//! [^1]: For discrete estimators, cross-entropy is only available for MLE, Miller-Madow, and Bayesian estimators. NSB, Chao-Shen, and Chao-Wang-Jost do not support cross-entropy due to theoretical inconsistencies in applying bias corrections to cross-entropy.
//!
//! # Estimation Approaches
//!
//! ## Discrete Estimation
//! Histogram-based probability estimation for categorical or binned data.
//! Supports 11+ bias-corrected estimators (MLE, Miller-Madow, NSB, etc.).
//!
//! ## Kernel Estimation
//! Non-parametric density estimation for continuous data using Box and Gaussian kernels.
//! Optional GPU acceleration for large datasets.
//!
//! ## Ordinal Pattern Analysis
//! Permutation pattern encoding for time series data, robust to amplitude variations.
//!
//! ## Exponential Family (k-NN)
//! Distance-based estimation using k-nearest neighbours for differential entropy.
//! Supports Rényi and Tsallis generalized entropies.
//!
//! # Architecture
//!
//! The library follows a four-layer architecture:
//!
//! 1. **Public API Layer**: Factory types (`Entropy`, `MutualInformation`, `TransferEntropy`)
//! 2. **Estimation Approaches**: Four independent strategies for different data types
//! 3. **Core Infrastructure**: Shared traits and data structures
//! 4. **Performance Layer**: Optional GPU acceleration and mathematical optimisations
//!
//! # Feature Flags
//!
//! - `gpu`: Enable GPU acceleration for kernel estimators
//! - `fast_exp`: Use fast exponential approximations (trades accuracy for speed)
//!
//! # Python Compatibility
//!
//! This crate is designed to be a high-performance Rust backend for the
//! [infomeasure](https://github.com/cbueth/infomeasure) Python package, maintaining
//! API parity while providing significant performance improvements.
//!
//! # Guides
//!
//! - [Estimator Usage Guide](crate::guide::estimator_usage) - How to use this crate
//! - [Estimator Selection Guide](crate::guide::estimator_selection) - Choosing the right estimator
//!
//! For more details on the theory behind these measures, see the Python package documentation.