1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
// SPDX-FileCopyrightText: 2025-2026 Carlson Büth <code@cbueth.de>
//
// SPDX-License-Identifier: MIT OR Apache-2.0
//! # Cross-Entropy $H_Q(P)$
//!
//! Cross-entropy measures the information required when using an encoding based on
//! distribution $Q$ to represent data that actually follows distribution $P$.
//!
//! ## Definition
//!
//! For discrete random variables $P$ and $Q$ over the same set $\mathcal{X}$:
//!
//! $$H_Q(P) = -\sum_{x \in \mathcal{X}} P(x) \log Q(x)$$
//!
//! For continuous variables, it is defined using the differential entropy form:
//!
//! $$H_Q(P) = -\int P(x) \log Q(x) \, dx$$
//!
//! ## Relationship to Other Measures
//!
//! - **Entropy**: $H(P) = H_P(P)$ (using P to encode itself)
//! - **KLD**: $D_{\mathrm{KL}}(P \parallel Q) = H_Q(P) - H(P)$
//!
//! Cross-entropy always satisfies $H_Q(P) \ge H(P)$, with equality if and only if $P = Q$.
//!
//! ## Implemented in This Crate
//!
//! Cross-entropy is available via the [`CrossEntropy`](crate::estimators::traits::CrossEntropy) trait.
//! It is supported by:
//!
//! - [`RenyiEntropy`](crate::estimators::approaches::expfam::renyi::RenyiEntropy)
//! - [`TsallisEntropy`](crate::estimators::approaches::expfam::tsallis::TsallisEntropy)
//! - [`OrdinalEntropy`](crate::estimators::approaches::ordinal::ordinal_estimator::OrdinalEntropy)
//!
//! ## Example
//!
//! ```rust
//! use infomeasure::estimators::approaches::RenyiEntropy;
//! use infomeasure::estimators::traits::CrossEntropy;
//! use ndarray::array;
//!
//! let p = RenyiEntropy::<1>::new_1d(array![0.1, 0.2, 0.3, 0.4], 2, 1.0, 0.0);
//! let q = RenyiEntropy::<1>::new_1d(array![0.15, 0.25, 0.35, 0.45], 2, 1.0, 0.0);
//!
//! let ce = p.cross_entropy(&q);
//! // Cross-entropy can be negative for continuous distributions
//! assert!(ce.is_finite());
//! ```
//!
//! ## See Also
//!
//! - [Entropy Guide](super::entropy) — Base entropy
//! - [KLD Guide](super::kld) — $D_{\mathrm{KL}}(P \parallel Q) = H_Q(P) - H(P)$
//! - [JSD Guide](super::jsd) — Symmetric divergence measure