infomeasure 0.1.0-alpha.1

Information theory measures and entropy calculations for Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

# Kernel Estimator Dimensions

This document explains the relationship between the various const generic parameters used in the Kernel-based estimators (MI, CMI, TE, CTE).

To avoid relying on the unstable `generic_const_exprs` feature in Rust, we explicitly pass all required dimensions as const generics. These dimensions are mathematically dependent on each other.

## Transfer Entropy (TE)

$TE(X \to Y) = I(Y_{future}; X_{past} | Y_{past})$

The estimator `KernelTransferEntropy` uses:
- `SRC_HIST`, `DEST_HIST`: History lengths.
- `STEP_SIZE`: Delay between observations.
- `D_SOURCE`, `D_TARGET`: Dimensionality of individual samples in X and Y.
- `D_JOINT`: $D_{target} + (SRC\_HIST \times D_{source}) + (DEST\_HIST \times D_{target})$
- `D_XP_YP`: $(SRC\_HIST \times D_{source}) + (DEST\_HIST \times D_{target})$
- `D_YP`: $DEST\_HIST \times D_{target}$
- `D_YF_YP`: $D_{target} + (DEST\_HIST \times D_{target})$

### Dimension Relations for TE
- `D_JOINT` = `D_YF_YP` + `SRC_HIST` * `D_SOURCE`
- `D_XP_YP` = `D_YP` + `SRC_HIST` * `D_SOURCE`
- `D_YF_YP` = `D_TARGET` + `D_YP`
- `D_YP` = `DEST_HIST` * `D_TARGET`

## Conditional Transfer Entropy (CTE)

$CTE(X \to Y | Z) = I(Y_{future}; X_{past} | Y_{past}, Z_{past})$

The estimator `KernelConditionalTransferEntropy` uses:
- `D_JOINT`: $D_{target} + (SRC\_HIST \times D_{source}) + (DEST\_HIST \times D_{target}) + (COND\_HIST \times D_{cond})$
- `D_XP_YP_ZP`: $(SRC\_HIST \times D_{source}) + (DEST\_HIST \times D_{target}) + (COND\_HIST \times D_{cond})$
- `D_YP_ZP`: $(DEST\_HIST \times D_{target}) + (COND\_HIST \times D_{cond})$
- `D_YF_YP_ZP`: $D_{target} + (DEST\_HIST \times D_{target}) + (COND\_HIST \times D_{cond})$

### Dimension Relations for CTE
- `D_JOINT` = `D_YF_YP_ZP` + `SRC_HIST` * `D_SOURCE`
- `D_XP_YP_ZP` = `D_YP_ZP` + `SRC_HIST` * `D_SOURCE`
- `D_YF_YP_ZP` = `D_TARGET` + `D_YP_ZP`
- `D_YP_ZP` = (`DEST_HIST` * `D_TARGET`) + (`COND_HIST` * `D_COND`)

## Mutual Information (MI)

$I(X_1; X_2; \dots; X_n) = \log p(X_1, \dots, X_n) - \sum \log p(X_i)$

The estimators `KernelMutualInformation2` through `KernelMutualInformation6` support 2 to 6 random variables respectively.

- `D_JOINT`: $\sum_{i=1}^n D_i$
- `D1`, `D2`, ..., `Dn`: Dimensions of individual random variables.

## Conditional Mutual Information (CMI)

$I(X; Y | Z) = \log \frac{p(X,Y,Z)p(Z)}{p(X,Z)p(Y,Z)}$

- `D_JOINT`: $D_1 + D_2 + D_{cond}$
- `D1_COND`: $D_1 + D_{cond}$
- `D2_COND`: $D_2 + D_{cond}$

## Helper Macros

To simplify instantiation and automatically calculate these dimensions, use the following macros:
- `new_kernel_te!`
- `new_kernel_cte!`
- `new_kernel_mi!`
- `new_kernel_cmi!`