pub fn entropy(probs: &[f64]) -> f64
Shannon entropy of a probability distribution (in nats).
Tokens with zero probability contribute 0 to the sum (0 ยท ln 0 = 0).