Skip to main content

entropy_ratio

Function entropy_ratio 

Source
pub fn entropy_ratio(entropy_bits: f64, token_count: usize) -> f64
Expand description

Entropy as a share of the largest value a token sequence of this length could have.

Absolute entropy grows simply because there are more positions to fill. Dividing by log2(token_count) makes the suppression floor describe diversity rather than clone length. Empty and one-token sequences carry no diversity evidence and return 0.0.