Skip to main content

Module scoring

Module scoring 

Source
Expand description

Surprisal-based line scoring for deterministic compression.

Each line receives an information density score based on:

  • Character-level entropy (Shannon)
  • Structural markers (paths, numbers, identifiers)
  • Repetition detection (overlap with previous lines)

Structs§

LineScore
Score for a single line — higher means more informative.

Functions§

score_lines
Scores all lines in the input text for information density.