ReadSight (Rust) — Multilingual Readability Engine

readsight measures text readability across 86 languages using 17 readability formulas, plus a syllable counter and word hyphenator based on the Frank M. Liang (TeX) hyphenation algorithm. It has no network access and no heavy dependencies — all language data and hyphenation patterns are embedded in the crate.

This is a byte-accurate Rust port of the canonical PHP library and its Python port:

PHP (canonical): https://github.com/MADEVAL/ReadSight
Python port: https://github.com/MADEVAL/ReadSightPy

Output parity with the reference implementations is verified with golden vectors generated from the PHP library (see tests/golden).

Installation

[dependencies]
readsight = "1.0"

Quick start

use readsight::ReadSight;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rs = ReadSight::new("en-us")?;

    // Syllables & hyphenation
    assert_eq!(rs.syllable_count("banana"), 3);
    assert_eq!(rs.split_word("hyphenation"), vec!["hy", "phen", "ation"]);

    // A readability score
    let text = "The cat sat on the mat. It was a sunny day.";
    let result = rs.flesch_reading_ease(text)?;
    println!(
        "{}: {} ({})",
        result.formula_name, result.score, result.interpretation
    );

    // Every formula supported for this language
    for formula in rs.supported_formulas() {
        let r = rs.score(&formula, text)?;
        println!("{:<28} {:>6}  {}", r.formula_name, r.score, r.interpretation);
    }
    Ok(())
}

Run the bundled demo:

cargo run --example demo
cargo run --example multilingual

Features

86 languages with language-specific letter/word/sentence tokenizers.
17 readability formulas (see below), each with the reference coefficients.
TeX (Liang) hyphenation via the bundled hyph-utf8 pattern files.
Three syllable modes per language — tex, heuristic (with vowelMode cluster/individual for Slavic languages), and composite.
Embedded data by default; optional [Config::from_dirs] to read from the filesystem instead.
Zero unsafe, #![forbid(unsafe_code)].

Readability formulas

Universal (all languages): Gunning Fog, SMOG, Coleman-Liau, Automated Readability Index (ARI), LIX.

Language-specific: Flesch Reading Ease, Flesch-Kincaid Grade Level, Wiener Sachtextformel (4 variants), Gulpease, Fernández-Huerta, Szigriszt-Pazos, Gutiérrez-Polini, Crawford, FOG-PL, Dale-Chall, Spache, OSMAN.

use readsight::ReadSight;

let rs = ReadSight::new("de-1996").unwrap();
// Wiener Sachtextformel supports 4 variants (1..=4)
let r = rs.wiener_sachtextformel("Ein einfacher deutscher Satz. Und noch einer.", 1).unwrap();
assert_eq!(r.formula_name, "wiener_sachtextformel_1");

API overview

ReadSight (aliased as Engine) is the entry point:

Text / syllable: split_word, split_syllables, syllable_count, word_count, sentence_count, letter_count, total_syllables, average_syllables_per_word, average_words_per_sentence, polysyllable_count, words_with_more_than_n_syllables, histogram_syllables, analyze, add_hyphenations.
Formulas: score(name, text) plus one convenience method per formula (flesch_reading_ease, gunning_fog, smog_index, coleman_liau, automated_readability_index, lix, gulpease, fernandez_huerta, szigriszt_pazos, gutierrez_polini, crawford, fog_pl, dale_chall, spache, osman, wiener_sachtextformel).
Static: ReadSight::supported_languages(config).

All formula methods return Result<FormulaResult, Error> — analyze (and hence every formula) returns [Error::EmptyText] for empty input.

Supported languages

ReadSight::supported_languages(None) returns all 86 codes (sorted). They mirror the JSON files under data/languages exactly.

Data source

By default all data is embedded at compile time (via include_dir!). To load from the filesystem instead:

use readsight::{Config, ReadSight};

let config = Config::from_dirs("data/patterns", "data/languages");
let rs = ReadSight::with_config("en-us", config).unwrap();

Testing

cargo test                 # unit + golden parity + multilingual smoke tests
cargo clippy --all-targets -- -D warnings
cargo fmt --check

The suite includes:

Golden parity against the PHP reference: analyze + all 17 formulas over every one of the 86 languages, plus per-word syllable vectors.
Ported unit tests from the PHP/Python suites (hyphenation, syllables, text splitting, formulas, grade-level interpretation).
Full-language smoke test that builds every language and runs every supported formula.

License

MIT — see LICENSE. Readability data and hyphenation patterns originate from the canonical PHP project and the hyph-utf8 package.

readsight 1.0.1