readsight 1.0.0

# ReadSight (Rust) — Multilingual Readability Engine

[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Languages](https://img.shields.io/badge/languages-86-9cf)](#supported-languages)
[![Formulas](https://img.shields.io/badge/formulas-17-orange)](#readability-formulas)

`readsight` measures **text readability across 86 languages** using **17 readability
formulas**, plus a syllable counter and word hyphenator based on the **Frank M. Liang
(TeX) hyphenation algorithm**. It has **no network access** and **no heavy
dependencies** — all language data and hyphenation patterns are embedded in the crate.

This is a **byte-accurate Rust port** of the canonical PHP library and its Python port:

- **PHP (canonical):** <https://github.com/MADEVAL/ReadSight>
- **Python port:** <https://github.com/MADEVAL/ReadSightPy>

Output parity with the reference implementations is verified with golden vectors
generated from the PHP library (see [`tests/golden`](tests/golden)).

## Installation

```toml
[dependencies]
readsight = "1.0"
```

## Quick start

```rust
use readsight::ReadSight;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let rs = ReadSight::new("en-us")?;

    // Syllables & hyphenation
    assert_eq!(rs.syllable_count("banana"), 3);
    assert_eq!(rs.split_word("hyphenation"), vec!["hy", "phen", "ation"]);

    // A readability score
    let text = "The cat sat on the mat. It was a sunny day.";
    let result = rs.flesch_reading_ease(text)?;
    println!(
        "{}: {} ({})",
        result.formula_name, result.score, result.interpretation
    );

    // Every formula supported for this language
    for formula in rs.supported_formulas() {
        let r = rs.score(&formula, text)?;
        println!("{:<28} {:>6}  {}", r.formula_name, r.score, r.interpretation);
    }
    Ok(())
}
```

Run the bundled demo:

```sh
cargo run --example demo
cargo run --example multilingual
```

## Features

- **86 languages** with language-specific letter/word/sentence tokenizers.
- **17 readability formulas** (see below), each with the reference coefficients.
- **TeX (Liang) hyphenation** via the bundled `hyph-utf8` pattern files.
- **Three syllable modes** per language — `tex`, `heuristic` (with `vowelMode`
  `cluster`/`individual` for Slavic languages), and `composite`.
- **Embedded data** by default; optional [`Config::from_dirs`] to read from the
  filesystem instead.
- **Zero `unsafe`**, `#![forbid(unsafe_code)]`.

## Readability formulas

Universal (all languages): **Gunning Fog**, **SMOG**, **Coleman-Liau**,
**Automated Readability Index (ARI)**, **LIX**.

Language-specific: **Flesch Reading Ease**, **Flesch-Kincaid Grade Level**,
**Wiener Sachtextformel** (4 variants), **Gulpease**, **Fernández-Huerta**,
**Szigriszt-Pazos**, **Gutiérrez-Polini**, **Crawford**, **FOG-PL**,
**Dale-Chall**, **Spache**, **OSMAN**.

```rust
use readsight::ReadSight;

let rs = ReadSight::new("de-1996").unwrap();
// Wiener Sachtextformel supports 4 variants (1..=4)
let r = rs.wiener_sachtextformel("Ein einfacher deutscher Satz. Und noch einer.", 1).unwrap();
assert_eq!(r.formula_name, "wiener_sachtextformel_1");
```

## API overview

`ReadSight` (aliased as `Engine`) is the entry point:

- Text / syllable: `split_word`, `split_syllables`, `syllable_count`,
  `word_count`, `sentence_count`, `letter_count`, `total_syllables`,
  `average_syllables_per_word`, `average_words_per_sentence`,
  `polysyllable_count`, `words_with_more_than_n_syllables`,
  `histogram_syllables`, `analyze`, `add_hyphenations`.
- Formulas: `score(name, text)` plus one convenience method per formula
  (`flesch_reading_ease`, `gunning_fog`, `smog_index`, `coleman_liau`,
  `automated_readability_index`, `lix`, `gulpease`, `fernandez_huerta`,
  `szigriszt_pazos`, `gutierrez_polini`, `crawford`, `fog_pl`, `dale_chall`,
  `spache`, `osman`, `wiener_sachtextformel`).
- Static: `ReadSight::supported_languages(config)`.

All formula methods return `Result<FormulaResult, Error>` — `analyze` (and hence
every formula) returns [`Error::EmptyText`] for empty input.

## Supported languages

`ReadSight::supported_languages(None)` returns all 86 codes (sorted). They mirror
the JSON files under [`data/languages`](data/languages) exactly.

## Data source

By default all data is embedded at compile time (via `include_dir!`). To load from
the filesystem instead:

```rust
use readsight::{Config, ReadSight};

let config = Config::from_dirs("data/patterns", "data/languages");
let rs = ReadSight::with_config("en-us", config).unwrap();
```

## Testing

```sh
cargo test                 # unit + golden parity + multilingual smoke tests
cargo clippy --all-targets -- -D warnings
cargo fmt --check
```

The suite includes:

- **Golden parity** against the PHP reference: `analyze` + all 17 formulas over
  every one of the 86 languages, plus per-word syllable vectors.
- **Ported unit tests** from the PHP/Python suites (hyphenation, syllables,
  text splitting, formulas, grade-level interpretation).
- **Full-language smoke test** that builds every language and runs every
  supported formula.

## License

MIT — see [LICENSE](LICENSE). Readability data and hyphenation patterns originate
from the canonical PHP project and the [`hyph-utf8`](https://ctan.org/pkg/hyph-utf8)
package.