Crate whatlang

Whatlang is a Rust library to detect(regonize) natural languages. Apart from it, the library also recognizes scripts (writing system). Every language and script are represented by determined list of enums.

Examples

Using detect function:

use whatlang::{detect, Lang, Script};

let text = "Ĉu vi ne volas eklerni Esperanton? Bonvolu! Estas unu de la plej bonaj aferoj!";
let info = detect(text).unwrap();
assert_eq!(info.lang(), Lang::Epo);
assert_eq!(info.script(), Script::Latin);

// Confidence is in the range from 0 to 1.
assert_eq!(info.confidence(), 1.0);
assert!(info.is_reliable());

Using Detector with specified denylist or allowlist:

use whatlang::{Detector, Lang};

let allowlist = vec![Lang::Eng, Lang::Rus];

// You can also create detector using with_denylist function
let detector = Detector::with_allowlist(allowlist);
let lang = detector.detect_lang("There is no reason not to learn Esperanto.");
assert_eq!(lang, Some(Lang::Eng));

Features

Feature	Description
enum-map	Lang and Script implement Enum trait from enum-map
arbitrary	Support Arbitrary
serde	Implements Serialize and Deserialize for Lang and Script
dev	Enables whatlang::dev module which provides some internal API. It exists for profiling purposes and normal users are discouraged to to rely on this API.

Structs

• Detector
  Configurable structure that holds detection options and provides functions to detect language and script.
• Info
  Represents a full outcome of language detection.

Enums

• Lang
  Represents a language following ISO 639-3 standard.
• Script
  Represents a writing system (Latin, Cyrillic, Arabic, etc).

Functions

• detect
  Detect a language and a script by a given text.
• detect_lang
  Detect only a language by a given text.
• detect_script
  Detect only a script by a given text. Works much faster than a complete detection with detect.