Whatlang

Natural language detection in Rust.

Features

• Supports 70 languages
• 100% written in Rust
• No external dependencies
• Super fast
• Recognizes not only a language, but also a script (Latin, Cyrillic, etc)

Get started

The library is still in active development. Here is the short example how to use it:

Add to you Cargo.toml:

[dependencies]

whatlang = "*"

In you program:

extern crate whatlang;

use whatlang::{detect_lang, Lang, Script, Query};

fn main() {
    let text = "Guten Abend, meine Damen und Herren!";
    let query = Query::new(text);
    let result = detect_lang(query).unwrap();
    assert_eq!(result.lang, Lang::Deu);
    assert_eq!(result.lang.to_code(), "deu");
    assert_eq!(result.script, Script::Latin);
}

Blacklist

Your can blacklist undesired languages, passing a vector. In the example blow English and Spanish will be ignored:

let list = [Lang::Eng, Lang::Spa];
let query = Query::new(&text).blacklist(&list);

Whitelist

In similar way, you can whitelist specified languages. In this example, the library will recognize only Esperanto and Russian. Note, if it detects a script that is different from Latin(Esperanto) or Cyrillic(Russian), e.g. Greek, it will return None.

let list = [Lang::Epo, Lang::Rus];
let query = Query::new(&text).whitelist(&list);

Roadmap

• Support 100 most popular languages
• Allow to specify blacklist for Query
• Allow to specify whitelist for Query
• Support new API
• Write doc for public structures and functions
• Improve README example
• Tune performance
• Create demo application
• Provide some metrics about reliability in Result struct

Supported languages

Missing languages

The language that I did not find trigrams for:

• Fula

License

MIT

Acknowledgments

• Thanks Franc JS for trigrams dataset.

Contributors

• greyblake Potapov Sergey - creator, maintainer.
• Dr-Emann Zachary Dremann - optimization and improvements