pub trait Language {
    fn name(&self) -> String;
    fn code(&self) -> String;
    fn tokenize(&self, text: &str) -> Vec<String>Notable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
    A: Allocator
; fn make_pipeline(&self) -> Pipeline; }

Required Methods

The name of the language in English

The ISO 639-1 language code of the language

Separates the input text into individual tokens. In most languages a token is a word, separated by whitespace.

Returns the Pipeline to process the tokens with

Implementors