Trait Tokenizer 

pub trait Tokenizer: Send + Sync {
    // Required methods
    fn name(&self) -> &'static str;
    fn split(&self, words: Vec<Word>, seg: &Segment) -> Vec<Word>;
}

Required Methods

fn name(&self) -> &'static str

fn split(&self, words: Vec<Word>, seg: &Segment) -> Vec<Word>

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors