Enum cang_jie::options::TokenizerOption

pub enum TokenizerOption {
    All,
    Default {
        hmm: bool,
    },
    ForSearch {
        hmm: bool,
    },
    Unicode,
}

Tokenizer Option

Variants

All

Cut the input text, return all possible words

Default

Cut the input text

Fields of Default

hmm: bool

hmm: enable HMM or not

ForSearch

Cut the input text in search mode

Fields of ForSearch

hmm: bool

hmm: enable HMM or not

Unicode

Cut the input text into UTF-8 characters

Trait Implementations

impl Debug for TokenizerOption

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl Clone for TokenizerOption

fn clone(&self) -> TokenizerOption

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.