Function mako::tokenization::load_wordpiece_tokenizer [−][src]
pub fn load_wordpiece_tokenizer() -> Tokenizer
Expand description
Loads the wordpiece tokenizer
pub fn load_wordpiece_tokenizer() -> Tokenizer
Loads the wordpiece tokenizer