Function mako::tokenization::batch_untokenize_wordpiece [−][src]
pub fn batch_untokenize_wordpiece(tokens: Vec<Vec<String>>) -> Vec<String>
Expand description
Untokenize wordpiece tokens
pub fn batch_untokenize_wordpiece(tokens: Vec<Vec<String>>) -> Vec<String>
Untokenize wordpiece tokens