Function mako::tokenization::untokenize_bpe [−][src]
pub fn untokenize_bpe(tokens: Vec<String>) -> String
Expand description
Untokenize BPE tokens
pub fn untokenize_bpe(tokens: Vec<String>) -> String
Untokenize BPE tokens