Function tokenizers::tokenizer::normalizer::char_to_bytes
source · pub fn char_to_bytes(s: &str, range: Range<usize>) -> Option<Range<usize>>Expand description
Convert the given range from char to bytes
pub fn char_to_bytes(s: &str, range: Range<usize>) -> Option<Range<usize>>Convert the given range from char to bytes