Module tokenizers::pre_tokenizers::metaspace
source · [−]Structs
Replaces all the whitespaces by the provided meta character and then splits on this character
Replaces all the whitespaces by the provided meta character and then splits on this character