Module stringmetrics::tokenizers
source · [−]Expand description
Tools for splitting strings into words
Tokenizers just provide handy methods to split a string into words, following certain rules.
Functions
Split by whitespace an remove all punctuation