Skip to main content

token_ngrams

Function token_ngrams 

Source
pub fn token_ngrams<'a>(words: &'a [&'a str], n: usize) -> Vec<Vec<&'a str>>
Expand description

Generate token n-grams as vectors of &str.

This is a convenience for callers that want to apply their own hashing/keying scheme instead of joining tokens into a single string.