Crate ngrams

Ngrams

Produce n-gram sequences from a sequence of tokens

Examples

use ngrams::Ngram;

let grams: Vec<_> = "foo".chars().ngrams(2).pad().collect();
assert_eq!(
    grams,
    vec![
          vec!['\u{2060}', 'f'],
          vec!['f', 'o'],
          vec!['o', 'o'],
          vec!['o', '\u{2060}']
    ]
);

use ngrams::Ngrams; // notice `Ngram` vs `Ngrams`

let iter = "one two three".split(' ');
let grams: Vec<_> = Ngrams::new(iter, 3).pad().collect();
assert_eq!(
    grams,
    vec![
          vec!["\u{2060}", "\u{2060}", "one"],
          vec!["\u{2060}", "one", "two"],
          vec!["one", "two", "three"],
          vec!["two", "three", "\u{2060}"],
          vec!["three", "\u{2060}", "\u{2060}"],
    ]
);

Structs

Ngrams

Main data type, implements the logic on splitting and grouping n-grams

Traits

Ngram

Iterator adaptor, allows you to call the method .ngrams(n) on your iterator, as long as the Item of the Iterator fits the trait bound

Pad

Implement this so ngrams knows how to pad the beginning and end of your input.