Enum NgramType 

pub enum NgramType {
    Window,
    Edge,
}

The style of ngram extraction to use.

The same style of ngram extraction is always used at index time and at query time.

Each ngram type uses the ngram size configuration differently.

All ngram styles used Unicode codepoints as the definition of a character. For example, a 3-gram might contain up to 4 bytes, if it contains 3 Unicode codepoints that each require 4 UTF-8 code units.

Variants

Window

A windowing ngram.

This is the tradition style of ngram, where sliding window of size N is moved across the entire content to be index. For example, the 3-grams for the string homer are hom, ome and mer.

Edge

An edge ngram.

This style of ngram produces ever longer ngrams, where each ngram is anchored to the start of a word. Words are determined simply by splitting whitespace.

For example, the edge ngrams of homer simpson, where the max ngram size is 5, would be: hom, home, homer, sim, simp, simps. Generally, for this ngram type, one wants to use a large maximum ngram size. Perhaps somewhere close to the maximum number of ngrams in any word in the corpus.

Note that there is no way to set the minimum ngram size (which is 3).

Implementations

impl NgramType

pub fn possible_names() -> &'static [&'static str]

Return all possible ngram types.

pub fn as_str(&self) -> &'static str

Return a string representation of this type.

Trait Implementations

impl Clone for NgramType

fn clone(&self) -> NgramType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for NgramType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for NgramType

fn default() -> NgramType

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for NgramType

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for NgramType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromStr for NgramType

type Err = Error

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<NgramType>

Parses a string s to return a value of this type.

impl Hash for NgramType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for NgramType

fn eq(&self, other: &NgramType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for NgramType

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for NgramType

impl Eq for NgramType

impl StructuralPartialEq for NgramType