Struct FtsToken 

pub struct FtsToken {
    pub text: String,
    pub position: usize,
    pub kind: TokenKind,
    pub is_stop: bool,
    pub synonyms: Vec<String>,
    pub trigrams: Vec<String>,
}

A token produced by the FTS pipeline, ready for lexeme indexing.

Fields

text: String

The token text (owned; may be normalised).

position: usize

Ordinal position in the token sequence (0-based, gaps for whitespace).

kind: TokenKind

Script / category of the original token.

is_stop: bool

true if this token matches the stopword list.

synonyms: Vec<String>

Synonym expansions (empty if none configured or no match).

trigrams: Vec<String>

Character trigrams — populated only for TokenKind::Unknown tokens.

Trait Implementations

impl Clone for FtsToken

fn clone(&self) -> FtsToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FtsToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for FtsToken

fn eq(&self, other: &FtsToken) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for FtsToken

impl StructuralPartialEq for FtsToken