Struct TokenSet 

pub struct TokenSet {
    pub keywords: Vec<String>,
    pub literals: Vec<String>,
}

Token bag produced by Stage 1.

Fields

keywords: Vec<String>

Lowercase keyword tokens (regex [A-Za-z][A-Za-z0-9_]+, length ≥ 2). Used by Stage 2 to probe SchemaVocabulary.

literals: Vec<String>

Literal-id tokens kept in their original case so Stage 4 can run case-sensitive equality / substring filters. Matches [A-Z0-9-]{3,} containing at least one digit OR [0-9]{6,}.

Implementations

impl TokenSet

pub fn is_empty(&self) -> bool

Trait Implementations

impl Clone for TokenSet

fn clone(&self) -> TokenSet

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenSet

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenSet

fn default() -> TokenSet

Returns the “default value” for a type.

impl PartialEq for TokenSet

fn eq(&self, other: &TokenSet) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for TokenSet

impl StructuralPartialEq for TokenSet