Trait Lexicon 

pub trait Lexicon:
    Debug
    + Clone
    + Copy
    + PartialEq
    + Eq
    + Hash
    + 'static {
    type Bit: Unsigned + Integer + BitAnd<Output = Self::Bit> + BitOr<Output = Self::Bit> + Not<Output = Self::Bit> + Copy;
    type Lexer<'s>: Lexer<'s, L = Self>;
    type Map<T: Default + Clone>: Default + Clone + Borrow<[T]> + BorrowMut<[T]>;

    // Required methods
    fn id(&self) -> usize;
    fn from_id_unchecked(id: usize) -> Self;
    fn to_bit(&self) -> Self::Bit;
    fn first() -> Self;
    fn next(&self) -> Option<Self>;
    fn should_extract(&self) -> bool;
    fn lexer(source: &str) -> Result<Self::Lexer<'_>, GrammarError>;
}

Trait for defining the token types of a grammar

See module-level documentation for more information

Required Associated Types

type Bit: Unsigned + Integer + BitAnd<Output = Self::Bit> + BitOr<Output = Self::Bit> + Not<Output = Self::Bit> + Copy

Bit flag representation of the token types

When derived, One of u8, u16, u32, u64, or u128 is chosen based on the number of variants in the enum

type Lexer<'s>: Lexer<'s, L = Self>

Lexer associated with this TokenType

type Map<T: Default + Clone>: Default + Clone + Borrow<[T]> + BorrowMut<[T]>

Map type used for storing token type mappings in tables for syntax analysis

Required Methods

fn id(&self) -> usize

Get the id of this token type (ordinal)

fn from_id_unchecked(id: usize) -> Self

fn to_bit(&self) -> Self::Bit

Convert to numeric representation for use in bit set

fn first() -> Self

Get the first type. Used to iterate over all types

fn next(&self) -> Option<Self>

Get the next type. Used to iterate over all types

fn should_extract(&self) -> bool

Whether this token should be excluded from AST, but still has value.

One example is comments

fn lexer(source: &str) -> Result<Self::Lexer<'_>, GrammarError>

Create a lexer for parsing this token type

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors