Enum Token

pub enum Token {
    Keyword(String),
    True,
    False,
    String(String),
    FormatString(String),
    Int8(i8),
    Int16(i16),
    Int32(i32),
    Int64(i64),
    Int(i64),
    Float32(f32),
    Float64(f64),
    Float(f64),
    Identifier(String),
    BraceLeft,
    BraceRight,
    BracketLeft,
    BracketRight,
    ParenLeft,
    ParenRight,
    Dot,
    Colon,
    ScopeSymbol,
    Assign,
    Comma,
    Plus,
    Minus,
    Star,
    Slash,
    Mod,
    Greater,
    Less,
    LessEqual,
    GreaterEqual,
    Equal,
    NotEqual,
    Arrow,
    Not,
    And,
    Or,
    Vertical,
    Annotation,
    BitAnd,
    Boolean(bool),
    Eof,
}

词法单元定义，使用 logos 库进行词法分析

Variants

Keyword(String)

True

False

String(String)

FormatString(String)

Int8(i8)

Int16(i16)

Int32(i32)

Int64(i64)

Int(i64)

Float32(f32)

Float64(f64)

Float(f64)

Identifier(String)

BraceLeft

BraceRight

BracketLeft

BracketRight

ParenLeft

ParenRight

Dot

Colon

ScopeSymbol

Assign

Comma

Plus

Minus

Star

Slash

Mod

Greater

Less

LessEqual

GreaterEqual

Equal

NotEqual

Arrow

Not

And

Or

Vertical

Annotation

BitAnd

Boolean(bool)

Eof

Implementations

impl Token

pub fn from_keyword_or_identifier(s: String) -> Self

将关键字字符串转换为对应的 Token

pub fn token_id(&self) -> i8

获取 token 的唯一标识符

pub fn is_colon(&self) -> bool

检查是否为冒号

pub fn is_parenthesis_left(&self) -> bool

检查是否为左花括号

pub fn is_keyword(&self, keyword: &str) -> bool

检查是否为指定的关键字

pub fn is_assign(&self) -> bool

检查是否为赋值操作符

pub fn get_identifier_value(&self) -> &str

获取标识符的值

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&Token> for TokenType

fn from(token: &Token) -> Self

Converts to this type from the input type.

impl<'s> Logos<'s> for Token

type Error = ()

Error type returned by the lexer. This can be set using #[logos(error = MyError)]. Defaults to () if not set.

type Extras = ()

Associated type Extras for the particular lexer. This can be set using #[logos(extras = MyExtras)] and accessed inside callbacks.

type Source = str

Source type this token can be lexed from. This will default to str, unless one of the defined patterns explicitly uses non-unicode byte values or byte slices, in which case that implementation will use [u8].

fn lex(lex: &mut Lexer<'s, Self>)

The heart of Logos. Called by the Lexer. The implementation for this function is generated by the logos-derive crate.

fn lexer(source: &'source Self::Source) -> Lexer<'source, Self>

where Self::Extras: Default,

Create a new instance of a Lexer that will produce tokens implementing this Logos.

fn lexer_with_extras( source: &'source Self::Source, extras: Self::Extras, ) -> Lexer<'source, Self>

Create a new instance of a Lexer with the provided Extras that will produce tokens implementing this Logos.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Token