Enum Symbol

pub enum Symbol {
    Word(String),
    Alpha(char),
    Digit(usize),
    Underscore,
    Dash,
    Assignment,
    Plus,
    LeftBrace,
    RightBrace,
    LeftBracket,
    RightBracket,
    LeftParenthesis,
    RightParenthesis,
    Colon,
    Semicolon,
    Join,
    Directive,
    Escape,
    Slash,
    Comma,
    Decimal,
    DoubleQuote,
    SingleQuote,
    LeftAngle,
    RightAngle,
    Unicode(String),
    Newline,
    Whitespace(Whitespace),
    Comment(String),
    Eoi,
    Void,
}

The symbol of a Token

Variants

Word(String)

A word is a contiguous sequence of letters, digits, and underscores. A word will never start with a digit.

Alpha(char)

A single alphanumeric character

Digit(usize)

Parsed digits will always be a single digit, but generated digits may have multiple digits.

Underscore

_

Dash

Assignment

=

Plus

LeftBrace

{

RightBrace

}

LeftBracket

[

RightBracket

]

LeftParenthesis

(

RightParenthesis

)

Colon

:

Semicolon

;

Join

Directive

Escape

\

Slash

/

Comma

,

Decimal

.

DoubleQuote

“”

SingleQuote

’

LeftAngle

<

RightAngle

Unicode(String)

A unicode character

Newline

A newline \n

Whitespace(Whitespace)

A Whitespace character

Comment(String)

A comment Comments are not parsed, but are kept in the token stream so that they can be outputted in the same format as the input.

Comments have two forms: Single line comments start with // and end with a newline. Multi line comments start with /* and end with */.

Eoi

End of input

Void

Void token, not outputted

Implementations

impl Symbol

pub fn from_word<S: Into<String>>(word: S) -> Self

Create a new Word symbol

pub const fn is_whitespace(&self) -> bool

Check if a symbol is Whitespace or Comment

pub const fn is_newline(&self) -> bool

Check if a symbol is Newline

pub fn output(&self) -> String

Get the output of a symbol

Trait Implementations

impl Clone for Symbol

fn clone(&self) -> Symbol

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Symbol

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Symbol

fn eq(&self, other: &Symbol) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl ToString for Symbol

fn to_string(&self) -> String

Converts the given value to a String.

impl Eq for Symbol

impl StructuralPartialEq for Symbol