Enum TokenType 

pub enum TokenType {
    Division,
    RegexMatch,
    Substitution,
    Transliteration,
    QuoteRegex,
    StringLiteral,
    QuoteSingle,
    QuoteDouble,
    QuoteWords,
    QuoteCommand,
    InterpolatedString(Vec<StringPart>),
    HeredocStart,
    HeredocBody(Arc<str>),
    FormatBody(Arc<str>),
    Version(Arc<str>),
    Pod,
    DataMarker(Arc<str>),
    DataBody(Arc<str>),
    UnknownRest,
    Identifier(Arc<str>),
    Number(Arc<str>),
    Operator(Arc<str>),
    Keyword(Arc<str>),
    LeftParen,
    RightParen,
    LeftBracket,
    RightBracket,
    LeftBrace,
    RightBrace,
    Semicolon,
    Comma,
    Colon,
    Arrow,
    FatComma,
    Whitespace,
    Newline,
    Comment(Arc<str>),
    EOF,
    Error(Arc<str>),
}

Token types for Perl

Variants

Division

Division operator: /

RegexMatch

Regex match: m// or //

Substitution

Substitution: s///

Transliteration

Transliteration: tr/// or y///

QuoteRegex

Quote regex: qr//

StringLiteral

String literal: “string” or ‘string’

QuoteSingle

Single quote: q//

QuoteDouble

Double quote: qq//

QuoteWords

Quote words: qw//

QuoteCommand

Quote command: qx// or backticks

InterpolatedString(Vec<StringPart>)

String with interpolated parts

HeredocStart

Heredoc start: <<EOF or <<‘EOF’

HeredocBody(Arc<str>)

Heredoc body content

FormatBody(Arc<str>)

Format body content

Version(Arc<str>)

Version string: v5.32.0

Pod

POD documentation block

DataMarker(Arc<str>)

Data section marker: DATA or END

DataBody(Arc<str>)

Data section body content

UnknownRest

Unknown rest of input (used when budget exceeded)

Identifier(Arc<str>)

Identifier or variable name

Number(Arc<str>)

Numeric literal

Operator(Arc<str>)

Operator

Keyword(Arc<str>)

Keyword

LeftParen

Left parenthesis: (

RightParen

Right parenthesis: )

LeftBracket

Left bracket: [

RightBracket

Right bracket: ]

LeftBrace

Left brace: {

RightBrace

Right brace: }

Semicolon

Semicolon: ;

Comma

Comma: ,

Colon

Colon: :

Arrow

Arrow: ->

FatComma

Fat comma: =>

Whitespace

Whitespace (usually not returned)

Newline

Newline character

Comment(Arc<str>)

Comment text

EOF

End of file

Error(Arc<str>)

Error token for invalid input

Trait Implementations

impl Clone for TokenType

fn clone(&self) -> TokenType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenType

fn eq(&self, other: &TokenType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for TokenType