Enum Token 

pub enum Token<'a, Position> {
    Comment(&'a str),
    HeadingHashes(usize),
    Newline,
    Brac,
    Ket,
    Bra,
    Ce,
    Paren,
    Thesis,
    Comma,
    Period,
    String(Cow<'a, String, str>),
    InvalidStringWithVerbatimCarriageReturn(&'a str),
    DataLiteral(DataLiteral<'a, Position>),
    InvalidDataLiteralWithVerbatimCarriageReturn(InvalidDataLiteral<'a, Position>),
    Decimal(&'a str),
    InvalidZeroPrefixedDecimal(&'a str),
    Integer(&'a str),
    InvalidZeroPrefixedInteger(&'a str),
    Colon,
    Identifier(Cow<'a, String, str>),
    InvalidIdentifierWithVerbatimCarriageReturn(&'a str),
    Error,
}

Variants

Comment(&'a str)

HeadingHashes(usize)

Newline

Brac

Ket

Bra

Ce

Paren

Thesis

Comma

Period

String(Cow<'a, String, str>)

InvalidStringWithVerbatimCarriageReturn(&'a str)

Unlike in Token::String, the quoted string is not unescaped in order to preserve the '\r' vs '\\r' distinction.

DataLiteral(DataLiteral<'a, Position>)

InvalidDataLiteralWithVerbatimCarriageReturn(InvalidDataLiteral<'a, Position>)

Unlike in Token::DataLiteral, the strings are not unescaped in order to preserve the '\r' vs '\\r' distinction.

Decimal(&'a str)

InvalidZeroPrefixedDecimal(&'a str)

Integer(&'a str)

InvalidZeroPrefixedInteger(&'a str)

Colon

Identifier(Cow<'a, String, str>)

InvalidIdentifierWithVerbatimCarriageReturn(&'a str)

Unlike in Token::Identifier, the quoted string is not unescaped in order to preserve the '\r' vs '\\r' distinction.

Error

Trait Implementations

impl<'a, Position: Clone> Clone for Token<'a, Position>

fn clone(&self) -> Token<'a, Position>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a, Position: Debug> Debug for Token<'a, Position>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a, Position> Display for Token<'a, Position>

Panics

This Display implementation panics when called on Token::Error.

It also panics when used on Invalid…WithCarriageReturn tokens that cannot be reparsed as such, for example by containing improper escape sequences.

fn fmt(&self, f: &mut Formatter<'_>) -> fmtResult

Formats the value using the given formatter.

impl<'a> IntoToken<'a, ()> for Token<'a, ()>

fn into_token(self) -> Token<'a, ()>

impl<'s> Logos<'s> for Token<'s, usize>

const ERROR: Self = Token::Error

Helper const of the variant marked as #[error].

type Extras = ()

Associated type Extras for the particular lexer. This can be set using #[logos(extras = MyExtras)] and accessed inside callbacks.

type Source = str

Source type this token can be lexed from. This will default to str, unless one of the defined patterns explicitly uses non-unicode byte values or byte slices, in which case that implementation will use [u8].

fn lex(lex: &mut Lexer<'s, Self>)

The heart of Logos. Called by the Lexer. The implementation for this function is generated by the logos-derive crate.

fn lexer(source: &'source Self::Source) -> Lexer<'source, Self>

where Self::Extras: Default,

Create a new instance of a Lexer that will produce tokens implementing this Logos.

fn lexer_with_extras( source: &'source Self::Source, extras: Self::Extras, ) -> Lexer<'source, Self>

Create a new instance of a Lexer with the provided Extras that will produce tokens implementing this Logos.

impl<'a, Position: PartialEq> PartialEq for Token<'a, Position>

fn eq(&self, other: &Token<'a, Position>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'a, Position: Eq> Eq for Token<'a, Position>

impl<'a, Position> StructuralPartialEq for Token<'a, Position>