Trait Lexer

pub trait Lexer<'a> {
    type Token: Copy + Eq;
    type Position: Copy + Eq + Ord + Hash;

    // Required methods
    fn new(input: &'a str) -> Self;
    fn pos(&self) -> Self::Position;
    fn next(&self) -> (Option<Self::Token>, usize);
    fn is_empty(&self) -> bool;
    fn advance_to_pos(&self, pos: Self::Position);
    fn fork(&self) -> Self;
}

A lexer for the parser.

Required Associated Types

type Token: Copy + Eq

The lexed token type.

type Position: Copy + Eq + Ord + Hash

The position type.

Required Methods

fn new(input: &'a str) -> Self

Create a new lexer from the given input.

fn pos(&self) -> Self::Position

Get the current parsing position.

fn next(&self) -> (Option<Self::Token>, usize)

Consume the next token.

fn is_empty(&self) -> bool

Whether the lexer is at the end of the input.

fn advance_to_pos(&self, pos: Self::Position)

Advance the lexer to the given position.

fn fork(&self) -> Self

Fork the lexer.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl<'a> Lexer<'a> for CharLexer<'a>

type Token = char

type Position = usize