Struct lrpar::Lexeme

pub struct Lexeme<StorageT> { /* fields omitted */ }

A Lexeme represents a segment of the user's input that conforms to a known type. Note that even if the type of a lexeme seemingly requires it to have len() > 0 (e.g. integers might match the regular expressions [0-9]+), error recovery might cause a lexeme to have a length of 0. Users can detect the difference between a lexeme with an intentionally zero length from a lexeme with zero length due to error recovery through the inserted method.

Methods

impl<StorageT: Copy> Lexeme<StorageT>

pub fn new(tok_id: StorageT, start: usize, len: Option<usize>) -> Self

Create a new token with ID tok_id and a starting position in the input start. If the token is the result of user input, then Some(n) should be passed to len; if the token is the result of error recovery, then None should be passed to len.

pub fn tok_id(&self) -> StorageT

The token ID.

pub fn start(&self) -> usize

Byte offset of the start of the lexeme

pub fn end(&self) -> usize

Byte offset of the end of the lexeme.

Note that if this lexeme was inserted by error recovery, it will end at the same place it started (i.e. self.start() == self.end()).

pub fn len(&self) -> usize

Length in bytes of the lexeme.

Note that if this lexeme was inserted by error recovery, it will have a length of 0.

pub fn inserted(&self) -> bool

Returns true if this lexeme was inserted as the result of error recovery, or false otherwise.

Trait Implementations

impl<StorageT: Clone> Clone for Lexeme<StorageT>

fn clone(&self) -> Lexeme<StorageT>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<StorageT: Copy> Copy for Lexeme<StorageT>

impl<StorageT: Debug> Debug for Lexeme<StorageT>

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<StorageT: Eq> Eq for Lexeme<StorageT>

impl<StorageT: Hash> Hash for Lexeme<StorageT>

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given [Hasher].

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given [Hasher].

impl<StorageT: PartialEq> PartialEq<Lexeme<StorageT>> for Lexeme<StorageT>

fn eq(&self, other: &Lexeme<StorageT>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Lexeme<StorageT>) -> bool

This method tests for !=.

impl<StorageT> StructuralEq for Lexeme<StorageT>

impl<StorageT> StructuralPartialEq for Lexeme<StorageT>