Struct lrpar::Lexeme

pub struct Lexeme<StorageT> { /* fields omitted */ }

A Lexeme represents a segment of the user's input that conforms to a known type. Note that even if the type of a lexeme seemingly requires it to have len() > 0 (e.g. integers might match the regular expressions [0-9]+), error recovery might cause a lexeme to have a length of 0. Users can detect the difference between a lexeme with an intentionally zero length from a lexeme with zero length due to error recovery through the inserted method.

Methods

impl<StorageT: Copy> Lexeme<StorageT>

pub fn new(tok_id: StorageT, start: usize, len: Option<usize>) -> Self

Create a new token with ID tok_id and a starting position in the input start. If the token is the result of user input, then Some(n) should be passed to len; if the token is the result of error recovery, then None should be passed to len.

pub fn tok_id(&self) -> StorageT

The token ID.

pub fn start(&self) -> usize

Deprecated since 0.6.1:

Please use span().start() instead

Byte offset of the start of the lexeme

pub fn end(&self) -> usize

Deprecated since 0.6.1:

Please use span().end() instead

Byte offset of the end of the lexeme.

Note that if this lexeme was inserted by error recovery, it will end at the same place it started (i.e. self.start() == self.end()).

pub fn len(&self) -> usize

Deprecated since 0.6.1:

Please use span().len() instead

Length in bytes of the lexeme.

Note that if this lexeme was inserted by error recovery, it will have a length of 0.

pub fn span(&self) -> Span

Obtain this Lexeme's Span.

pub fn inserted(&self) -> bool

Returns true if this lexeme was inserted as the result of error recovery, or false otherwise.

Trait Implementations

impl<StorageT: Clone> Clone for Lexeme<StorageT>

fn clone(&self) -> Lexeme<StorageT>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<StorageT: Copy> Copy for Lexeme<StorageT>

impl<StorageT: Debug> Debug for Lexeme<StorageT>

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<StorageT: Eq> Eq for Lexeme<StorageT>

impl<StorageT: Hash> Hash for Lexeme<StorageT>

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given [Hasher].

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given [Hasher].

impl<StorageT: PartialEq> PartialEq<Lexeme<StorageT>> for Lexeme<StorageT>

fn eq(&self, other: &Lexeme<StorageT>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Lexeme<StorageT>) -> bool

This method tests for !=.

impl<StorageT> StructuralEq for Lexeme<StorageT>

impl<StorageT> StructuralPartialEq for Lexeme<StorageT>