Struct lrpar::Lexeme

pub struct Lexeme<StorageT> { /* fields omitted */ }

A Lexeme represents a segment of the user’s input that conforms to a known type. Note that even if the type of a lexeme seemingly requires it to have len() > 0 (e.g. integers might match the regular expressions [0-9]+), error recovery might cause a lexeme to have a length of 0. Users can detect the difference between a lexeme with an intentionally zero length from a lexeme with zero length due to error recovery through the inserted method.

Implementations

impl<StorageT: Copy> Lexeme<StorageT>

pub fn new(tok_id: StorageT, start: usize, len: Option<usize>) -> Self

Create a new token with ID tok_id and a starting position in the input start. If the token is the result of user input, then Some(n) should be passed to len; if the token is the result of error recovery, then None should be passed to len.

pub fn tok_id(&self) -> StorageT

The token ID.

pub fn start(&self) -> usize

👎 Deprecated since 0.6.1:

Please use span().start() instead

Byte offset of the start of the lexeme

pub fn end(&self) -> usize

👎 Deprecated since 0.6.1:

Please use span().end() instead

Byte offset of the end of the lexeme.

Note that if this lexeme was inserted by error recovery, it will end at the same place it started (i.e. self.start() == self.end()).

pub fn len(&self) -> usize

👎 Deprecated since 0.6.1:

Please use span().len() instead

Length in bytes of the lexeme.

Note that if this lexeme was inserted by error recovery, it will have a length of 0.

pub fn span(&self) -> Span

Obtain this Lexeme’s Span.

pub fn inserted(&self) -> bool

Returns true if this lexeme was inserted as the result of error recovery, or false otherwise.

Trait Implementations

impl<StorageT: Clone> Clone for Lexeme<StorageT>

fn clone(&self) -> Lexeme<StorageT>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<StorageT: Debug> Debug for Lexeme<StorageT>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<StorageT: Copy> Display for Lexeme<StorageT>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<StorageT: Copy + Debug> Error for Lexeme<StorageT>

fn source(&self) -> Option<&(dyn Error + 'static)>

The lower-level source of this error, if any.

fn backtrace(&self) -> Option<&Backtrace>

🔬 This is a nightly-only experimental API. (backtrace)

Returns a stack backtrace, if available, of where this error occurred.

fn description(&self) -> &str

👎 Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎 Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

impl<StorageT: Hash> Hash for Lexeme<StorageT>

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl<StorageT: PartialEq> PartialEq<Lexeme<StorageT>> for Lexeme<StorageT>

fn eq(&self, other: &Lexeme<StorageT>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Lexeme<StorageT>) -> bool

This method tests for !=.

impl<StorageT: Copy> Copy for Lexeme<StorageT>

impl<StorageT: Eq> Eq for Lexeme<StorageT>

impl<StorageT> StructuralEq for Lexeme<StorageT>

impl<StorageT> StructuralPartialEq for Lexeme<StorageT>