Struct full_moon::tokenizer::Token

pub struct Token<'a> { /* fields omitted */ }

A token such consisting of its Position and a TokenType

Methods

impl<'a> Token<'a>

pub fn start_position(&self) -> Position

The position a token begins at

pub fn end_position(&self) -> Position

The position a token ends at

pub fn token_type(&self) -> AtomicRef<TokenType<'a>>

The type of token as well as the data needed to represent it If you don't need any other information, use token_kind instead.

pub fn token_kind(&self) -> TokenKind

The kind of token with no additional data. If you need any information such as idenitfier names, use token_type instead.

Trait Implementations

impl<'a, '_> Borrow<Token<'a>> for &'_ TokenReference<'a>

fn borrow(&self) -> &Token<'a>

Immutably borrows from an owned value.

impl<'a> Clone for Token<'a>

fn clone(&self) -> Token<'a>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for Token<'a>

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<'de: 'a, 'a> Deserialize<'de> for Token<'a>

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl<'a> Display for Token<'a>

fn fmt(&self, formatter: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<'a> Eq for Token<'a>

impl<'a> Node for Token<'a>

fn start_position(&self) -> Option<Position>

The start position of a node. None if can't be determined

fn end_position(&self) -> Option<Position>

The end position of a node. None if it can't be determined

fn similar(&self, other: &Self) -> bool

Whether another node of the same type is the same as this one semantically, ignoring position

fn range(&self) -> Option<(Position, Position)>

The full range of a node, if it has both start and end positions

fn surrounding_ignore_tokens<'ast, 'b>(
    &self,
    ast: &'b Ast<'ast>
) -> Option<(Vec<&'b Token<'ast>>, Vec<&'b Token<'ast>>)>

The tokens surrounding a node that are ignored and not accessible through the node's own accessors. Use this if you want to get surrounding comments or whitespace. Return value is None if a token doesn't have both a start and end position. Otherwise, it is a tuple of two token vectors, first being the preceding and the second being the following.

impl<'a> Ord for Token<'a>

fn cmp(&self, other: &Self) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

🔬 This is a nightly-only experimental API. (clamp)

Restrict a value to a certain interval.

impl<'_> Owned for Token<'_>

type Owned = Token<'static>

What an owned version of the object looks like. Usually contains a 'static lifetime.

fn owned(&self) -> Self::Owned

Returns an owned version of the object.

impl<'a> PartialEq<Token<'a>> for Token<'a>

fn eq(&self, rhs: &Self) -> bool

This method tests for self and other values to be equal, and is used by ==.

#[must_use] fn ne(&self, other: &Rhs) -> bool

This method tests for !=.

impl<'a> PartialOrd<Token<'a>> for Token<'a>

fn partial_cmp(&self, other: &Self) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

#[must_use] fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

#[must_use] fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

#[must_use] fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

#[must_use] fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl<'a> Serialize for Token<'a>

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.