Struct Token 

pub struct Token { /* private fields */ }

Encapsulates the information contained in the current token. We only ever create one of these, but it is cleaner to keep the data in a separate class. Port of Token.

Implementations

impl Token

pub fn new(source: SourceId) -> Token

A fresh none token with an empty range in source.

pub fn kind(&self) -> TokenKind

\return the kind of this token.

pub fn is_res_word(&self) -> bool

\return true if this token is a reserved word.

pub fn is_template_literal(&self) -> bool

\return true if this token is one of the four template-literal kinds.

pub fn start_loc(&self) -> SMLoc

\return the location of the first byte of the token.

pub fn end_loc(&self) -> SMLoc

\return the location one past the last byte of the token.

pub fn source_range(&self) -> SMRange

\return the half-open source range covered by the token.

pub fn get_numeric_literal(&self) -> f64

\return the value of a numeric_literal token.

pub fn get_identifier(&self) -> AtomBytes

\return the interned name of an identifier token.

pub fn get_private_identifier(&self) -> AtomBytes

\return the interned name of a private_identifier token, without the leading # (which is still part of the token’s source range).

pub fn get_res_word_identifier(&self) -> AtomBytes

\return the interned spelling of a reserved-word token.

pub fn get_res_word_or_identifier(&self) -> AtomBytes

\return the interned spelling of an identifier or reserved word, for the many places where the grammar accepts either.

pub fn get_string_literal(&self) -> AtomBytes

\return the cooked value of a string_literal token, with escapes and line continuations already processed.

pub fn get_string_literal_raw_value(&self) -> AtomBytes

\return the raw (undecoded) text of a JSX string literal. Only set by Token::set_jsx_string_literal; the normal string path leaves it unset.

pub fn get_string_literal_contains_escapes(&self) -> bool

\return whether a string_literal contained any escape or line continuation. The parser uses this to build a directive’s raw text: with no escapes the raw equals the cooked value, otherwise it must re-slice the source between the quotes.

pub fn get_template_literal_contains_not_escapes(&self) -> bool

\return whether the template literal token contains a NotEscapeSequence.

pub fn get_template_value(&self) -> Option<AtomBytes>

\return the cooked value of a template-literal token, or None if it contains a NotEscapeSequence (legal only in a tagged template).

pub fn get_template_raw_value(&self) -> AtomBytes

\return the Template Raw Value (TRV) of a template-literal token.

pub fn get_bigint_literal(&self) -> AtomBytes

\return the value of a bigint_literal token as text, never converted to a number: the literal without its trailing n and with numeric separators removed, but with any radix prefix kept (0xF_Fn -> 0xFF). This is the ESTree bigint property.

pub fn get_bigint_literal_raw_value(&self) -> AtomBytes

\return the raw source text of a bigint_literal token, including any radix prefix and the trailing n.

pub fn get_regexp_literal(&self) -> RegExpLiteral

\return the body and flags of a regexp_literal token.

pub fn get_jsx_text_value(&self) -> AtomBytes

\return the value of a jsx_text token, with HTML entities decoded.

pub fn get_jsx_text_raw(&self) -> AtomBytes

\return the raw source text of a jsx_text token, with HTML entities left as written.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.