Struct Token 

pub struct Token { /* private fields */ }

Represents a token in the source code with its position and associated comments or whitespaces.

Implementations

impl Token

pub fn new_with_line(start: usize, end: usize, line_number: usize) -> Self

Creates a token where the position refers to the original code where the token was parsed with the line number where it starts.

pub fn from_content<IntoCowStr: Into<Cow<'static, str>>>( content: IntoCowStr, ) -> Self

Creates a new token that is not constrained to any existing position.

pub fn from_position(position: Position) -> Self

Creates a new token from a position.

pub fn with_leading_trivia(self, trivia: Trivia) -> Self

Adds leading trivia to the token and returns the updated token.

pub fn with_trailing_trivia(self, trivia: Trivia) -> Self

Adds trailing trivia to the token and returns the updated token.

pub fn has_trivia(&self) -> bool

Returns whether the token has any trivia (leading or trailing).

pub fn push_leading_trivia(&mut self, trivia: Trivia)

Adds leading trivia to the token.

pub fn insert_leading_trivia(&mut self, index: usize, trivia: Trivia)

Inserts leading trivia at the given index.

pub fn push_trailing_trivia(&mut self, trivia: Trivia)

Adds trailing trivia to the token.

pub fn iter_leading_trivia(&self) -> impl Iterator<Item = &Trivia>

Returns an iterator over the leading trivia.

pub fn drain_leading_trivia(&mut self) -> impl Iterator<Item = Trivia> + '_

Returns an iterator over the leading trivia and removes them from the token.

pub fn iter_trailing_trivia(&self) -> impl Iterator<Item = &Trivia>

Returns an iterator over the trailing trivia.

pub fn drain_trailing_trivia(&mut self) -> impl Iterator<Item = Trivia> + '_

Returns an iterator over the trailing trivia and removes them from the token.

pub fn read<'a: 'b, 'b>(&'a self, code: &'b str) -> &'b str

Reads the content of the token from the source code.

Panics

Panics if the position is a line number reference and the range is invalid.

pub fn get_line_number(&self) -> Option<usize>

Returns the line number of the token, if available.

pub fn replace_with_content<IntoCowStr: Into<Cow<'static, str>>>( &mut self, content: IntoCowStr, )

Replaces the token’s content with new content while preserving line number information.

pub fn clear_comments(&mut self)

Clears all comments from the tokens in this node.

pub fn clear_whitespaces(&mut self)

Clears all whitespaces information from the tokens in this node.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Token

fn eq(&self, other: &Token) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for Token

impl StructuralPartialEq for Token