Struct syntax::tokenstream::TokenStream

pub struct TokenStream(pub Option<Lrc<Vec<TreeAndJoint>>>);

Token Streams

A TokenStream is an abstract sequence of tokens, organized into TokenTrees. The goal is for procedural macros to work with TokenStreams and TokenTrees instead of a representation of the abstract syntax tree. Today's TokenTrees can still contain AST via token::Interpolated for back-compat.

The use of Option is an optimization that avoids the need for an allocation when the stream is empty. However, it is not guaranteed that an empty stream is represented with None; it may be represented as a Some around an empty Vec.

Methods

impl TokenStream

pub fn len(&self) -> usize

pub fn empty() -> TokenStream

pub fn is_empty(&self) -> bool

pub fn new(streams: Vec<TreeAndJoint>) -> TokenStream

pub fn append_to_tree_and_joint_vec(self, vec: &mut Vec<TreeAndJoint>)

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn trees(&self) -> Cursor

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn into_trees(self) -> Cursor

pub fn eq_unspanned(&self, other: &TokenStream) -> bool

Compares two TokenStreams, checking equality without regarding span information.

pub fn probably_equal_for_proc_macro(&self, other: &TokenStream) -> bool

pub fn map_enumerated<F: FnMut(usize, TokenTree) -> TokenTree>(
    self,
    f: F
) -> TokenStream

pub fn map<F: FnMut(TokenTree) -> TokenTree>(self, f: F) -> TokenStream

Trait Implementations

impl Eq for TokenStream

impl PartialEq<TokenStream> for TokenStream

fn eq(&self, other: &TokenStream) -> bool

This method tests for self and other values to be equal, and is used by ==.

#[must_use]

fn ne(&self, other: &Rhs) -> bool

This method tests for !=.

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl From<TokenTree> for TokenStream

fn from(tree: TokenTree) -> TokenStream

Performs the conversion.

impl Debug for TokenStream

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl Display for TokenStream

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<T: Into<TokenStream>> FromIterator<T> for TokenStream

fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self

Creates a value from an iterator.

impl Encodable for TokenStream

fn encode<E: Encoder>(&self, encoder: &mut E) -> Result<(), E::Error>

impl Decodable for TokenStream

fn decode<D: Decoder>(decoder: &mut D) -> Result<TokenStream, D::Error>