Struct TokenStream 

pub struct TokenStream { /* private fields */ }

An abstract stream of tokens, or more concretely a sequence of token trees.

This type provides interfaces for iterating over token trees and for collecting token trees into one stream.

Token stream is both the input and output of #[proc_macro], #[proc_macro_attribute] and #[proc_macro_derive] definitions.

Implementations

impl TokenStream

pub fn new() -> TokenStream

Returns an empty TokenStream containing no token trees.

pub fn is_empty(&self) -> bool

Checks if this TokenStream is empty.

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

Prints token in a form convenient for debugging.

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for TokenStream

TokenStream::default() returns an empty stream, i.e. this is equivalent with TokenStream::new().

fn default() -> TokenStream

Returns the “default value” for a type.

impl Display for TokenStream

Prints the token stream as a string that is supposed to be losslessly convertible back into the same token stream (modulo spans), except for possibly TokenTree::Groups with Delimiter::None delimiters and negative numeric literals.

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Extend<TokenStream> for TokenStream

fn extend<I>(&mut self, streams: I)

where I: IntoIterator<Item = TokenStream>,

Extends a collection with the contents of an iterator.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one)

Reserves capacity in a collection for the given number of additional elements.

impl Extend<TokenTree> for TokenStream

fn extend<I>(&mut self, streams: I)

where I: IntoIterator<Item = TokenTree>,

Extends a collection with the contents of an iterator.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one)

Reserves capacity in a collection for the given number of additional elements.

impl From<TokenStream> for TokenStream

fn from(inner: TokenStream) -> TokenStream

Converts to this type from the input type.

impl From<TokenTree> for TokenStream

fn from(token: TokenTree) -> TokenStream

Converts to this type from the input type.

impl FromIterator<TokenStream> for TokenStream

fn from_iter<I>(streams: I) -> TokenStream

where I: IntoIterator<Item = TokenStream>,

Creates a value from an iterator.

impl FromIterator<TokenTree> for TokenStream

Collects a number of token trees into a single stream.

fn from_iter<I>(streams: I) -> TokenStream

where I: IntoIterator<Item = TokenTree>,

Creates a value from an iterator.

impl FromStr for TokenStream

Attempts to break the string into tokens and parse those tokens into a token stream.

May fail for a number of reasons, for example, if the string contains unbalanced delimiters or characters not existing in the language.

NOTE: Some errors may cause panics instead of returning LexError. We reserve the right to change these errors into LexErrors later.

type Err = LexError

The associated error which can be returned from parsing.

fn from_str(src: &str) -> Result<TokenStream, LexError>

Parses a string s to return a value of this type.

impl IntoIterator for TokenStream

type Item = TokenTree

The type of the elements being iterated over.

type IntoIter = IntoIter

Which kind of iterator are we turning this into?

fn into_iter(self) -> IntoIter

Creates an iterator from a value.

impl Parser for TokenStream

Parses a TokenStream from the input tokens. This is the primary entity to parse when dealing with opaque entities where internal details are left out. Note that this matches a empty stream (see EndOfStream) as well.

fn parser( tokens: &mut ShadowCountedIter<'_, <TokenStream as IntoIterator>::IntoIter>, ) -> Result<TokenStream, Error>

The actual parsing function that must be implemented. This mutates the tokens iterator directly. It should not be called from user code except for implementing parsers itself and then only when the rules below are followed.

impl ToTokens for TokenStream

fn to_tokens(&self, tokens: &mut TokenStream)

Write &self to the given TokenStream.

fn to_token_iter( &self, ) -> ShadowCountedIter<'_, <TokenStream as IntoIterator>::IntoIter>

Convert &self into a TokenIter object.

fn into_token_iter<'a>( self, ) -> ShadowCountedIter<'a, <TokenStream as IntoIterator>::IntoIter>

where Self: Sized,

Convert self into a TokenIter object.

fn to_token_stream(&self) -> TokenStream

Convert &self into a TokenStream object.

fn into_token_stream(self) -> TokenStream

where Self: Sized,

Convert self into a TokenStream object.

fn tokens_to_string(&self) -> String

Convert &self into a String object. This is mostly used in the test suite to compare the outputs. When the input is a &str then this parses it and returns a normalized String.

impl TryFrom<TokenStream> for NonEmptyTokenStream

type Error = Error

The type returned in the event of a conversion error.

fn try_from(value: TokenStream) -> Result<NonEmptyTokenStream, Error>

Performs the conversion.