Struct TokenStream 

pub struct TokenStream(/* private fields */);

An owned sequence of raw M2 token trees.

Implementations

impl TokenStream

pub fn new() -> Self

Creates an empty token stream.

pub fn iter( &self, ) -> impl DoubleEndedIterator<Item = &TokenTree> + ExactSizeIterator

Iterates over the stream’s top-level token trees.

pub fn is_empty(&self) -> bool

Reports whether the stream contains no token trees.

pub fn pop(&mut self) -> Option<TokenTree>

Removes and returns the final token tree, if any.

pub fn to_source(&self) -> String

Migrates source text from the shared sources retained by its spans.

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenStream

fn default() -> TokenStream

Returns the “default value” for a type.

impl Display for TokenStream

fn fmt(&self, formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenStream

impl<T: Lex> Extend<T> for TokenStream

Appends anything that can emit tokens, one item at a time.

Lex already states which types produce tokens; this states how a TokenStream accepts them, by forwarding to Lex::to_tokens. It subsumes the two concrete impls this replaced (TokenTree and TokenStream itself both implement Lex), and works for any other Lex type without a bespoke impl.

fn extend<I: IntoIterator<Item = T>>(&mut self, iter: I)

Extends a collection with the contents of an iterator.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one)

Reserves capacity in a collection for the given number of additional elements.

impl FromIterator<TokenStream> for CellStream

fn from_iter<T: IntoIterator<Item = TokenStream>>(iter: T) -> Self

Creates a value from an iterator.

impl FromIterator<TokenTree> for TokenStream

fn from_iter<T: IntoIterator<Item = TokenTree>>(iter: T) -> Self

Creates a value from an iterator.

impl IntoIterator for TokenStream

type Item = TokenTree

The type of the elements being iterated over.

type IntoIter = IntoIter<TokenTree>

Which kind of iterator are we turning this into?

fn into_iter(self) -> Self::IntoIter

Creates an iterator from a value.

impl Lex for TokenStream

fn to_tokens(&self, output: &mut TokenStream)

Appends this value’s token trees to output.

fn to_token_stream(&self) -> TokenStream

Emits this value into a fresh token stream.

fn to_m2(&self) -> String

Renders this value as normalized Macaulay2 source.

impl Parse for TokenStream

fn parse(input: &mut ParseStream) -> Result<Self, TokenParseError>

Consumes this value from input.

impl PartialEq for TokenStream

fn eq(&self, other: &TokenStream) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Spanned for TokenStream

fn try_span(&self) -> Option<Span>

Returns the smallest span covering this value, when its constituents share one source snapshot.

fn span(&self) -> Span

Returns the smallest span covering this value.

fn expect_span(&self) -> Span

Returns this value’s span when one source snapshot covers it.

impl StructuralPartialEq for TokenStream

impl Tokenize for TokenStream

fn tokenize(&self, output: &mut CellStream) -> Result<(), CellStreamError>

Appends this value’s independently parseable cells to output.

fn to_cell_stream(&self) -> Result<CellStream, CellStreamError>

Tokenizes this value into a fresh cell stream.