Module fundamental 

This module contains the fundamental parsers. These are the basic tokens from proc_macro2/proc_macro and a few other ones defined by unsynn. These are the terminal entities when parsing tokens. Being able to parse TokenTree and TokenStream allows one to parse opaque entities where internal details are left out. The Cached type is used to cache the string representation of the parsed entity. The Nothing type is used to match without consuming any tokens. The Except type is used to match when the next token does not match the given type. The EndOfStream type is used to match the end of the stream when no tokens are left. The HiddenState type is used to hold additional information that is not part of the parsed syntax.

Note: When the proc_macro2 feature is disabled, format macros (format_ident!, format_literal!) are unavailable, but Cached<T> remains fully functional using .to_string() from proc_macro types.

Structs

Cached

Getting the underlying string expensive as it always allocates a new String. This type caches the string representation of a given entity. Note that this is only reliable for fundamental entities that represent a single token. Spacing between composed tokens is not stable and should be considered informal only.

EndOfStream

Matches the end of the stream when no tokens are left.

Except

Succeeds when the next token does not match T. Will not consume any tokens. Usually this has to be followed with a conjunctive match such as Cons<Except<T>, U> or followed by another entry in a struct or tuple.

Expect

Succeeds when the next token would match T. Will not consume any tokens. This is similar to peeking.

Group

A delimited token stream.

HiddenState

Sometimes one want to compose types or create structures for unsynn that have members that are not part of the parsed syntax but add some additional information. This struct can be used to hold such members while still using the Parser and ToTokens trait implementations automatically generated by the [unsynn!{}] macro or composition syntax. HiddenState will not consume any tokens when parsing and will not emit any tokens when generating a TokenStream. On parsing it is initialized with a default value. It has Deref and DerefMut implemented to access the inner value.

Ident

A word of Rust code, which may be a keyword or legal variable name.

Invalid

A unit that always fails to match. This is useful as default for generics. See how Either<A, B, C, D> uses this for unused alternatives.

Literal

A literal string ("hello"), byte string (b"hello"), character ('a'), byte character (b'a'), an integer or floating point number with or without a suffix (1, 1u8, 2.3, 2.3f32).

NonEmptyTokenStream

Since parsing a TokenStream succeeds even when no tokens are left, this type is used to parse a TokenStream that is not empty.

NonParseable

A unit that can not be parsed. This is useful as diagnostic placeholder for parsers that are (yet) unimplemented. The nonparseable feature flag controls if Parser and ToTokens will be implemented for it. This is useful in release builds that should not have any NonParseable left behind.

Nothing

A unit that always matches without consuming any tokens. This is required when one wants to parse a Repeats without a delimiter. Note that using Nothing as primary entity in a Vec, LazyVec, DelimitedVec or Repeats will result in an infinite loop.

Punct

A Punct is a single punctuation character like +, - or #.

TokenStream

An abstract stream of tokens, or more concretely a sequence of token trees.

Enums

TokenTree

A single token or a delimited sequence of token trees (e.g. [1, (), ..]).

Type Aliases

CachedGroup

Group with cached string representation.

CachedIdent

Ident with cached string representation.

CachedLiteral

Literal with cached string representation.

CachedLiteralInteger

LiteralInteger with cached string representation.

CachedLiteralString

LiteralString with cached string representation.

CachedPunct

Punct with cached string representation.

CachedTokenTree

TokenTree (any token) with cached string representation.