Module ucglib::tokenizer [−] [src]

The tokenization stage of the ucg compiler.

Structs

TokenIter wraps a slice of Tokens and implements the various necessary nom traits to use it as an input to nom parsers.

Functions

pos	pos gets the current position from a TokenIter input without consuming it.
token_clone
tokenize	Consumes an input Span and returns either a Vec or a nom::ErrorKind.

Type Definitions

Keyboard Shortcuts

?: Show this help dialog
S: Focus the search field
↑: Move up in search results
↓: Move down in search results
↹: Switch tab
⏎: Go to active search result
+: Expand all sections
-: Collapse all sections

Search Tricks

Prefix searches with a type followed by a colon (e.g. fn:) to restrict the search to a given type.

Accepted types are: fn, mod, struct, enum, trait, type, macro, and const.

Search functions by type signature (e.g. vec -> usize or * -> vec)