Crate tokenizer_lib[−][src]
Modules
| StreamedTokenChannel |
Macros
| expect_next |
Structs
| StaticTokenChannel | A synchronous “channel” which can be used as a sender and reader. Will
buffer all tokens into a |
| Token | Defines a Token of type T and with a Position |
Traits
| TokenReader | Trait for a reader which returns tokens over a current sequence |
| TokenSender | Trait for a sender that can append a token to a sequence |