Module token_factory 

How Lexer should produce tokens

Structs

ArenaFactory

This is a wrapper for Token factory that allows to allocate tokens in separate arena. It can allow to significantly improve performance by passing Tokens by references everywhere.

CommonTokenFactory

Default token factory

OwningTokenFactory

Token factory that produces heap allocated OwningTokens

Traits

TokenFactory

Trait for creating tokens.

Type Aliases

ArenaCommonFactory

Arena token factory that contains CommonTokens

ArenaOwningFactory

Arena token factory that contains OwningTokens