Module lexer

Lexer for Jingo. See scan_code for main lexing capabilities.

Structs

Token

Main token representation, combining TokenType with metadata.

Enums

TokenType

The token type, represents the type of a Token after scanning.

Functions

keyword_match

Matches keywords like and to TokenType::And for easy keyword recognition outside of this lexer.

scan_code

Lexes code into Vec<Token> or provides an error in the form of JingoError.

scan_next_token

Gets next token in line of peekable chars (as this lexer is lookahead, it needs chars.peekable() enabled).