Module mcvm_parse::lex

source ·
Expand description

Token generation from a string, which is passed into the parser

Structs§

  • TextPos
    Text positional information with row, column, and absolute index

Enums§

  • Side
    Generic side for something like a bracket
  • Token
    A token that we derive from text

Functions§

  • lex
    Create a list of tokens from package text contents that we will then use for parsing
  • reduce_tokens
    Removes whitespace characters and comments from an iterator of tokens

Type Aliases§