Expand description
This module takes care of lexing Python source text.
This means source code is scanned and translated into separate tokens. The rules governing what is and is not a valid token are defined in the Python reference guide section on Lexical analysis.
Structsยง
- Lexer
- A lexer for Python source code.