Module lexer 

This module takes care of lexing Python source text.

This means source code is scanned and translated into separate tokens. The rules governing what is and is not a valid token are defined in the Python reference guide section on Lexical analysis.

Structs

Lexer

A lexer for Python source code.

Functions

lex

Create a new Lexer for the given source code and Mode.