Crate lexical_scanner
source · [−]Modules
Functions
Constructs a vector of tokens. This straight forward lexical scanner is preset to support over 75 tokens. The list of tokens can be found at this sites github page. Usually, this is the main method for generating tokens by passing in a file path to the document you want to perform a lexical scan on. Example
Constructs a vector of tokens. This straight forward lexical scanner is preset to support over 75 tokens. The list of tokens can be found at this sites github page. Usually, passing in a text string is used for debugging and testing. Example